Paper link: https://papers.nips.cc/paper/2014/hash/dfd7468ac613286cdbb40872c8ef3b06-Abstract.html

“We also propose that grid cells compute the eigendecomposition of place fields in part because is useful for segmenting an enclosure along natural boundaries. When applied recursively, this segmentation can be used to discover a hierarchical decomposition of space. Thus, grid cells might be involved in computing subgoals for hierarchical reinforcement learning.”

I learned about spectral graph analysis being used for RL to find subgoals. Interesting idea.

“Spectral analysis has frequently been invoked as a computational motivation for entorhinal grid cells (e.g., [23]). The fact that any function can be reconstructed by sums of sinusoids suggested that the entorhinal cortex implements a kind of Fourier transform of space, and that place cells are the result of reconstructing spatial signals from their spectral decomposition. Two problems face this interpretation. Fist, recent evidence suggests that the emergence of place cells does not depend on grid cell input [4, 47]. Second, and more importantly for our purposes, Fourier analysis is not the right mathematical tool when dealing with spatial representation in a topologically structured environment, since we do not expect functions to be smooth over boundaries in the environment. This is precisely the purpose of spectral graph theory: the eigenvectors of the graph Laplacian encode the smoothest approximation of a function that respects the graph topology [26].”

There are a bunch of interesting papers from it that I will read:

Identifying useful subgoals in reinforcement learning by local graph partitioning.

Grid cells, place cells, and geodesic generalization for spatial reinforcement learning.