Vector-based Navigation using Grid-like Representations in Artificial Agents.

Alexander Pritzel
Andrea Banino
Benigno Uria
Brian C Zhang
Caswell Barry
Charles Blundell
Charlie Beattie
Demis Hassabis
Dharshan Kumaran
Greg Wayne
Helen King
Hubert Soyer
Joseph Modayil
Koray Kavukcuoglu
Martin J. Chadwick
Neil Rabinowitz
Raia Hadsell
Razvan Pascanu
Stephen Gaffney
Stig Vilholm Petersen
Thomas Degris
Timothy Lillicrap
Nature (2018)
Google Scholar

Abstract

Efficient navigation is a fundamental component of mammalian behaviour but remains challenging for artificial agents. Mammalian spatial behaviour is underpinned by grid cells in the entorhinal cortex, providing a multi-scale periodic representation that functions as a metric for coding space. Grid cells are viewed as critical for integrating self-motion (path integration) and planning direct trajectories to goals (vector-based navigation).We report, for the first time, that brain-like grid representations can emerge as the product of optimizing a recurrent network to perform the task of path integration - providing a normative perspective on the role of grid cells as a compact code for representing space. We show that grid cells provide an effective basis set to optimize the primary objective of navigation through deep reinforcement learning (RL) - the rapid discovery and exploitation of goals in complex, unfamiliar, and changeable environments. The performance of agents endowed with grid-like representations was found to surpass that of an expert human and comparison agents. Further, we demonstrate that grid-like representations enable agents to conduct shortcut behaviours reminiscent of those performed by mammals - with decoding analyses confirming that the metric quantities necessary for vector-based navigation (e.g. Euclidean distance and direction to goal) are represented within the network. Our findings show that emergent grid-like responses furnish agents with a Euclidean spatial metric and associated vector operations, providing a foundation for proficient navigation. As such, our results support neuroscientific theories that see grid cells as critical for path integration and vector-based navigation, demonstrating that the latter can be combined with path- based strategies to support navigation in complex environments

Research Areas