Sparse imitation learning for text based games with combinatorial action spaces

Chen Tessler
Tom Zahavy
Daniel J. Mankowitz
Shie Mannor
The Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM) (2019)

Abstract

We propose a computationally efficient algorithm that combines compressed sensing with imitation learning to solve text-based games with combinatorial action spaces. Specifically, we introduce a new compressed sensing algorithm, named IK-OMP, which can be seen as an extension to the Orthogonal Matching Pursuit (OMP). We incorporate IK-OMP into a supervised imitation learning setting and show that the combined approach (Sparse Imitation Learning, Sparse-IL) solves the entire text-based game of Zork1 with an action space of approximately 10 million actions given both perfect and noisy demonstrations.