Robust and Efficient Transfer Learning with Hidden-Parameter Markov Decision Processes

About

We introduce a new formulation of the Hidden Parameter Markov Decision Process (HiP-MDP), a framework for modeling families of related tasks using low-dimensional latent embeddings. Our new framework correctly models the joint uncertainty in the latent parameters and the state space. We also replace the original Gaussian Process-based model with a Bayesian Neural Network, enabling more scalable inference. Thus, we expand the scope of the HiP-MDP to applications with higher dimensions and more complex dynamics.

Taylor Killian, Samuel Daulton, George Konidaris, Finale Doshi-Velez · 2017

Related benchmarks

Task	Dataset	Result
Reinforcement Learning	Hopper-Bodyparts Start OpenAI Gym Mujoco (initialization)	Average Reward181.94	5
Reinforcement Learning	Cheetah-Gravity Start OpenAI Gym Mujoco (initialization)	Average Reward126.95	5
Reinforcement Learning	Cheetah-Gravity OpenAI Gym Mujoco (train)	Average Reward170.2	5
Reinforcement Learning	Cheetah-Bodyparts Start OpenAI Gym Mujoco (initialization)	Average Reward78.95	5
Reinforcement Learning	Cheetah-Bodyparts OpenAI Gym Mujoco (train)	Average Reward87.2	5
Reinforcement Learning	Walker-Gravity OpenAI Gym Mujoco (train)	Average Reward217.94	5
Reinforcement Learning	Walker-Bodyparts Start OpenAI Gym Mujoco (initialization)	Average Reward173.33	5
Reinforcement Learning	Walker-Bodyparts OpenAI Gym Mujoco (train)	Average Reward266.97	5
Reinforcement Learning	Hopper-Bodyparts OpenAI Gym Mujoco (train)	Average Reward227.5	5
Reinforcement Learning	Walker-Gravity Start OpenAI Gym Mujoco (initialization)	Average Reward213.93	5