Towards Generalization and Simplicity in Continuous Control

About

This work shows that policies with simple linear and RBF parameterizations can be trained to solve a variety of continuous control tasks, including the OpenAI gym benchmarks. The performance of these trained policies are competitive with state of the art results, obtained with more elaborate parameterizations such as fully connected neural networks. Furthermore, existing training and testing scenarios are shown to be very limited and prone to over-fitting, thus giving rise to only trajectory-centric policies. Training with a diverse initial state distribution is shown to produce more global policies with better generalization. This allows for interactive control scenarios where the system recovers from large on-line perturbations; as shown in the supplementary video.

Aravind Rajeswaran, Kendall Lowrey, Emanuel Todorov, Sham Kakade · 2017

Related benchmarks

Task	Dataset	Result
Reinforcement Learning	OpenAI Gym Hopper	Episode Count13920	3
Reinforcement Learning	OpenAI Gym Cheetah	Number of Episodes11250	3
Reinforcement Learning	OpenAI Gym Humanoid	Episodes96720	3
Reinforcement Learning	OpenAI Gym Swimmer v1	Mean Reward366	3
Reinforcement Learning	OpenAI Gym Hopper v1	Mean Reward3810	3
Reinforcement Learning	OpenAI Gym Cheetah v1	Mean Reward6620	3
Reinforcement Learning	OpenAI Gym Walker v1	Mean Reward5867	3
Reinforcement Learning	OpenAI Gym Humanoid v1	Mean Reward6849	3
Reinforcement Learning	OpenAI Gym Walker	Episode Count25680	3
Reinforcement Learning	OpenAI Gym Ant	Number of Episodes39240	3