Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU

About

We introduce a hybrid CPU/GPU version of the Asynchronous Advantage Actor-Critic (A3C) algorithm, currently the state-of-the-art method in reinforcement learning for various gaming tasks. We analyze its computational traits and concentrate on aspects critical to leveraging the GPU's computational power. We introduce a system of queues and a dynamic scheduling strategy, potentially helpful for other asynchronous algorithms as well. Our hybrid CPU/GPU version of A3C, based on TensorFlow, achieves a significant speed up compared to a CPU implementation; we make it publicly available to other researchers at https://github.com/NVlabs/GA3C .

Mohammad Babaeizadeh, Iuri Frosio, Stephen Tyree, Jason Clemons, Jan Kautz · 2016

Related benchmarks

Task	Dataset	Result
Reinforcement Learning	Atari 2600 SpaceInvaders	Best Stable Score600	7
Reinforcement Learning	Atari 2600 Pong	Best Score18	4
Reinforcement Learning	Atari 2600 Breakout	Best Stable Score350	4
Reinforcement Learning	Atari 2600 Seaquest	Best Stable Score1706	4
Reinforcement Learning	Atari 2600 BeamRider	Best Stable Score3000	3
Reinforcement Learning	Atari 2600 Boxing	Best Stable Score92	2