Noisy Networks for Exploration

About

We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to aid efficient exploration. The parameters of the noise are learned with gradient descent along with the remaining network weights. NoisyNet is straightforward to implement and adds little computational overhead. We find that replacing the conventional exploration heuristics for A3C, DQN and dueling agents (entropy reward and $\epsilon$-greedy respectively) with NoisyNet yields substantially higher scores for a wide range of Atari games, in some cases advancing the agent from sub to super-human performance.

Meire Fortunato, Mohammad Gheshlaghi Azar, Bilal Piot, Jacob Menick, Ian Osband, Alex Graves, Vlad Mnih, Remi Munos, Demis Hassabis, Olivier Pietquin, Charles Blundell, Shane Legg · 2017

Related benchmarks

Task	Dataset	Result
Reinforcement Learning	Atari 2600 Montezuma's Revenge ALE (test)	Score3700	24
Reinforcement Learning	Atari 57	Atlantis923733	21
Reinforcement Learning	Atari 2600 57 games	Median Human-Normalized Score118	20
Reinforcement Learning	Atari 2600 Gravitar ALE (test)	Score2209	19
Reinforcement Learning	Atari-57 (test)	Median Human Norm Return172	15
Reinforcement Learning	Solaris	Final Mean Performance12380	5
Reinforcement Learning	PrivateEye	Final Mean Performance15806	5
Reinforcement Learning	Venture	Final Mean Performance1813	5
Reinforcement Learning	Pitfall!	Final Mean Performance0	5