# A Survey of Zero-shot Generalisation in Deep Reinforcement Learning Source: [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning]() Collection index: [llms.txt]() Data updated: 2025-10-26T00:56:59.119Z Authors: Robert Kirk, Amy Zhang, Edward Grefenstette, Tim Rockt\\"aschel Published: 2021-11-18T16:53:02.000Z ## Abstract The study of zero-shot generalisation (ZSG) in deep Reinforcement Learning (RL) aims to produce RL algorithms whose policies generalise well to novel unseen situations at deployment time, avoiding overfitting to their training environments. Tackling this is vital if we are to deploy reinforcement learning algorithms in real world scenarios, where the environment will be diverse, dynamic and unpredictable. This survey is an overview of this nascent field. We rely on a unifying formalism and terminology for discussing different ZSG problems, building upon previous works. We go on to categorise existing benchmarks for ZSG, as well as current methods for tackling these problems. Finally, we provide a critical discussion of the current state of the field, including recommendations for future work. Among other conclusions, we argue that taking a purely procedural content generation approach to benchmark design is not conducive to progress in ZSG, we suggest fast online adaptation and tackling RL-specific problems as some areas for future work on methods for ZSG, and we recommend building benchmarks in underexplored problem settings such as offline RL ZSG and reward-function variation. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Traffic Signal Control on Traffic Signal Road Length variation](): Traffic Signal Control; Traffic Signal Road Length variation; Normalized Reward: 0.9409. - [Advisory autonomy on Single lane ring (Acceleration guidance)](): Advisory autonomy; Advisory Autonomy Single lane ring (Acceleration guidance); Normalized Reward: 92.19. - [Advisory autonomy on Advisory Autonomy Single lane ring (Speed guidance)](): Advisory autonomy; Advisory Autonomy Single lane ring (Speed guidance); Normalized Reward: 0.9688. - [Advisory autonomy on Advisory Autonomy Highway ramp (Acceleration guidance)](): Advisory autonomy; Advisory Autonomy Highway ramp (Acceleration guidance); Normalized Reward: 0.5374. - [Advisory autonomy on Advisory Autonomy Highway ramp (Speed guidance)](): Advisory autonomy; Advisory Autonomy Highway ramp (Speed guidance); Normalized Reward: 54.73. - [Dynamic eco-driving on Eco-Driving Penetration Rate variation](): Dynamic eco-driving; Eco-Driving Penetration Rate variation; Normalized Reward: 0.526. - [Dynamic eco-driving on Eco-Driving Inflow variation](): Dynamic eco-driving; Eco-Driving Inflow variation; Normalized Reward: 0.4061. - [Dynamic eco-driving on Eco-Driving Green Phase variation](): Dynamic eco-driving; Eco-Driving Green Phase variation; Normalized Reward: 0.4228. - [Traffic Signal Control on Traffic Signal Inflow variation](): Traffic Signal Control; Traffic Signal Inflow variation; Normalized Reward: 0.8646. - [Traffic signal control on Traffic Signal Speed Limit variation](): Traffic Signal Control; Traffic Signal Speed Limit variation; Normalized Reward: 88.57. ## Original sources and implementation [Original paper]() [PDF]()