# One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL

Source: [One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL](<https://www.sota2.com/research/paper/68fa68e21e209131cea1ef68>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:48:52.480Z

Authors: Saurabh Kumar, Aviral Kumar, Sergey Levine, Chelsea Finn

Published: 2020-10-27T17:41:57.000Z

## Abstract

While reinforcement learning algorithms can learn effective policies for complex tasks, these policies are often brittle to even minor task variations, especially when variations are not explicitly provided during training. One natural approach to this problem is to train agents with manually specified variation in the training task or environment. However, this may be infeasible in practical situations, either because making perturbations is not possible, or because it is unclear how to choose suitable perturbation strategies without sacrificing performance. The key insight of this work is that learning diverse behaviors for accomplishing a task can directly lead to behavior that generalizes to varying environments, without needing to perform explicit perturbations during training. By identifying multiple solutions for the task in a single environment during training, our approach can generalize to new situations by abandoning solutions that are no longer effective and adopting those that are. We theoretically characterize a robustness set of environments that arises from our algorithm and empirically find that our diversity-driven approach can extrapolate to various changes in the environment and task.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Multi-Agent Reinforcement Learning on SMAC 2c64zg](<https://www.sota2.com/research/sota/multi-agent-reinforcement-learning-on-smac-2c64zg.md>): Multi-Agent Reinforcement Learning; SMAC 2c64zg; Win Rate: 100.

- [State Entropy Estimation on GRF (3v1)](<https://www.sota2.com/research/sota/state-entropy-estimation-on-grf-3v1.md>): State Entropy Estimation; GRF 3v1; State Entropy: 0.011.

- [Robot Locomotion on Humanoid](<https://www.sota2.com/research/sota/robot-locomotion-on-humanoid.md>): Robot Locomotion; Humanoid; Cumulative Reward: 4253.

- [Multi-Agent Reinforcement Learning on GRF Corner (Win Rate and Ball Passing Routes)](<https://www.sota2.com/research/sota/multi-agent-reinforcement-learning-on-grf-corner-win-rate-and-ball-passing-routes.md>): Multi-Agent Reinforcement Learning; GRF Corner; Win Rate: 67.

- [Multi-Agent Reinforcement Learning on GRF 3v1 hard](<https://www.sota2.com/research/sota/multi-agent-reinforcement-learning-on-grf-3v1-hard.md>): Multi-Agent Reinforcement Learning; GRF 3v1 hard; Win Rate: 91.

- [Adaptability Evaluation on BipedalWalker friction variations (test)](<https://www.sota2.com/research/sota/adaptability-evaluation-on-bipedalwalker-friction-variations-test.md>): Adaptability Evaluation; BipedalWalker friction variations (test); AUC: 496.23.

- [Adaptability Evaluation on BipedalWalker mass variations (test)](<https://www.sota2.com/research/sota/adaptability-evaluation-on-bipedalwalker-mass-variations-test.md>): Adaptability Evaluation; BipedalWalker mass variations (test); AUC: 71.38.

- [Multi-Agent Reinforcement Learning on GRF CA](<https://www.sota2.com/research/sota/multi-agent-reinforcement-learning-on-grf-ca.md>): Multi-Agent Reinforcement Learning; GRF (CA); Win Rate: 45.

- [Quality-Diversity on Ant](<https://www.sota2.com/research/sota/quality-diversity-on-ant.md>): Quality Diversity; Ant; GT QD Score: 1.02.

- [Quality-Diversity on HalfCheetah](<https://www.sota2.com/research/sota/quality-diversity-on-halfcheetah.md>): Quality Diversity; Halfcheetah; GT QD Score: 1.61.

- [Quality-Diversity on Hopper](<https://www.sota2.com/research/sota/quality-diversity-on-hopper.md>): Quality Diversity; Hopper; GT QD Score: 0.97.

- [Quality-Diversity on Walker2d](<https://www.sota2.com/research/sota/quality-diversity-on-walker2d.md>): Quality Diversity; Walker2D; GT QD Score: 1.17.

- [Quality-Diversity on BipedalWalker](<https://www.sota2.com/research/sota/quality-diversity-on-bipedalwalker.md>): Quality Diversity; BipedalWalker; GT QD Score: 0.14.

- [Multi-Agent Reinforcement Learning on SMAC 2c\_vs\_64zg](<https://www.sota2.com/research/sota/multi-agent-reinforcement-learning-on-smac-2c_vs_64zg.md>): Multi-Agent Reinforcement Learning; SMAC 2c\_vs\_64zg; State Entropy: 0.042.

- [Quality-Diversity on Swimmer](<https://www.sota2.com/research/sota/quality-diversity-on-swimmer.md>): Quality Diversity; Swimmer; GT QD Score: 0.02.

- [Strategy Discovery on GRF (CA)](<https://www.sota2.com/research/sota/strategy-discovery-on-grf-ca.md>): Strategy Discovery; GRF (CA); Distinct Strategies: 1.3.

- [Strategy Discovery on GRF (corner)](<https://www.sota2.com/research/sota/strategy-discovery-on-grf-corner.md>): Strategy Discovery; GRF Corner; Distinct Strategies: 1.

- [Multi-Agent Reinforcement Learning on SMAC 2m1z](<https://www.sota2.com/research/sota/multi-agent-reinforcement-learning-on-smac-2m1z.md>): Multi-Agent Reinforcement Learning; SMAC 2m1z; State Entropy: 0.028.

- [Strategy Discovery on GRF 3v1](<https://www.sota2.com/research/sota/strategy-discovery-on-grf-3v1.md>): Strategy Discovery; GRF 3v1; Distinct Strategies: 1.3.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2010.14484>)

[PDF](<https://arxiv.org/pdf/2010.14484.pdf>)
