# AutoQD: Automatic Discovery of Diverse Behaviors with Quality-Diversity Optimization

Source: [AutoQD: Automatic Discovery of Diverse Behaviors with Quality-Diversity Optimization](<https://www.sota2.com/research/paper/68fa792c1e209131cea7c639>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-06T05:00:05.533Z

Authors: Saeed Hedayatian, Stefanos Nikolaidis

Published: 2025-06-05T23:34:53.000Z

## Abstract

Quality-Diversity (QD) algorithms have shown remarkable success in discovering diverse, high-performing solutions, but rely heavily on hand-crafted behavioral descriptors that constrain exploration to predefined notions of diversity. Leveraging the equivalence between policies and occupancy measures, we present a theoretically grounded approach to automatically generate behavioral descriptors by embedding the occupancy measures of policies in Markov Decision Processes. Our method, AutoQD, leverages random Fourier features to approximate the Maximum Mean Discrepancy (MMD) between policy occupancy measures, creating embeddings whose distances reflect meaningful behavioral differences. A low-dimensional projection of these embeddings that captures the most behaviorally significant dimensions can then be used as behavioral descriptors for CMA-MAE, a state of the art blackbox QD method, to discover diverse policies. We prove that our embeddings converge to true MMD distances between occupancy measures as the number of sampled trajectories and embedding dimensions increase. Through experiments in multiple continuous control tasks we demonstrate AutoQD's ability in discovering diverse policies without predefined behavioral descriptors, presenting a well-motivated alternative to prior methods in unsupervised Reinforcement Learning and QD optimization. Our approach opens new possibilities for open-ended learning and automated behavior discovery in sequential decision making settings without requiring domain-specific knowledge. Source code is available at https://github.com/conflictednerd/autoqd-code.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Adaptability Evaluation on BipedalWalker friction variations (test)](<https://www.sota2.com/research/sota/adaptability-evaluation-on-bipedalwalker-friction-variations-test.md>): Adaptability Evaluation; BipedalWalker friction variations (test); AUC: 1429.66.

- [Adaptability Evaluation on BipedalWalker mass variations (test)](<https://www.sota2.com/research/sota/adaptability-evaluation-on-bipedalwalker-mass-variations-test.md>): Adaptability Evaluation; BipedalWalker mass variations (test); AUC: 295.65.

- [Quality-Diversity on Ant](<https://www.sota2.com/research/sota/quality-diversity-on-ant.md>): Quality Diversity; Ant; GT QD Score: 361.43.

- [Quality-Diversity on HalfCheetah](<https://www.sota2.com/research/sota/quality-diversity-on-halfcheetah.md>): Quality Diversity; Halfcheetah; GT QD Score: 30.78.

- [Quality-Diversity on Hopper](<https://www.sota2.com/research/sota/quality-diversity-on-hopper.md>): Quality Diversity; Hopper; GT QD Score: 1.84.

- [Quality-Diversity on Swimmer](<https://www.sota2.com/research/sota/quality-diversity-on-swimmer.md>): Quality Diversity; Swimmer; GT QD Score: 21.31.

- [Quality-Diversity on Walker2d](<https://www.sota2.com/research/sota/quality-diversity-on-walker2d.md>): Quality Diversity; Walker2D; GT QD Score: 18.36.

- [Quality-Diversity on BipedalWalker](<https://www.sota2.com/research/sota/quality-diversity-on-bipedalwalker.md>): Quality Diversity; BipedalWalker; GT QD Score: 6.09.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2506.05634>)

[PDF](<https://arxiv.org/pdf/2506.05634.pdf>)
