# The Two-Hump Problem: Bridging the Difficulty Gap in Mathematical Reinforcement Learning

Source: [The Two-Hump Problem: Bridging the Difficulty Gap in Mathematical Reinforcement Learning](<https://www.sota2.com/research/paper/6a3a134e206542251d7c72a8>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-24T05:00:13.022Z

Authors: Lucas Fagan, Michele Tarquini, Ali Shehper, Maksymilian Manko, Angus Gruen, Coco Huang, Giorgi Butbaia, Davide Passaro, Sergei Gukov

Published: 2026-06-19T17:14:47.000Z

## Abstract

Mathematical search problems present a unique challenge for Reinforcement Learning (RL) due to vast search spaces and sparse rewards. In previous works, the Andrews-Curtis (AC) conjecture was established as an illustrative example of such problems. In this work, we identify a critical structural barrier in the AC landscape: a "Two-Hump" distribution, where problem instances are either trivially solvable or effectively impossible, with a scarcity of intermediate "hard-but-solvable" instances required for effective learning. We tackle this challenge through two primary avenues: novel data generation techniques to populate the difficulty gap, and significant algorithmic enhancements including the introduction of supermoves and Transformer-based architectures. We demonstrate substantial performance improvements over previous baselines, and release new comprehensive benchmark datasets including AC-19 (125,192 AC-trivial presentations of varying difficulty with length at most 19) and AC-1M (1,136,154 hard AC-trivial presentations of length at most 30), the first large-scale, publicly available datasets of this kind.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Presentation Solving on Miller–Schupp benchmark 1190-presentation](<https://www.sota2.com/research/sota/presentation-solving-on-miller-schupp-benchmark-1190-presentation.md>): Presentation Solving; Miller–Schupp benchmark 1190-presentation; Presentations Solved: 640.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.21611>)

[PDF](<https://arxiv.org/pdf/2606.21611.pdf>)