# Mathematical Reasoning on OlympiadBench (pass@3)

Source: [Mathematical Reasoning on OlympiadBench (pass@3)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-olympiadbench-pass-at-3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-04T07:48:25.570Z

Evaluation of model performance on mathematical reasoning problems using the OlympiadBench dataset, measured by pass@3.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Mathematical Reasoning |
| Dataset (including any stated split) | OlympiadBench |
| Metric | Pass@3 |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Pass@3](<https://www.sota2.com/research/sota/mathematical-reasoning-on-olympiadbench-pass-at-3.md?metric=Pass%403>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Pass@3 | Result date | Report paper |
| --- | --- | --- | --- |
| HDPO | 64.76 | Not recorded | 6a210686c10484bac23f010b |
| HDPO | 62.37 | Not recorded | 6a210686c10484bac23f010b |
| Critique-GRPO | 61.46 | Not recorded | 6a210686c10484bac23f010b |
| HDPO | 56.19 | Not recorded | 6a210686c10484bac23f010b |
| LUFFY | 56.09 | Not recorded | 6a210686c10484bac23f010b |
| GRPO | 55.5 | Not recorded | 6a210686c10484bac23f010b |
| GRPO | 54.74 | Not recorded | 6a210686c10484bac23f010b |
| GRPO | 53.68 | Not recorded | 6a210686c10484bac23f010b |
| Oat | 50.82 | Not recorded | 6a210686c10484bac23f010b |
| SimpleRL | 49.84 | Not recorded | 6a210686c10484bac23f010b |
| EDGE-GRPO | 48.79 | Not recorded | 6a210686c10484bac23f010b |
| Base Model | 43.73 | Not recorded | 6a210686c10484bac23f010b |
| Base Model | 40.11 | Not recorded | 6a210686c10484bac23f010b |
| Base Model | 38.88 | Not recorded | 6a210686c10484bac23f010b |

### HDPO

Pass@3: 64.76. Result ID: 6a212cecfbc373592c1c6e05.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-Math-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### HDPO

Pass@3: 62.37. Result ID: 6a212cecfbc373592c1c6e02.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B","Number of samples":"16"}. This record does not assert independent reproduction.

### Critique-GRPO

Pass@3: 61.46. Result ID: 6a212cecfbc373592c1c6dff.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ac21e209131cea850cf.md>)

Recorded evaluation configuration: {"Backbone":"8B","Number of samples":"16"}. This record does not assert independent reproduction.

### HDPO

Pass@3: 56.19. Result ID: 6a212cecfbc373592c1c6e08.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

Recorded evaluation configuration: {"Backbone":"DS-R1-Llama-8B","Number of samples":"16"}. This record does not assert independent reproduction.

### LUFFY

Pass@3: 56.09. Result ID: 6a212cecfbc373592c1c6dfe.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79961e209131cea7e682.md>)

Recorded evaluation configuration: {"Backbone":"Qwen-Math-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### GRPO

Pass@3: 55.5. Result ID: 6a212cecfbc373592c1c6e01.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B","Number of samples":"16"}. This record does not assert independent reproduction.

### GRPO

Pass@3: 54.74. Result ID: 6a212cecfbc373592c1c6e07.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Backbone":"DS-R1-Llama-8B","Number of samples":"16"}. This record does not assert independent reproduction.

### GRPO

Pass@3: 53.68. Result ID: 6a212cecfbc373592c1c6e04.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-Math-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### Oat

Pass@3: 50.82. Result ID: 6a212cecfbc373592c1c6dfc.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7be51e209131cea8b201.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-Math-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### SimpleRL

Pass@3: 49.84. Result ID: 6a212cecfbc373592c1c6dfd.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a721e209131cea8345b.md>)

Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### EDGE-GRPO

Pass@3: 48.79. Result ID: 6a212cecfbc373592c1c6dfb.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a431e209131cea82028.md>)

Recorded evaluation configuration: {"Backbone":"Qwen-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### Base Model

Pass@3: 43.73. Result ID: 6a212cecfbc373592c1c6e03.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-Math-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### Base Model

Pass@3: 40.11. Result ID: 6a212cecfbc373592c1c6e00.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B","Number of samples":"16"}. This record does not assert independent reproduction.

### Base Model

Pass@3: 38.88. Result ID: 6a212cecfbc373592c1c6e06.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

Recorded evaluation configuration: {"Backbone":"DS-R1-Llama-8B","Number of samples":"16"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
