Math Problem Solving on 200 IMO-level math problems IMO-AnswerBench, IMO-ProofBench, ArXivMath (test)
50.6Pass@1 AccuracyMeta-Harness
Evaluation Results
| Method | Links | |
|---|---|---|
| Meta-HarnessBackbone Model=GPT-20B, Retrieval Strategy=Meta-Harness2026.03 | 50.6 | |
| BM25 RetrievalBackbone Model=GPT-20B, Retrieval Strategy=BM25 Retrieval2026.03 | 48.9 | |
| No RetrieverBackbone Model=GPT-20B, Retrieval Strategy=None2026.03 | 47.6 | |
| Dense RetrievalBackbone Model=Gem-3F, Retrieval Strategy=Dense Retrieval, k (Retrieval Count)=52026.03 | 47.2 | |
| Dense RetrievalBackbone Model=GPT-20B, Retrieval Strategy=Dense Retrieval, k (Retrieval Count)=12026.03 | 46.9 | |
| Dense RetrievalBackbone Model=GPT-20B, Retrieval Strategy=Dense Retrieval, k (Retrieval Count)=52026.03 | 46.7 | |
| BM25 RetrievalBackbone Model=Gem-3F, Retrieval Strategy=BM25 Retrieval2026.03 | 46.6 | |
| Meta-HarnessBackbone Model=Gem-3F, Retrieval Strategy=Meta-Harness2026.03 | 46.3 | |
| No RetrieverBackbone Model=Gem-3F, Retrieval Strategy=None2026.03 | 42.6 | |
| Dense RetrievalBackbone Model=Gem-3F, Retrieval Strategy=Dense Retrieval, k (Retrieval Count)=12026.03 | 42.3 | |
| Random Few-shotBackbone Model=GPT-20B, Retrieval Strategy=Random Few-shot2026.03 | 41.8 | |
| Random Few-shotBackbone Model=Gem-3F, Retrieval Strategy=Random Few-shot2026.03 | 40.4 | |
| Meta-HarnessBackbone Model=Avg., Retrieval Strategy=Meta-Harness2026.03 | 38.8 | |
| Dense RetrievalBackbone Model=Avg., Retrieval Strategy=Dense Retrieval, k (Retrieval Count)=52026.03 | 38.1 | |
| BM25 RetrievalBackbone Model=Avg., Retrieval Strategy=BM25 Retrieval2026.03 | 37.5 | |
| Dense RetrievalBackbone Model=Gem-3.1FL, Retrieval Strategy=Dense Retrieval, k (Retrieval Count)=52026.03 | 37.1 | |
| Meta-HarnessBackbone Model=Gem-3.1FL, Retrieval Strategy=Meta-Harness2026.03 | 34.9 | |
| Dense RetrievalBackbone Model=Avg., Retrieval Strategy=Dense Retrieval, k (Retrieval Count)=12026.03 | 34.4 | |
| No RetrieverBackbone Model=Avg., Retrieval Strategy=None2026.03 | 34.1 | |
| BM25 RetrievalBackbone Model=Gem-3.1FL, Retrieval Strategy=BM25 Retrieval2026.03 | 32.8 | |
| Random Few-shotBackbone Model=Avg., Retrieval Strategy=Random Few-shot2026.03 | 32.2 | |
| Meta-HarnessBackbone Model=GPT-5.4n, Retrieval Strategy=Meta-Harness2026.03 | 31.7 | |
| Dense RetrievalBackbone Model=Gem-3.1FL, Retrieval Strategy=Dense Retrieval, k (Retrieval Count)=12026.03 | 31.3 | |
| Dense RetrievalBackbone Model=GPT-5.4n, Retrieval Strategy=Dense Retrieval, k (Retrieval Count)=52026.03 | 31.1 | |
| Random Few-shotBackbone Model=Gem-3.1FL, Retrieval Strategy=Random Few-shot2026.03 | 31 | |
| Meta-HarnessBackbone Model=GPT-5.4m, Retrieval Strategy=Meta-Harness2026.03 | 30.4 | |
| BM25 RetrievalBackbone Model=GPT-5.4n, Retrieval Strategy=BM25 Retrieval2026.03 | 30.2 | |
| BM25 RetrievalBackbone Model=GPT-5.4m, Retrieval Strategy=BM25 Retrieval2026.03 | 29.2 | |
| No RetrieverBackbone Model=GPT-5.4m, Retrieval Strategy=None2026.03 | 28.8 | |
| No RetrieverBackbone Model=Gem-3.1FL, Retrieval Strategy=None2026.03 | 28.6 | |
| Dense RetrievalBackbone Model=GPT-5.4m, Retrieval Strategy=Dense Retrieval, k (Retrieval Count)=52026.03 | 28.3 | |
| Dense RetrievalBackbone Model=GPT-5.4n, Retrieval Strategy=Dense Retrieval, k (Retrieval Count)=12026.03 | 27.1 | |
| Dense RetrievalBackbone Model=GPT-5.4m, Retrieval Strategy=Dense Retrieval, k (Retrieval Count)=12026.03 | 24.5 | |
| Random Few-shotBackbone Model=GPT-5.4m, Retrieval Strategy=Random Few-shot2026.03 | 24.5 | |
| Random Few-shotBackbone Model=GPT-5.4n, Retrieval Strategy=Random Few-shot2026.03 | 23.1 | |
| No RetrieverBackbone Model=GPT-5.4n, Retrieval Strategy=None2026.03 | 23 |