Aggregate Reasoning Benchmark (GSM8K, GSM-Hard, MATH, StrategyQA, Bamboogle, Coin, Letter, LB, Headline) (test)
92.6GSM8K AccuracyRM-Regen
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| RM-RegenBase Model=Llama 3.1-8B, Evaluation Protocol=Oracle Verification2026.03 | 92.6 | 41 | 80.6 | 83.25 | 63 | 85.75 | 89.33 | 66.32 | 75 | 75.85 | |
| RM-RegenBase Model=Llama 3.1-8B, Evaluation Protocol=Oracle Verification, Verification Feedback=R+2026.03 | 91.8 | 38.2 | 79.4 | 81.5 | 60 | 85 | 86.67 | 65.26 | 74 | 74.28 | |
| Best-of-NBase Model=Llama 3.1-8B, Evaluation Protocol=Oracle Verification, Number of Candidates (N)=102026.03 | 91.6 | 36 | 81 | 81.25 | 52 | 72.5 | 70.67 | 72.63 | 73 | 71.33 | |
| RM-RegenBase Model=GPT-3.5, Evaluation Protocol=Oracle Verification2026.03 | 90.2 | 68.6 | 63.4 | 77.25 | 73 | 85.75 | 87.33 | 75.79 | 73 | 76.94 | |
| ReflexionBase Model=Llama 3.1-8B, Evaluation Protocol=Oracle Verification, Iterations=5 iters2026.03 | 89.8 | 38.2 | 80.6 | 90.25 | 60 | 82.75 | 82 | 87.37 | 71 | 75.48 | |
| ReflexionBase Model=Llama 3.1-8B, Evaluation Protocol=Oracle Verification, Iterations=3 iters2026.03 | 89.6 | 36.2 | 77.8 | 85 | 60 | 81.25 | 80.67 | 81.05 | 70 | 73.26 | |
| Best-of-NBase Model=Llama 3.1-8B, Evaluation Protocol=Oracle Verification, Number of Candidates (N)=32026.03 | 89.6 | 34.2 | 76.8 | 77.5 | 53 | 72 | 70.67 | 64.21 | 75 | 69.08 | |
| ReflexionBase Model=Llama 3.1-8B, Evaluation Protocol=Oracle Verification, Iterations=4 iters2026.03 | 89.4 | 37.6 | 78.8 | 90.25 | 60 | 82 | 82 | 82.11 | 73 | 74.75 | |
| ReflexionBase Model=Llama 3.1-8B, Evaluation Protocol=Oracle Verification, Iterations=2 iters2026.03 | 88.6 | 34.4 | 75.4 | 80.75 | 59 | 75.5 | 76.67 | 78.95 | 68 | 70.46 | |
| ReflexionBase Model=GPT-3.5, Evaluation Protocol=Oracle Verification, Iterations=4 iters2026.03 | 88.2 | 48.8 | 62.8 | 73 | 76 | 90.5 | 78.67 | 91.58 | 74 | 73.15 | |
| RM-RegenBase Model=GPT-3.5, Evaluation Protocol=Oracle Verification, Verification Feedback=R+2026.03 | 88.2 | 64.2 | 64.6 | 78.75 | 72 | 85.25 | 84.67 | 71.58 | 71 | 75.74 | |
| ReflexionBase Model=GPT-3.5, Evaluation Protocol=Oracle Verification, Iterations=5 iters2026.03 | 86.2 | 47.4 | 64.2 | 75.5 | 70 | 91.5 | 81.33 | 92.63 | 73 | 73.22 | |
| ReflexionBase Model=GPT-3.5, Evaluation Protocol=Oracle Verification, Iterations=3 iters2026.03 | 85.6 | 45.2 | 62.25 | 71 | 66 | 88.25 | 78.67 | 85.26 | 72 | 70.65 | |
| RflecEvoBase Model=Llama 3.1-8B, Evaluation Protocol=Oracle Verification2026.03 | 85.4 | 32 | 73.6 | 71.25 | 53 | 67.75 | 68.67 | 50.53 | 62 | 64.74 | |
| ReflexionBase Model=GPT-3.5, Evaluation Protocol=Oracle Verification, Iterations=2 iters2026.03 | 84 | 44 | 59 | 67.75 | 63 | 87.25 | 76 | 62.11 | 72 | 67.87 |