Reasoning on GSM8K, GSM-Hard, MATH, StrategyQA, Bamboogle, Coin, Letter, LB, Headline
85.2GSM8KRM-Primed
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| RM-PrimedBackbone=Llama 3.1-8B2026.03 | 85.2 | 34.2 | 70 | 73.25 | 54 | 80.75 | 81.33 | 64.21 | 73 | 68.23 | |
| Few-shot CoTBackbone=Llama 3.1-8B2026.03 | 84.8 | 31.8 | 71.4 | 69.25 | 51 | 65.75 | 68 | 48.42 | 67 | 64.19 | |
| RM-Primed (R+)Backbone=Llama 3.1-8B2026.03 | 84.6 | 35.8 | 64.4 | 70.25 | 51 | 78.75 | 72 | 62.11 | 72 | 65.94 | |
| RM-PrimedBackbone=GPT-3.52026.03 | 81.8 | 44.6 | 54.8 | 68.75 | 62 | 79.75 | 76 | 67.37 | 72 | 66.01 | |
| RM-Primed (R+)Backbone=GPT-3.52026.03 | 81.5 | 42.5 | 55 | 65.25 | 59 | 78 | 84 | 64.21 | 70 | 64.99 | |
| Few-shot CoTBackbone=GPT-3.52026.03 | 79.25 | 41.25 | 51.6 | 61.5 | 55 | 74.25 | 72 | 44.21 | 70 | 61.15 | |
| Contrastive CoT (w r)Backbone=GPT-3.5, with reflection memory=true2026.03 | 78.4 | 38 | 52.2 | 63 | 50 | 69.25 | 73.33 | 55.79 | 71 | 60.33 | |
| Contrastive CoT (w/o r)Backbone=GPT-3.52026.03 | 77.2 | 39.4 | 47 | 60.75 | 49 | 59 | 75.33 | 35.75 | 72 | 57.01 | |
| Contrastive CoT (w r)Backbone=Llama 3.1-8B, with reflection memory=true2026.03 | 76.4 | 33 | 67.6 | 70.25 | 29 | 66.25 | 62.67 | 53.68 | 73 | 61.13 | |
| Contrastive CoT (w/o r)Backbone=Llama 3.1-8B2026.03 | 76.2 | 33.2 | 66 | 69.25 | 25 | 60.75 | 61.33 | 57.89 | 66 | 59.56 |