Mathematical Reasoning on MATH Hard
92.8AccuracyIn-place
Evaluation results
| Method | Links | ||
|---|---|---|---|
| In-placeModel=Qwen, Number of turns=92025.10 | 92.8 | — | |
| In-placeModel=Gemma-3-27B, Number of turns=42025.10 | 92.4 | — | |
| In-placeModel=Qwen, Number of turns=82025.10 | 92.4 | — | |
| In-placeModel=Qwen, Number of turns=72025.10 | 91.8 | — | |
| Multi-turnModel=Gemma-3-27B, Number of turns=42025.10 | 91 | — | |
| In-placeModel=Gemma-3-27B, Number of turns=32025.10 | 91 | — | |
| MemoryModel=Gemma-3-27B, Number of turns=42025.10 | 90.8 | — | |
| Multi-turnModel=Gemma-3-27B, Number of turns=32025.10 | 90.4 | — | |
| In-placeModel=Qwen, Number of turns=62025.10 | 90.4 | — | |
| MemoryModel=Gemma-3-27B, Number of turns=32025.10 | 89.8 | — | |
| In-placeModel=Gemma-3-27B, Number of turns=22025.10 | 89.6 | — | |
| Multi-turnModel=Gemma-3-27B, Number of turns=22025.10 | 89.2 | — | |
| In-placeModel=Qwen, Number of turns=52025.10 | 88.8 | — | |
| MemoryModel=Gemma-3-27B, Number of turns=22025.10 | 87.8 | — | |
| In-placeModel=Gemma, Number of turns=92025.10 | 87.6 | — | |
| In-placeModel=Gemma-3-27B, Number of turns=12025.10 | 87.2 | — | |
| Multi-turnModel=Gemma-3-27B, Number of turns=12025.10 | 86.6 | — | |
| In-placeModel=Gemma, Number of turns=82025.10 | 86.6 | — | |
| MemoryModel=Gemma-3-27B, Number of turns=12025.10 | 86.2 | — | |
| Multi-turnModel=Qwen, Number of turns=92025.10 | 86.2 | — | |
| Multi-turnModel=Qwen, Number of turns=82025.10 | 85.8 | — | |
| In-placeModel=Qwen, Number of turns=42025.10 | 85.2 | — | |
| Multi-turnModel=Qwen, Number of turns=72025.10 | 85 | — | |
| In-placeModel=Gemma, Number of turns=72025.10 | 85 | — | |
| RPDI-EEBackbone=Qwen3-235B-Thinking, Reasoning Strategy=RPDI-EE2026.03 | 84.8 | 16,642 | |
| RPDI-EEBackbone=Qwen3-30B-A3B-Thinking-2507, Reasoning Strategy=RPDI-EE2026.03 | 84.6 | 14,927 | |
| In-placeModel=Llama, Number of turns=92025.10 | 84 | — | |
| Multi-turnModel=Qwen, Number of turns=62025.10 | 83.8 | — | |
| Multi-turnModel=Gemma, Number of turns=92025.10 | 82.8 | — | |
| In-placeModel=Gemma, Number of turns=62025.10 | 82.8 | — | |
| In-placeModel=Llama, Number of turns=82025.10 | 82.8 | — | |
| DEERBackbone=Qwen3-235B-Thinking, Reasoning Strategy=DEER2026.03 | 82.5 | 16,332 | |
| Vanilla CoTBackbone=Qwen3-30B-A3B-Thinking-2507, Reasoning Strategy=Vanilla CoT2026.03 | 82.1 | 15,621 | |
| Multi-turnModel=Gemma, Number of turns=82025.10 | 82 | — | |
| ThinkLessBackbone=Qwen3-30B-A3B-Thinking-2507, Reasoning Strategy=ThinkLess2026.03 | 81.8 | 14,418 | |
| Multi-turnModel=Qwen, Number of turns=52025.10 | 81.6 | — | |
| In-placeModel=Gemma, Number of turns=52025.10 | 81.4 | — | |
| In-placeModel=Qwen, Number of turns=32025.10 | 81.2 | — | |
| In-placeModel=Llama, Number of turns=72025.10 | 80.6 | — | |
| DEERBackbone=Qwen3-30B-A3B-Thinking-2507, Reasoning Strategy=DEER2026.03 | 80.5 | 15,285 | |
| Multi-turnModel=Gemma, Number of turns=72025.10 | 80.4 | — | |
| Vanilla CoTBackbone=Qwen3-235B-Thinking, Reasoning Strategy=Vanilla CoT2026.03 | 79.8 | 17,185 | |
| Multi-turnModel=Qwen, Number of turns=42025.10 | 79.2 | — | |
| Multi-turnModel=Gemma, Number of turns=62025.10 | 79.2 | — | |
| Multi-turnModel=Gemma-3-27B, Number of turns=02025.10 | 78.2 | — | |
| MemoryModel=Gemma-3-27B, Number of turns=02025.10 | 78.2 | — | |
| In-placeModel=Gemma-3-27B, Number of turns=02025.10 | 78.2 | — | |
| ACOERTraining Step=1,2002026.06 | 78.1 | 3,509 | |
| MemoryModel=Gemma, Number of turns=92025.10 | 78 | — | |
| GRPO-LEADTraining Step=200, Pre-collapse peak=true2026.06 | 77.9 | 7,110 | |
| In-placeModel=Gemma, Number of turns=42025.10 | 77.8 | — | |
| GRPO+LPTraining Step=400, Pre-collapse peak=true, Length Penalty (LP)=true2026.06 | 77.8 | 5,186 | |
| Multi-turnModel=Gemma, Number of turns=52025.10 | 77.6 | — | |
| GRPO-acc*Training Step=1,000, Accuracy-only GRPO=true2026.06 | 77.6 | 6,277 | |
| MemoryModel=Qwen, Number of turns=92025.10 | 77.4 | — | |
| MemoryModel=Gemma, Number of turns=82025.10 | 77.2 | — | |
| In-placeModel=Llama-3.1-70B, Number of turns=42025.10 | 76.8 | — | |
| In-placeModel=Llama, Number of turns=62025.10 | 76.8 | — | |
| Multi-turnModel=Qwen, Number of turns=32025.10 | 76.6 | — | |
| In-placeModel=Qwen, Number of turns=22025.10 | 76.6 | — | |
| MemoryModel=Qwen, Number of turns=82025.10 | 76.4 | — | |
| MemoryModel=Gemma, Number of turns=72025.10 | 76.4 | — | |
| BaseMethod variant=Baseline2026.06 | 76.4 | 7,958 | |
| Multi-turnModel=Gemma, Number of turns=42025.10 | 76 | — | |
| Multi-turnModel=Llama, Number of turns=92025.10 | 75.6 | — | |
| MemoryModel=Qwen, Number of turns=72025.10 | 75 | — | |
| In-placeModel=Gemma, Number of turns=32025.10 | 74.8 | — | |
| Multi-turnModel=Llama-3.1-70B, Number of turns=42025.10 | 74.6 | — | |
| MemoryModel=Gemma, Number of turns=62025.10 | 74.6 | — | |
| In-placeModel=Llama, Number of turns=52025.10 | 74.4 | — | |
| MemoryModel=Qwen, Number of turns=62025.10 | 74.2 | — | |
| Multi-turnModel=Qwen, Number of turns=22025.10 | 73.8 | — | |
| ReCutTraining Step=400, Pre-collapse peak=true2026.06 | 73.6 | 4,191 | |
| Multi-turnModel=Llama, Number of turns=82025.10 | 73.4 | — | |
| Multi-turnModel=Gemma, Number of turns=32025.10 | 73 | — | |
| MemoryModel=Qwen, Number of turns=52025.10 | 72.8 | — | |
| MemoryModel=Gemma, Number of turns=52025.10 | 72.6 | — | |
| MemoryModel=Llama-3.1-70B, Number of turns=42025.10 | 72.4 | — | |
| In-placeModel=Llama-3.1-70B, Number of turns=32025.10 | 71.8 | — | |
| In-placeModel=Qwen, Number of turns=12025.10 | 71.4 | — | |
| MemoryModel=Qwen, Number of turns=42025.10 | 71.2 | — | |
| Multi-turnModel=Llama, Number of turns=72025.10 | 71.2 | — | |
| Multi-turnModel=Llama-3.1-70B, Number of turns=32025.10 | 71 | — | |
| β=0 (s42)Training Step=1,200, Beta coefficient (β)=02026.06 | 70.9 | 4,149 | |
| In-placeModel=Gemma, Number of turns=22025.10 | 70.8 | — | |
| MemoryModel=Gemma, Number of turns=42025.10 | 70.6 | — | |
| MemoryModel=Llama, Number of turns=92025.10 | 69.8 | — | |
| MemoryModel=Qwen, Number of turns=32025.10 | 69.2 | — | |
| gemma-3-12b-itBackbone=Gemma 3, RL Type=/, Supervision=/, Environment=/2026.05 | 69.1 | — | |
| Multi-turnModel=Qwen, Number of turns=12025.10 | 68.4 | — | |
| Multi-turnModel=Gemma, Number of turns=22025.10 | 68.4 | — | |
| MemoryModel=Gemma, Number of turns=32025.10 | 68 | — | |
| MemoryModel=Llama, Number of turns=82025.10 | 67.6 | — | |
| Multi-turnModel=Llama, Number of turns=62025.10 | 67.4 | — | |
| MemoryModel=Llama-3.1-70B, Number of turns=32025.10 | 67 | — | |
| In-placeModel=Llama, Number of turns=42025.10 | 66.8 | — | |
| MemoryModel=Qwen, Number of turns=22025.10 | 66.2 | — | |
| MemoryModel=Llama, Number of turns=72025.10 | 65.6 | — | |
| MemoryModel=Gemma, Number of turns=22025.10 | 65 | — | |
| In-placeModel=Llama-3.1-70B, Number of turns=22025.10 | 64.6 | — |