Mathematical Reasoning on MATH-500 (Accuracy, ∆ Accuracy)
99.3AccuracyTeamTR (Homogeneous)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TeamTR (Homogeneous)Size=3×8B2026.05 | 99.3 | — | |
| TeamTR (Heterogeneous)Size=1.7B+8B+14B2026.05 | 98.1 | — | |
| o4-miniSize=–2026.05 | 98 | — | |
| Qwen3-A3BSize=30B2026.05 | 98 | — | |
| QwQSize=32B2026.05 | 98 | — | |
| Qwen3 (thinking)Size=32B2026.05 | 97.2 | — | |
| Debate + JudgeSize=3×8B2026.05 | 95.9 | — | |
| Role-playSize=3×8B2026.05 | 95.7 | — | |
| ReasonFlux-PRM-1.5BBase Model=GPT-OSS-20B, Size=1.5B, Infer (ms)=119.32026.04 | 95.5 | — | |
| DebateSize=3×8B2026.05 | 95.2 | — | |
| SCATRBase Model=GPT-OSS-20B, Size=2.3M, Infer (ms)=0.17, Speedup=684.3×2026.04 | 95 | -0.5 | |
| DeepSeek-R1 DistillSize=70B2026.05 | 94.5 | — | |
| DAPOSize=8B2026.05 | 91.5 | — | |
| SCATRBase Model=Qwen-30B-A3B, Size=2.6M, Infer (ms)=0.1, Speedup=1148.1×2026.04 | 91.3 | 2.3 | |
| GRPOSize=8B2026.05 | 90.7 | — | |
| ReasonFlux-PRM-1.5BBase Model=Qwen-30B-A3B, Size=1.5B, Infer (ms)=167.82026.04 | 88.9 | — | |
| SCATRBase Model=Qwen-1.7B, Size=1.7M, Infer (ms)=0.15, Speedup=1115.2×2026.04 | 88.3 | 1 | |
| ReasonFlux-PRM-1.5BBase Model=Qwen-1.7B, Size=1.5B, Infer (ms)=170.92026.04 | 87.3 | — | |
| SCATRBase Model=Qwen2.5-14B rollouts2026.04 | 80.4 | 1.2 | |
| ReasonFlux-1.5BBase Model=Qwen2.5-14B rollouts2026.04 | 79.2 | — | |
| GPT-4o-miniSize=–2026.05 | 78.2 | — |