Mathematical Reasoning on MATH 500 (Accuracy, Total Tokens)
83.6AccuracySC@5 + IRIS+
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SC@5 + IRIS+n=500, Hardware=H800, Extraction Strategy=Fixed non-oracle agreement gate2026.05 | 83.6 | 3,499 | |
| IRIS+Inference Budget (b)=4096, n=500, Hardware=H800, Extraction Strategy=Strengthened extraction2026.05 | 78.8 | 2,645 | |
| SC@5 nothinkInference Budget (b)=1024, n=500, Hardware=H8002026.05 | 76.6 | 2,685 | |
| IRISInference Budget (b)=4096, n=500, Hardware=H800, Extraction Strategy=full2026.05 | 74 | 2,401 | |
| TOWN+Inference Budget (b)=4096, n=500, Hardware=H800, Extraction Strategy=Strengthened extraction2026.05 | 72.4 | 2,565 | |
| TOWNInference Budget (b)=4096, n=500, Hardware=H800, Extraction Strategy=full2026.05 | 71.8 | 2,565 | |
| ThinkInference Budget (b)=4096, n=500, Hardware=H8002026.05 | 71 | 2,567 | |
| NothinkInference Budget (b)=4096, n=500, Hardware=H8002026.05 | 68.6 | 684 | |
| NothinkInference Budget (b)=2048, n=500, Hardware=H8002026.05 | 68.4 | 585 | |
| IRISInference Budget (b)=2048, n=500, Hardware=H800, Extraction Strategy=full2026.05 | 67.2 | 1,573 | |
| NothinkInference Budget (b)=1024, n=500, Hardware=A1002026.05 | 59.8 | 606 | |
| SC@5 nothinkInference Budget (b)=512, n=500, Hardware=H8002026.05 | 57 | 1,970 | |
| SC@3 nothinkInference Budget (b)=512, n=500, Hardware=H8002026.05 | 55.8 | 1,182 | |
| TOWNInference Budget (b)=2048, n=500, Hardware=H800, Extraction Strategy=full2026.05 | 55 | 1,590 | |
| ThinkInference Budget (b)=2048, n=500, Hardware=H8002026.05 | 54.8 | 1,706 | |
| NothinkInference Budget (b)=512, n=200, Hardware=A100, Mode=pilot2026.05 | 47.5 | 418 |