Mathematical Reasoning on HMMT (Accuracy, Variance, and Efficiency Metrics)
88.9AccuracyInsightReplay (IR3)
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| InsightReplay (IR3)Model=Gemma4-31B, Inference Mode=IR32026.05 | 88.9 | — | — | 0.8 | 13,730 | — | — | 1.62 | |
| Verify-Only (VO)Model=Gemma4-31B, Inference Mode=VO2026.05 | 88.7 | 0.6 | — | — | 10,430 | 1.23 | — | — | |
| InsightReplay (IR3)Model=Qwen3.5-35B, Inference Mode=IR32026.05 | 88.6 | — | — | 0.1 | 51,840 | — | — | 1.29 | |
| Base (CoT)Model=Qwen3.5-35B, Inference Mode=Base2026.05 | 88.5 | — | — | — | 40,124 | — | — | — | |
| Verify-Only (VO)Model=Qwen3.5-35B, Inference Mode=VO2026.05 | 88.5 | 0 | — | — | 44,029 | 1.1 | — | — | |
| InsightReplay (IR1)Model=Qwen3.5-35B, Inference Mode=IR12026.05 | 88.4 | — | 0.1 | — | 45,161 | — | 1.13 | — | |
| Base (CoT)Model=Gemma4-31B, Inference Mode=Base2026.05 | 88.1 | — | — | — | 8,499 | — | — | — | |
| InsightReplay (IR1)Model=Gemma4-31B, Inference Mode=IR12026.05 | 88.1 | — | 0 | — | 10,541 | — | 1.24 | — | |
| InsightReplay (IR3)Model=Qwen3.5-9B, Inference Mode=IR32026.05 | 84.5 | — | — | 1 | 58,490 | — | — | 1.29 | |
| InsightReplay (IR1)Model=Qwen3.5-9B, Inference Mode=IR12026.05 | 84.2 | — | 0.7 | — | 51,088 | — | 1.12 | — | |
| Verify-Only (VO)Model=Qwen3.5-9B, Inference Mode=VO2026.05 | 83.9 | 0.4 | — | — | 49,796 | 1.09 | — | — | |
| Base (CoT)Model=Qwen3.5-9B, Inference Mode=Base2026.05 | 83.5 | — | — | — | 45,477 | — | — | — | |
| InsightReplay (IR3)Model=R1-Distill-32B, Inference Mode=IR32026.05 | 45.5 | — | — | 1.7 | 17,278 | — | — | 1.34 | |
| InsightReplay (IR1)Model=R1-Distill-32B, Inference Mode=IR12026.05 | 44.5 | — | 0.7 | — | 14,271 | — | 1.11 | — | |
| Verify-Only (VO)Model=R1-Distill-32B, Inference Mode=VO2026.05 | 44.1 | 0.3 | — | — | 13,867 | 1.07 | — | — | |
| Base (CoT)Model=R1-Distill-32B, Inference Mode=Base2026.05 | 43.8 | — | — | — | 12,912 | — | — | — | |
| Verify-Only (VO)Model=Gemma4-E4B, Inference Mode=VO2026.05 | 37.7 | 1.3 | — | — | 9,203 | 1.23 | — | — | |
| InsightReplay (IR1)Model=Gemma4-E4B, Inference Mode=IR12026.05 | 36.7 | — | 0.3 | — | 9,064 | — | 1.21 | — | |
| InsightReplay (IR3)Model=Gemma4-E4B, Inference Mode=IR32026.05 | 36.6 | — | — | 0.2 | 11,906 | — | — | 1.59 | |
| Base (CoT)Model=Gemma4-E4B, Inference Mode=Base2026.05 | 36.4 | — | — | — | 7,481 | — | — | — | |
| Verify-Only (VO)Model=R1-Distill-7B, Inference Mode=VO2026.05 | 29.1 | 0.8 | — | — | 15,449 | 1.07 | — | — | |
| InsightReplay (IR1)Model=R1-Distill-7B, Inference Mode=IR12026.05 | 29 | — | 0.7 | — | 15,818 | — | 1.1 | — | |
| InsightReplay (IR3)Model=R1-Distill-7B, Inference Mode=IR32026.05 | 29 | — | — | 0.7 | 19,853 | — | — | 1.38 | |
| Base (CoT)Model=R1-Distill-7B, Inference Mode=Base2026.05 | 28.3 | — | — | — | 14,417 | — | — | — | |
| DAPO w/ EDASBackbone=Qwen3-4B2026.05 | 23.12 | — | — | — | — | — | — | — | |
| DAPOBackbone=Qwen3-4B2026.05 | 22.4 | — | — | — | — | — | — | — | |
| DAPO w/ EDASBackbone=Qwen3-8B2026.05 | 20.1 | — | — | — | — | — | — | — | |
| DAPOBackbone=Qwen3-8B2026.05 | 15.94 | — | — | — | — | — | — | — | |
| DAPO w/ En AdvBackbone=Qwen3-8B2026.05 | 15.21 | — | — | — | — | — | — | — | |
| DAPO w/ En AdvBackbone=Qwen3-4B2026.05 | 13.65 | — | — | — | — | — | — | — | |
| GRPO w/ EDASBackbone=Qwen3-8B2026.05 | 12.5 | — | — | — | — | — | — | — | |
| PKPOBackbone=Qwen3-8B2026.05 | 12.08 | — | — | — | — | — | — | — | |
| GRPOBackbone=Qwen3-8B2026.05 | 11.25 | — | — | — | — | — | — | — | |
| DAPO w/ EDASBackbone=Qwen3-4B-Base2026.05 | 10.94 | — | — | — | — | — | — | — | |
| BaselineBackbone=Qwen3-4B2026.05 | 10.83 | — | — | — | — | — | — | — | |
| BaselineBackbone=Qwen3-8B2026.05 | 10.42 | — | — | — | — | — | — | — | |
| DAPOBackbone=Qwen3-4B-Base2026.05 | 8.33 | — | — | — | — | — | — | — | |
| INFUSERModel Backbone=Qwen3-8B-Base2026.06 | 7.04 | — | — | — | — | — | — | — | |
| DAPO w/ En AdvBackbone=Qwen3-4B-Base2026.05 | 6.67 | — | — | — | — | — | — | — | |
| AZRModel Backbone=Qwen3-8B-Base2026.06 | 6.01 | — | — | — | — | — | — | — | |
| AZRModel Backbone=Qwen3-4B-Base2026.06 | 4.5 | — | — | — | — | — | — | — | |
| R-ZeroModel Backbone=Qwen3-8B-Base2026.06 | 3.86 | — | — | — | — | — | — | — | |
| BaseModel Backbone=Qwen3-8B-Base2026.06 | 2.96 | — | — | — | — | — | — | — | |
| INFUSERModel Backbone=Qwen3-4B-Base2026.06 | 2.94 | — | — | — | — | — | — | — | |
| R-ZeroModel Backbone=Qwen3-4B-Base2026.06 | 2.65 | — | — | — | — | — | — | — | |
| BaseModel Backbone=Qwen3-4B-Base2026.06 | 2.49 | — | — | — | — | — | — | — | |
| BaselineBackbone=Qwen3-4B-Base2026.05 | 1.56 | — | — | — | — | — | — | — |