Mathematical Reasoning on HMMT25 (Accuracy, Token count (x10^7))
86.7AccuracyMajority
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MajorityModel=PaCoRe-8B, Sampling Mode=Parallel, Sample Count=@5122026.02 | 86.7 | — | — | — | — | |
| DeepConfModel=PaCoRe-8B, Sample Count=@5122026.02 | 86.7 | — | — | — | — | |
| HyPERModel=Qwen3-Next2026.02 | 85.3 | — | 0.74 | — | — | |
| CoRefine TreeModel=PaCoRe-8B2026.02 | 83.3 | — | — | — | — | |
| CoRefine TreeModel=DeepSeek-8B, Compute Budget=@5122026.02 | 82.7 | 0.76 | — | — | — | |
| CoRefine TreeModel=DeepSeek-8B2026.02 | 82.7 | — | — | — | — | |
| DeepConfModel=DeepSeek-8B, Compute Budget=@5122026.02 | 82.6 | 34.3 | — | — | — | |
| DeepConfModel=DeepSeek-8B, Sample Count=@5122026.02 | 82.6 | — | — | — | — | |
| DeepConfModel=DeepSeek-8B, Sample Count=@202026.02 | 82 | — | — | — | — | |
| CoRefineModel=DeepSeek-8B, Compute Budget=@5122026.02 | 81.3 | 0.73 | — | — | — | |
| CoRefineModel=DeepSeek-8B2026.02 | 81.3 | — | — | — | — | |
| DeepConfModel=PaCoRe-8B, Sample Count=@202026.02 | 81.3 | — | — | — | — | |
| MajorityModel=PaCoRe-8B, Sampling Mode=Parallel, Sample Count=@202026.02 | 80.7 | — | — | — | — | |
| MajorityModel=PaCoRe-8B, Sampling Mode=Sequential, Sample Count=@202026.02 | 80.7 | — | — | — | — | |
| CoRefineModel=PaCoRe-8B2026.02 | 80.7 | — | — | — | — | |
| DeepConfModel=Qwen3-Next2026.02 | 80.7 | — | 0.76 | — | — | |
| Cons@nBackbone=OSS-120B-medium, Aggregation Method=Standard self-consistency2026.02 | 80 | — | — | 355.6 | — | |
| Think@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over highest DTR scores, Early Stopping Prefix Length=502026.02 | 80 | — | — | 181.9 | -49 | |
| HyPERModel=Qwen3-30B2026.02 | 78.7 | — | 0.77 | — | — | |
| RoEModel=Qwen3-Next2026.02 | 78 | — | 1.76 | — | — | |
| Self-Certainty@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over highest Self-Certainty score, Early Stopping Prefix Length=502026.02 | 78 | — | — | 177 | -50 | |
| Recursive Self-AggregationAggregation=Majority Vote, Cost ($)=4.032026.03 | 77.8 | — | — | — | — | |
| Short@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over shortest responses2026.02 | 77.3 | — | — | 290.4 | -18 | |
| SCModel=Qwen3-Next2026.02 | 76.7 | — | 1 | — | — | |
| Agentic DebateAggregation=LLM Aggregate, Cost ($)=4.802026.03 | 75.6 | — | — | — | — | |
| PRISMAggregation=LLM Aggregate, Cost ($)=7.112026.03 | 75.6 | — | — | — | — | |
| PRISMAggregation=PRM-score Vote, Cost ($)=6.762026.03 | 75.4 | — | — | — | — | |
| Recursive Self-AggregationAggregation=LLM Aggregate, Cost ($)=4.152026.03 | 74.4 | — | — | — | — | |
| DeepConfModel=Qwen3-30B2026.02 | 74 | — | 0.84 | — | — | |
| Long@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over longest responses2026.02 | 73.3 | — | — | 355.6 | — | |
| MAD FollowerAggregation=LLM Aggregate, Cost ($)=3.42026.03 | 73.3 | — | — | — | — | |
| MajorityModel=DeepSeek-8B, Sampling Mode=Parallel, Sample Count=@202026.02 | 72.6 | — | — | — | — | |
| Agentic DebateAggregation=Majority Vote, Cost ($)=4.672026.03 | 72.2 | — | — | — | — | |
| PRISMAggregation=Majority Vote, Cost ($)=6.762026.03 | 72.2 | — | — | — | — | |
| MajorityModel=DeepSeek-8B, Sampling Mode=Sequential, Sample Count=@202026.02 | 72 | — | — | — | — | |
| RoEModel=Qwen3-30B2026.02 | 71.3 | — | 1.78 | — | — | |
| Self-CertaintyModel=Qwen3-Next2026.02 | 70 | — | 1 | — | — | |
| SciMasterAggregation=Majority Vote, Cost ($)=6.602026.03 | 70 | — | — | — | — | |
| SciMasterAggregation=LLM Aggregate, Cost ($)=6.872026.03 | 70 | — | — | — | — | |
| SCModel=Qwen3-30B2026.02 | 69.4 | — | 1 | — | — | |
| MajorityModel=DeepSeek-8B, Compute Budget=@5122026.02 | 69.3 | 44.9 | — | — | — | |
| MajorityModel=DeepSeek-8B, Sampling Mode=Parallel, Sample Count=@5122026.02 | 69.3 | — | — | — | — | |
| gpt-oss-120bMode=zero-shot, Aggregation=–, Cost ($)=0.202026.03 | 68.9 | — | — | — | — | |
| Self-CertaintyModel=Qwen3-30B2026.02 | 68.7 | — | 1.03 | — | — | |
| Think@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over highest DTR scores, Early Stopping Prefix Length=502026.02 | 66.7 | — | — | 641.4 | -50 | |
| MAD ConformistAggregation=Majority Vote, Cost ($)=1.712026.03 | 66.7 | — | — | — | — | |
| CoRefine TreeModel=Qwen3-32B, Compute Budget=@5122026.02 | 66 | 0.6 | — | — | — | |
| CoRefine TreeModel=Qwen3-32B2026.02 | 66 | — | — | — | — | |
| MAD ConformistAggregation=LLM Aggregate, Cost ($)=1.812026.03 | 65.6 | — | — | — | — | |
| CoRefineModel=Qwen3-32B, Compute Budget=@5122026.02 | 64.7 | 0.4 | — | — | — | |
| CoRefineModel=Qwen3-32B2026.02 | 64.7 | — | — | — | — | |
| MajorityModel=Qwen3-32B, Compute Budget=@5122026.02 | 63.3 | 27.6 | — | — | — | |
| MajorityModel=Qwen3-32B, Sampling Mode=Parallel, Sample Count=@5122026.02 | 63.3 | — | — | — | — | |
| Cons@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Standard self-consistency2026.02 | 63.3 | — | — | 1,275.7 | — | |
| Short@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over shortest responses2026.02 | 63.3 | — | — | 1,165.7 | -9 | |
| Self-Certainty@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over highest Self-Certainty score, Early Stopping Prefix Length=502026.02 | 63.3 | — | — | 641.4 | -50 | |
| DeepConfModel=Qwen3-32B, Compute Budget=@5122026.02 | 62.6 | 2.24 | — | — | — | |
| DeepConfModel=Qwen3-32B, Sample Count=@5122026.02 | 62.6 | — | — | — | — | |
| Mean@nBackbone=OSS-120B-medium, Aggregation Method=Average accuracy2026.02 | 62.6 | — | — | 355.6 | — | |
| MAD FollowerAggregation=Majority Vote, Cost ($)=3.302026.03 | 62.2 | — | — | — | — | |
| DeepConfModel=Qwen3-32B, Sample Count=@202026.02 | 62 | — | — | — | — | |
| MajorityModel=Qwen3-32B, Sampling Mode=Parallel, Sample Count=@202026.02 | 61.3 | — | — | — | — | |
| MajorityModel=Qwen3-32B, Sampling Mode=Sequential, Sample Count=@202026.02 | 60.7 | — | — | — | — | |
| PaCoRe-8BStrategy=Pass, Sample Count=@12026.02 | 60 | — | — | — | — | |
| PRM-score VoteAggregation=PRM-score Vote, Cost ($)=1.012026.03 | 59.7 | — | — | — | — | |
| Majority VoteAggregation=Majority Vote, Cost ($)=0.872026.03 | 58.9 | — | — | — | — | |
| DeepSeek-8BStrategy=Pass, Sample Count=@12026.02 | 58 | — | — | — | — | |
| Mean@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Average accuracy2026.02 | 55.7 | — | — | 1,275.7 | — | |
| Long@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over longest responses2026.02 | 52.7 | — | — | 1,275.7 | — | |
| Qwen3-32BStrategy=Pass, Sample Count=@12026.02 | 52 | — | — | — | — | |
| AutoTTS (β=1.0)Base Model=Qwen3-8B, Type=Discovered2026.05 | 49.5 | — | — | 749.1 | — | |
| SC@64Base Model=Qwen3-8B, Type=Parallel2026.02 | 48.9 | 0.1267 | — | 34,800 | — | |
| ESCBase Model=Qwen3-8B, Type=Hybrid2026.02 | 48.9 | 0.1062 | — | 184,500 | — | |
| SC @ 64Base Model=Qwen3-8B, Type=Handcrafted2026.05 | 48.9 | — | — | 1,267 | — | |
| ESCBase Model=Qwen3-8B, Type=Handcrafted2026.05 | 48.9 | — | — | 1,062.1 | — | |
| ASCBase Model=Qwen3-8B, Type=Seq.2026.02 | 48.8 | 0.0565 | — | 565,100 | — | |
| ASCBase Model=Qwen3-8B, Type=Handcrafted2026.05 | 48.8 | — | — | 565.1 | — | |
| AutoTTS (β=0.5)Base Model=Qwen3-8B, Type=Discovered2026.05 | 48.1 | — | — | 396.7 | — | |
| Parallel-ProbeBase Model=Qwen3-8B, Type=Parallel2026.02 | 47.1 | 0.0897 | — | 22,400 | — | |
| Parallel-ProbeBase Model=Qwen3-8B, Type=Handcrafted2026.05 | 47.1 | — | — | 897.2 | — | |
| AutoTTS (β=1.0)Base Model=Qwen3-4B, Type=Discovered2026.05 | 46.5 | — | — | 686.8 | — | |
| AutoTTS (β=0.5)Base Model=Qwen3-4B, Type=Discovered2026.05 | 45.7 | — | — | 365 | — | |
| Parallel-ProbeBase Model=Qwen3-4B, Type=Parallel2026.02 | 44.7 | 0.0872 | — | 21,500 | — | |
| Parallel-ProbeBase Model=Qwen3-4B, Type=Handcrafted2026.05 | 44.7 | — | — | 872.3 | — | |
| gpt-oss-20bMode=zero-shot, Aggregation=–, Cost ($)=0.082026.03 | 44.4 | — | — | — | — | |
| ASCBase Model=Qwen3-4B, Type=Handcrafted2026.05 | 44 | — | — | 388.9 | — | |
| SC@64Base Model=Qwen3-4B, Type=Parallel2026.02 | 43.6 | 0.1168 | — | 33,900 | — | |
| ASCBase Model=Qwen3-4B, Type=Seq.2026.02 | 43.6 | 0.0487 | — | 487,300 | — | |
| ESCBase Model=Qwen3-4B, Type=Hybrid2026.02 | 43.6 | 0.099 | — | 174,000 | — | |
| SC @ 64Base Model=Qwen3-4B, Type=Handcrafted2026.05 | 43.6 | — | — | 1,168.3 | — | |
| ESCBase Model=Qwen3-4B, Type=Handcrafted2026.05 | 43.6 | — | — | 990.2 | — | |
| SC@64 + SACBase Model=Qwen3-8B, Type=Parallel2026.02 | 42.7 | 0.0897 | — | 28,500 | — | |
| SC@64 + SACBase Model=Qwen3-4B, Type=Parallel2026.02 | 41.9 | 0.0863 | — | 27,100 | — | |
| AutoTTS (β=1.0)Base Model=Qwen3-1.7B, Type=Discovered2026.05 | 32.1 | — | — | 679.6 | — | |
| AutoTTS (β=0.5)Base Model=Qwen3-1.7B, Type=Discovered2026.05 | 30.5 | — | — | 359.1 | — | |
| SC@64Base Model=Qwen3-1.7B, Type=Parallel2026.02 | 24.2 | 0.1133 | — | 32,400 | — | |
| ASCBase Model=Qwen3-1.7B, Type=Seq.2026.02 | 24.2 | 0.0586 | — | 586,300 | — | |
| ESCBase Model=Qwen3-1.7B, Type=Hybrid2026.02 | 24.2 | 0.1014 | — | 174,900 | — | |
| SC @ 64Base Model=Qwen3-1.7B, Type=Handcrafted2026.05 | 24.2 | — | — | 1,132.9 | — | |
| ASCBase Model=Qwen3-1.7B, Type=Handcrafted2026.05 | 24.2 | — | — | 586.3 | — |