Logical Reasoning on HLE
0.7226AccuracyPass
Evaluation Results
| Method | Links | |
|---|---|---|
| PassK=82026.04 | 0.7226 | |
| PassK=42026.04 | 0.6528 | |
| AggAgentK=82026.04 | 0.6 | |
| BoNK=82026.04 | 0.5806 | |
| SummAggK=82026.04 | 0.5806 | |
| WMVK=82026.04 | 0.5742 | |
| AggAgentK=42026.04 | 0.5699 | |
| PassK=22026.04 | 0.5661 | |
| SummAggK=42026.04 | 0.5634 | |
| SolAggK=82026.04 | 0.5419 | |
| MVK=82026.04 | 0.5355 | |
| AggAgentK=22026.04 | 0.529 | |
| BoNK=42026.04 | 0.5276 | |
| WMVK=42026.04 | 0.5259 | |
| SolAggK=42026.04 | 0.5097 | |
| BoNK=22026.04 | 0.4975 | |
| SummAggK=22026.04 | 0.4968 | |
| WMVK=22026.04 | 0.4952 | |
| MVK=42026.04 | 0.4909 | |
| SolAggK=22026.04 | 0.4753 | |
| FewToolK=42026.04 | 0.4748 | |
| FewToolK=22026.04 | 0.4677 | |
| FewToolK=82026.04 | 0.4645 | |
| PassK=12026.04 | 0.4573 | |
| MVK=22026.04 | 0.4514 | |
| VERGE Fullbackbone=GPT-120B2026.01 | 0.305 | |
| VERGE w/o MCSbackbone=GPT-120B2026.01 | 0.21 | |
| VERGE Fullbackbone=GPT-20B2026.01 | 0.199 | |
| Qwen-3.5 27BSpeedup @32k=0.55x, Speedup @16k=0.5x2026.04 | 0.18 | |
| VERGE Fullbackbone=Sonnet-3.72026.01 | 0.172 | |
| VERGE w/o MCSbackbone=Sonnet-3.72026.01 | 0.167 | |
| VERGE w/o Rtbackbone=GPT-120B2026.01 | 0.152 | |
| SCbackbone=GPT-20B2026.01 | 0.15 | |
| VERGE w/o Rtbackbone=Sonnet-3.72026.01 | 0.147 | |
| CoTbackbone=GPT-120B2026.01 | 0.142 | |
| SCbackbone=GPT-120B2026.01 | 0.14 | |
| VERGE w/o Rtbackbone=GPT-20B2026.01 | 0.137 | |
| SRbackbone=GPT-120B2026.01 | 0.128 | |
| VERGE w/o MCSbackbone=GPT-20B2026.01 | 0.122 | |
| Falcon-H1R 7BSpeedup @32k=4.61x, Speedup @16k=3.4x2026.04 | 0.111 | |
| Nemotron-3-Nano 30BSpeedup @32k=4.09x, Speedup @16k=2.8x2026.04 | 0.106 | |
| Reg|Lklhd–26Speedup @32k=2.85x, Speedup @16k=1.5x2026.04 | 0.1 | |
| CoTbackbone=GPT-20B2026.01 | 0.096 | |
| Apriel-1.6Speedup @32k=1.0x, Speedup @16k=1.0x2026.04 | 0.088 | |
| SRbackbone=GPT-20B2026.01 | 0.086 | |
| all-attentionSpeedup @32k=1.0x, Speedup @16k=1.0x2026.04 | 0.082 | |
| Idealized|All–18Speedup @32k=1.99x, Speedup @16k=1.1x2026.04 | 0.069 | |
| SRbackbone=Sonnet-3.72026.01 | 0.068 | |
| Reg|Lklhd–18Speedup @32k=4.76x, Speedup @16k=2.2x2026.04 | 0.066 | |
| DSBbackbone=GPT-120B2026.01 | 0.064 | |
| Idealized|Lklhd–6Speedup @32k=6.2x, Speedup @16k=2.4x2026.04 | 0.061 | |
| Idealized|All–6Speedup @32k=6.13x, Speedup @16k=2.5x2026.04 | 0.059 | |
| CoTbackbone=Sonnet-3.72026.01 | 0.058 | |
| Apriel-H1 15BSpeedup @32k=1.97x, Speedup @16k=1.9x2026.04 | 0.057 | |
| Reg|Lklhd–13Speedup @32k=6.9x, Speedup @16k=2.7x2026.04 | 0.055 | |
| OLMo-Hybrid-Think 7BSpeedup @32k=2.51x, Speedup @16k=2.1x2026.04 | 0.051 | |
| SCbackbone=Sonnet-3.72026.01 | 0.05 | |
| S1: Distil. Idealized|All–6Speedup @32k=6.13x, Speedup @16k=2.5x2026.04 | 0.05 | |
| Reg|Lklhd–10Speedup @32k=10.69x, Speedup @16k=4.2x2026.04 | 0.045 | |
| Nemotron-Nano 12B v2Speedup @32k=5.85x, Speedup @16k=4.3x2026.04 | 0.042 | |
| DSBbackbone=GPT-20B2026.01 | 0.014 | |
| DSBbackbone=Sonnet-3.72026.01 | 0.006 |