List your company
ResearchFormerly WizWand

Logical Reasoning on HLE

0.7226AccuracyPass
-0.0226640.1708180.36430.557782Jan 27, 2026Feb 10, 2026Feb 24, 2026Mar 10, 2026Mar 24, 2026Apr 7, 2026Apr 21, 2026
Updated 6mo ago

Evaluation results

MethodLinks
PassK=82026.040.7226
PassK=42026.040.6528
AggAgentK=82026.040.6
BoNK=82026.040.5806
SummAggK=82026.040.5806
WMVK=82026.040.5742
AggAgentK=42026.040.5699
PassK=22026.040.5661
SummAggK=42026.040.5634
SolAggK=82026.040.5419
MVK=82026.040.5355
AggAgentK=22026.040.529
BoNK=42026.040.5276
WMVK=42026.040.5259
SolAggK=42026.040.5097
BoNK=22026.040.4975
SummAggK=22026.040.4968
WMVK=22026.040.4952
MVK=42026.040.4909
SolAggK=22026.040.4753
FewToolK=42026.040.4748
FewToolK=22026.040.4677
FewToolK=82026.040.4645
PassK=12026.040.4573
MVK=22026.040.4514
VERGE Fullbackbone=GPT-120B2026.010.305
VERGE w/o MCSbackbone=GPT-120B2026.010.21
VERGE Fullbackbone=GPT-20B2026.010.199
Qwen-3.5 27BSpeedup @32k=0.55x, Speedup @16k=0.5x2026.040.18
VERGE Fullbackbone=Sonnet-3.72026.010.172
VERGE w/o MCSbackbone=Sonnet-3.72026.010.167
VERGE w/o Rtbackbone=GPT-120B2026.010.152
SCbackbone=GPT-20B2026.010.15
VERGE w/o Rtbackbone=Sonnet-3.72026.010.147
CoTbackbone=GPT-120B2026.010.142
SCbackbone=GPT-120B2026.010.14
VERGE w/o Rtbackbone=GPT-20B2026.010.137
SRbackbone=GPT-120B2026.010.128
VERGE w/o MCSbackbone=GPT-20B2026.010.122
Falcon-H1R 7BSpeedup @32k=4.61x, Speedup @16k=3.4x2026.040.111
Nemotron-3-Nano 30BSpeedup @32k=4.09x, Speedup @16k=2.8x2026.040.106
Reg|Lklhd–26Speedup @32k=2.85x, Speedup @16k=1.5x2026.040.1
CoTbackbone=GPT-20B2026.010.096
Apriel-1.6Speedup @32k=1.0x, Speedup @16k=1.0x2026.040.088
SRbackbone=GPT-20B2026.010.086
all-attentionSpeedup @32k=1.0x, Speedup @16k=1.0x2026.040.082
Idealized|All–18Speedup @32k=1.99x, Speedup @16k=1.1x2026.040.069
SRbackbone=Sonnet-3.72026.010.068
Reg|Lklhd–18Speedup @32k=4.76x, Speedup @16k=2.2x2026.040.066
DSBbackbone=GPT-120B2026.010.064
Idealized|Lklhd–6Speedup @32k=6.2x, Speedup @16k=2.4x2026.040.061
Idealized|All–6Speedup @32k=6.13x, Speedup @16k=2.5x2026.040.059
CoTbackbone=Sonnet-3.72026.010.058
Apriel-H1 15BSpeedup @32k=1.97x, Speedup @16k=1.9x2026.040.057
Reg|Lklhd–13Speedup @32k=6.9x, Speedup @16k=2.7x2026.040.055
OLMo-Hybrid-Think 7BSpeedup @32k=2.51x, Speedup @16k=2.1x2026.040.051
SCbackbone=Sonnet-3.72026.010.05
S1: Distil. Idealized|All–6Speedup @32k=6.13x, Speedup @16k=2.5x2026.040.05
Reg|Lklhd–10Speedup @32k=10.69x, Speedup @16k=4.2x2026.040.045
Nemotron-Nano 12B v2Speedup @32k=5.85x, Speedup @16k=4.3x2026.040.042
DSBbackbone=GPT-20B2026.010.014
DSBbackbone=Sonnet-3.72026.010.006