List your company
ResearchFormerly WizWand

Mathematical Reasoning on AMC12 (ECE, NLL)

0.0563Expected Calibration Error (ECE)MSV 16
0.0479880.1040940.16020.216306Mar 3, 2026
Updated 8mo ago

Evaluation results

MethodLinks
MSV 16Scale=162026.030.05630.3672
MSV 4Scale=42026.030.06380.3709
MSV 1 + WV 4Base Scale=1, Weighted Voting=42026.030.07670.4272
MSV 1 + WV 16Base Scale=1, Weighted Voting=162026.030.07670.4272
MSV 1 + WV 64Base Scale=1, Weighted Voting=642026.030.07670.4272
MSV 1Scale=12026.030.08620.4532
MSV 64Scale=642026.030.10540.4293
ProbeVerifier Type=Probe, Number of Sequences (N)=12026.030.10640.468
MSV 64Verifier Type=MSV, Number of Sequences (N)=642026.030.10850.4986
Probe + WV 4Weighted Voting=42026.030.12720.441
Probe + WV 16Weighted Voting=162026.030.12720.441
Probe + WV 64Weighted Voting=642026.030.12720.441
MSV 1Verifier Type=MSV, Number of Sequences (N)=12026.030.12790.4913
ProbeContext=Standard Comparison2026.030.12920.4841
MSV 4Verifier Type=MSV, Number of Sequences (N)=42026.030.13060.4793
MSV 16Verifier Type=MSV, Number of Sequences (N)=162026.030.13170.4724
Probe + WV 64Verifier Type=Probe, Number of Sequences (N)=64, Weighted Voting (WV)=true2026.030.16110.6564
MSV 1 + WV 64Verifier Type=MSV, Training Sequences=1, Number of Sequences (N)=64, Weighted Voting (WV)=true2026.030.16640.6646
Probe + WV 16Verifier Type=Probe, Number of Sequences (N)=16, Weighted Voting (WV)=true2026.030.180.812
MSV 1 + WV 16Verifier Type=MSV, Training Sequences=1, Number of Sequences (N)=16, Weighted Voting (WV)=true2026.030.18240.8198
Probe + WV 4Verifier Type=Probe, Number of Sequences (N)=4, Weighted Voting (WV)=true2026.030.26331.4417
MSV 1 + WV 4Verifier Type=MSV, Training Sequences=1, Number of Sequences (N)=4, Weighted Voting (WV)=true2026.030.26411.4449