Mathematical Reasoning on AMC12 (ECE, NLL)
0.0563Expected Calibration Error (ECE)MSV 16
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MSV 16Scale=162026.03 | 0.0563 | 0.3672 | |
| MSV 4Scale=42026.03 | 0.0638 | 0.3709 | |
| MSV 1 + WV 4Base Scale=1, Weighted Voting=42026.03 | 0.0767 | 0.4272 | |
| MSV 1 + WV 16Base Scale=1, Weighted Voting=162026.03 | 0.0767 | 0.4272 | |
| MSV 1 + WV 64Base Scale=1, Weighted Voting=642026.03 | 0.0767 | 0.4272 | |
| MSV 1Scale=12026.03 | 0.0862 | 0.4532 | |
| MSV 64Scale=642026.03 | 0.1054 | 0.4293 | |
| ProbeVerifier Type=Probe, Number of Sequences (N)=12026.03 | 0.1064 | 0.468 | |
| MSV 64Verifier Type=MSV, Number of Sequences (N)=642026.03 | 0.1085 | 0.4986 | |
| Probe + WV 4Weighted Voting=42026.03 | 0.1272 | 0.441 | |
| Probe + WV 16Weighted Voting=162026.03 | 0.1272 | 0.441 | |
| Probe + WV 64Weighted Voting=642026.03 | 0.1272 | 0.441 | |
| MSV 1Verifier Type=MSV, Number of Sequences (N)=12026.03 | 0.1279 | 0.4913 | |
| ProbeContext=Standard Comparison2026.03 | 0.1292 | 0.4841 | |
| MSV 4Verifier Type=MSV, Number of Sequences (N)=42026.03 | 0.1306 | 0.4793 | |
| MSV 16Verifier Type=MSV, Number of Sequences (N)=162026.03 | 0.1317 | 0.4724 | |
| Probe + WV 64Verifier Type=Probe, Number of Sequences (N)=64, Weighted Voting (WV)=true2026.03 | 0.1611 | 0.6564 | |
| MSV 1 + WV 64Verifier Type=MSV, Training Sequences=1, Number of Sequences (N)=64, Weighted Voting (WV)=true2026.03 | 0.1664 | 0.6646 | |
| Probe + WV 16Verifier Type=Probe, Number of Sequences (N)=16, Weighted Voting (WV)=true2026.03 | 0.18 | 0.812 | |
| MSV 1 + WV 16Verifier Type=MSV, Training Sequences=1, Number of Sequences (N)=16, Weighted Voting (WV)=true2026.03 | 0.1824 | 0.8198 | |
| Probe + WV 4Verifier Type=Probe, Number of Sequences (N)=4, Weighted Voting (WV)=true2026.03 | 0.2633 | 1.4417 | |
| MSV 1 + WV 4Verifier Type=MSV, Training Sequences=1, Number of Sequences (N)=4, Weighted Voting (WV)=true2026.03 | 0.2641 | 1.4449 |