Mathematical Reasoning Verification on AIME (Brier, AUROC)
0.0283Brier ScoreMSV64
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MSV64Backbone=DeepSeek-R1-Distill-Llama-8B, Number of sequences=642026.03 | 0.0283 | 99.23 | |
| MSV16Backbone=DeepSeek-R1-Distill-Llama-8B, Number of sequences=162026.03 | 0.0298 | 99.08 | |
| MSVBackbone=Qwen3-1.7B, Mode=thinking mode, N=642026.03 | 0.0317 | 98.78 | |
| MSVBackbone=Qwen3-1.7B, Mode=thinking mode, N=162026.03 | 0.044 | 98.25 | |
| MSVBackbone=Qwen3-1.7B, Mode=thinking mode, N=42026.03 | 0.0595 | 96.96 | |
| ProbeBackbone=DeepSeek-R1-Distill-Llama-8B2026.03 | 0.0668 | 96.68 | |
| MSV1Backbone=DeepSeek-R1-Distill-Llama-8B, Number of sequences=12026.03 | 0.077 | 97.46 | |
| MSVBackbone=Qwen3-1.7B, Mode=thinking mode, N=12026.03 | 0.077 | 95.7 | |
| ProbeBackbone=Qwen3-1.7B, Mode=thinking mode2026.03 | 0.0849 | 95.3 |