Mathematical Reasoning on GSM8K (test) (Consensus Metrics)
96.13Accuracy (Raw Consensus)Raw consensus
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Raw consensusModel=Phi-4, Policy=no-framing majority baseline, Sample count (N)=13192026.05 | 96.13 | — | — | — | — | — | — | |
| ARBITER-∆Model=Phi-4, Policy=fixed framed+guided parameter-free, Sample count (N)=13192026.05 | 96.13 | 96.44 | 0.31 | 9 | 6 | 2 | 4 | |
| Raw consensusModel=Qwen3-4B, Policy=no-framing majority baseline, Sample count (N)=13192026.05 | 94.54 | — | — | — | — | — | — | |
| ARBITER-∆Model=Qwen3-4B, Policy=fixed framed+guided parameter-free, Sample count (N)=13192026.05 | 94.54 | 94.84 | 0.3 | 16 | 9 | 5 | 4 | |
| Raw consensusModel=Llama-3.1-8B, Policy=no-framing majority baseline, Sample count (N)=13192026.05 | 91.81 | — | — | — | — | — | — | |
| ARBITER-∆Model=Llama-3.1-8B, Policy=fixed framed+guided parameter-free, Sample count (N)=13192026.05 | 91.81 | 92.65 | 0.84 | 35 | 20 | 9 | 11 |