Logical Reasoning on KK 4–5 ppl.
60.3AccuracyOracle Verifier
Evaluation results
| Method | Links | |
|---|---|---|
| Oracle VerifierModel Size=12B2026.05 | 60.3 | |
| RevisionSEModel Size=12B2026.05 | 58.7 | |
| SimpleSEModel Size=12B, Verifier threshold (τ)=0.72026.05 | 56.3 | |
| SimpleSEModel Size=12B, Verifier threshold (τ)=0.62026.05 | 55 | |
| SimpleSEModel Size=12B, Verifier threshold (τ)=0.82026.05 | 53.9 | |
| SimpleSEModel Size=12B, Verifier threshold (τ)=0.52026.05 | 53.7 | |
| Oracle VerifierModel Size=4B2026.05 | 52.7 | |
| gemma-3-12b-itModel Size=12B2026.05 | 51.9 | |
| RevisionSEModel Size=4B2026.05 | 46.4 | |
| SimpleSEModel Size=4B, Verifier threshold (τ)=0.62026.05 | 45.4 | |
| SimpleSEModel Size=4B, Verifier threshold (τ)=0.82026.05 | 44.6 | |
| SimpleSEModel Size=4B, Verifier threshold (τ)=0.72026.05 | 43.9 | |
| SimpleSEModel Size=4B, Verifier threshold (τ)=0.52026.05 | 39.1 | |
| gemma-3-4b-itModel Size=4B2026.05 | 31 | |
| Oracle VerifierModel Size=1B2026.05 | 10 | |
| gemma-3-1b-itModel Size=1B2026.05 | 4.9 | |
| RevisionSEModel Size=1B2026.05 | 4.7 | |
| SimpleSEModel Size=1B, Verifier threshold (τ)=0.82026.05 | 4.5 | |
| SimpleSEModel Size=1B, Verifier threshold (τ)=0.52026.05 | 3.5 | |
| SimpleSEModel Size=1B, Verifier threshold (τ)=0.72026.05 | 3.3 | |
| SimpleSEModel Size=1B, Verifier threshold (τ)=0.62026.05 | 2 |