Logical Reasoning on KK All
53.6AccuracyOracle Verifier
Evaluation Results
| Method | Links | |
|---|---|---|
| Oracle VerifierModel Size=12B2026.05 | 53.6 | |
| RevisionSEModel Size=12B2026.05 | 52.8 | |
| SimpleSEModel Size=12B, Verifier threshold (τ)=0.62026.05 | 51.1 | |
| SimpleSEModel Size=12B, Verifier threshold (τ)=0.72026.05 | 50.1 | |
| SimpleSEModel Size=12B, Verifier threshold (τ)=0.52026.05 | 48 | |
| gemma-3-12b-itModel Size=12B2026.05 | 47.5 | |
| SimpleSEModel Size=12B, Verifier threshold (τ)=0.82026.05 | 47.5 | |
| Oracle VerifierModel Size=4B2026.05 | 46.6 | |
| RevisionSEModel Size=4B2026.05 | 42.2 | |
| SimpleSEModel Size=4B, Verifier threshold (τ)=0.62026.05 | 40.7 | |
| SimpleSEModel Size=4B, Verifier threshold (τ)=0.82026.05 | 39.7 | |
| SimpleSEModel Size=4B, Verifier threshold (τ)=0.72026.05 | 39.6 | |
| SimpleSEModel Size=4B, Verifier threshold (τ)=0.52026.05 | 38.4 | |
| gemma-3-4b-itModel Size=4B2026.05 | 31 | |
| Oracle VerifierModel Size=1B2026.05 | 12.5 | |
| SimpleSEModel Size=1B, Verifier threshold (τ)=0.82026.05 | 8.4 | |
| gemma-3-1b-itModel Size=1B2026.05 | 7.8 | |
| RevisionSEModel Size=1B2026.05 | 7.8 | |
| SimpleSEModel Size=1B, Verifier threshold (τ)=0.72026.05 | 6.5 | |
| SimpleSEModel Size=1B, Verifier threshold (τ)=0.52026.05 | 5.7 | |
| SimpleSEModel Size=1B, Verifier threshold (τ)=0.62026.05 | 5.6 |