Physical Reasoning on PIQA (Accuracy & Speedup)
91.3AccuracyMistral Small 24B Base 2501
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Mistral Small 24B Base 2501Classifier=Self-labeled2026.01 | 91.3 | — | 8.2 | |
| Mistral Small 24B Base 2501Classifier=Majority-labeled2026.01 | 91.3 | — | 8.1 | |
| Llama 3.1 8BClassifier=Self-labeled2026.01 | 91 | — | 9.7 | |
| Llama 3.1 8BClassifier=Majority-labeled2026.01 | 90.9 | — | 9.7 | |
| Mistral Nemo Base 2407Classifier=Self-labeled2026.01 | 89.5 | — | 7.3 | |
| Mistral Nemo Base 2407Classifier=Majority-labeled2026.01 | 89.2 | — | 7 | |
| Mistral 7B v0.3Classifier=Self-labeled2026.01 | 88.8 | — | 7.6 | |
| Mistral Small 24B Inst 2501Classifier=Self-labeled2026.01 | 88.8 | — | 3.8 | |
| Mistral 7B v0.3Classifier=Majority-labeled2026.01 | 88.6 | — | 7.4 | |
| Llama 3.2 1BClassifier=Self-labeled2026.01 | 88.4 | — | 11.7 | |
| Llama 3.2 1BClassifier=Majority-labeled2026.01 | 88.4 | — | 11.7 | |
| Mistral Small 24B Inst 2501Classifier=Majority-labeled2026.01 | 88.3 | — | 3.3 | |
| Llama 3.2 3BClassifier=Self-labeled2026.01 | 86.6 | — | 8.8 | |
| Llama 3.2 3BClassifier=Majority-labeled2026.01 | 86.4 | — | 8.6 | |
| Mistral Nemo Inst 2407Classifier=Self-labeled2026.01 | 86 | — | 4.3 | |
| Mistral Nemo Inst 2407Classifier=Majority-labeled2026.01 | 85.4 | — | 3.7 | |
| Mistral 7B Inst v0.3Classifier=Majority-labeled2026.01 | 83.4 | — | 0.7 | |
| Llama 3.1 8B InstClassifier=Self-labeled2026.01 | 82.2 | — | 2 | |
| Llama 3.2 1B InstClassifier=Self-labeled2026.01 | 82 | — | 7.8 | |
| Mistral 7B Inst v0.3Classifier=Self-labeled2026.01 | 81.5 | — | -1.1 | |
| Llama 3.2 1B InstClassifier=Majority-labeled2026.01 | 80.6 | — | 6.4 | |
| Llama 3.1 8B InstClassifier=Majority-labeled2026.01 | 80.1 | — | -0.2 | |
| Llama 3.2 3B InstClassifier=Self-labeled2026.01 | 79.3 | — | 2.5 | |
| Llama 3.2 3B InstClassifier=Majority-labeled2026.01 | 74 | — | -2.8 | |
| Exp-k=2 (7.5, 2.5)Schedule Type=Exponential, k=2, Thresholds (τ_high, τ_low)=(7.5, 2.5)2025.12 | 73.25 | 1.11 | — | |
| Linear (7.5, 0)Schedule Type=Linear, Thresholds (τ_high, τ_low)=(7.5, 0)2025.12 | 73.14 | 1.14 | — | |
| Exp-k=2 (7.5, 0)Schedule Type=Exponential, k=2, Thresholds (τ_high, τ_low)=(7.5, 0)2025.12 | 73.11 | 1.21 | — | |
| Linear (7.5, 2.5)Schedule Type=Linear, Thresholds (τ_high, τ_low)=(7.5, 2.5)2025.12 | 73.09 | 1.1 | — | |
| Cosine (7.5, 2.5)Schedule Type=Cosine, Thresholds (τ_high, τ_low)=(7.5, 2.5)2025.12 | 73.08 | 1.11 | — | |
| Cosine (7.5, 0)Schedule Type=Cosine, Thresholds (τ_high, τ_low)=(7.5, 0)2025.12 | 73.06 | 1.14 | — | |
| BaselineSchedule=Standard2025.12 | 73.02 | 1 | — | |
| Exp-k=16 (7.5, 2.5)Schedule Type=Exponential, k=16, Thresholds (τ_high, τ_low)=(7.5, 2.5)2025.12 | 72.78 | 1.55 | — | |
| ProphetSchedule=Prophet2025.12 | 72.33 | 1.01 | — | |
| Exp-k=16 (7.5, 0)Schedule Type=Exponential, k=16, Thresholds (τ_high, τ_low)=(7.5, 0)2025.12 | 58.81 | 2.5 | — |