Physical Reasoning on PIQA
79.71PIQA Normalized PerformanceFP16
Evaluation results
| Method | Links | |
|---|---|---|
| FP16Bit-width=FP16, Evaluation Protocol=zero-shot2026.05 | 79.71 | |
| QTIP 2-bit PTQBit-width=2-bit, Evaluation Protocol=zero-shot2026.05 | 77.97 | |
| BCJR-QAT-N4Bit-width=2 bpw, Evaluation Protocol=zero-shot, Configuration=N42026.05 | 76.93 | |
| Dense 1.3BArchitecture=Dense, Total Parameters=1.3B, Precision=bf16, Shot count=0-shot, Evaluation Framework=lm-eval-harness 0.4.72026.05 | 65.56 | |
| MoL 0.61B/2.08BArchitecture=MoL, Total Parameters=2.08B, Active Parameters=0.61B, Precision=bf16, Shot count=0-shot, Evaluation Framework=lm-eval-harness 0.4.72026.05 | 64.09 | |
| Dense 0.7BArchitecture=Dense, Total Parameters=0.7B, Precision=bf16, Shot count=0-shot, Evaluation Framework=lm-eval-harness 0.4.72026.05 | 63.98 | |
| DualAlpha (α)=63/64, Data repetitions=12025.12 | 40.9 | |
| AutoregressiveAlpha (α)=1, Data repetitions=12025.12 | 39.4 | |
| DualAlpha (α)=3/4, Data repetitions=322025.12 | 36.1 | |
| AutoregressiveAlpha (α)=1, Data repetitions=322025.12 | 33.3 | |
| DualAlpha (α)=1/8, Data repetitions=1282025.12 | 28.1 | |
| AutoregressiveAlpha (α)=1, Data repetitions=1282025.12 | 15.8 |