Commonsense Reasoning on Winogrande (Average Score)
73.88Winogrande ScoreDPO
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DPO2025.09 | 73.88 | 70.68 | |
| SFT Model2025.09 | 73.72 | 70.28 | |
| SPPO2025.09 | 73.48 | 70.19 | |
| INPO2025.09 | 73.48 | 70.25 | |
| HT-MNPOReward Model=ArmoRM-Llama32025.09 | 73.48 | 70.83 | |
| HT-MNPOReward Model=Skywork-Reward-V22025.09 | 73.4 | 71.8 | |
| HT-MNPOReward Model=Athene-RM-8B2025.09 | 73.32 | 70.99 | |
| TD-MNPO2025.09 | 73.09 | 71.08 | |
| SimPO2025.09 | 72.93 | 69.6 | |
| FP32Model=LLaMA3-8B, W-A-KV=–, W. Gran.=–2026.05 | 72.85 | — | |
| Quant.npuModel=LLaMA3-8B, W-A-KV=4-8-8, W. Gran.=per-channel2026.05 | 71.9 | — | |
| executorchModel=LLaMA3-8B, W-A-KV=4-16-8, W. Gran.=per-block2026.05 | 71.82 | — | |
| QuaRotModel=LLaMA3-8B, W-A-KV=4-8-8, W. Gran.=per-channel2026.05 | 68.19 | — | |
| executorchModel=LLaMA3-8B, W-A-KV=4-8-8, W. Gran.=per-channel2026.05 | 67.56 | — | |
| SpinQuantModel=LLaMA3-8B, W-A-KV=4-8-8, W. Gran.=per-channel2026.05 | 64.48 | — | |
| Echo-180MParams=180M, Tokens=10B, Zero-shot=true2026.05 | 55.4 | — | |
| Mamba-3-MIMO-180MParams=180M, Tokens=100B, Zero-shot=true2026.05 | 52.9 | — | |
| Mamba-2-180MParams=180M, Tokens=100B, Zero-shot=true2026.05 | 52 | — | |
| Mamba-3-SISO-180MParams=180M, Tokens=100B, Zero-shot=true2026.05 | 52 | — | |
| GDN-180MParams=180M, Tokens=100B, Zero-shot=true2026.05 | 51.7 | — | |
| Transformer-180MParams=180M, Tokens=100B, Zero-shot=true2026.05 | 51.2 | — | |
| Echo-50MParams=50M, Tokens=3B, Zero-shot=true2026.05 | 49.5 | — |