Commonsense Reasoning on WinoGrande (WG)
77.5AccuracyQwen3-8B
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen3-8BType=AR2026.06 | 77.5 | |
| TWLABackbone=LLaMA2-70B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 77.35 | |
| Qwen2.5-7BType=AR2026.06 | 76.1 | |
| MiMo-7BType=AR2026.06 | 75.9 | |
| Dream-v0-7BType=Diffusion2026.06 | 74.5 | |
| LLaDA-8BType=Diffusion2026.06 | 74.5 | |
| DreamReasoner-8BType=Block Diffusion2026.06 | 74.2 | |
| OLMo3-7BType=AR2026.06 | 73.5 | |
| FP16Bits=16.02026.07 | 72.8 | |
| HQQBits=4.02026.07 | 72.3 | |
| TASA b4.0Bits=4.02026.07 | 71.9 | |
| AWQBits=4.02026.07 | 71.3 | |
| TWLABackbone=Qwen3-32B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 71.27 | |
| GPTQBits=4.02026.07 | 70.9 | |
| TWLABackbone=Qwen3-14B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 70.64 | |
| SpQR†Bits=4.12 (W3)2026.07 | 70.6 | |
| RTNBits=4.02026.07 | 70.5 | |
| TWLABackbone=LLaMA2-13B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 68.75 | |
| TASA b3.0Bits=3.02026.07 | 68.4 | |
| TWLABackbone=LLaMA3-8B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 68.35 | |
| HQQBits=3.02026.07 | 68.2 | |
| OWQBits=3.02026.07 | 66.4 | |
| GPTQBits=3.02026.07 | 65.7 | |
| TWLABackbone=LLaMA2-7B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 65.19 | |
| AuroraModel Size=1.1B, Evaluation Protocol=5-shot2026.06 | 64.9 | |
| NorMuonModel Size=1.1B, Evaluation Protocol=5-shot2026.06 | 64.8 | |
| TWLABackbone=Qwen3-8B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 64.72 | |
| MuonModel Size=1.1B, Evaluation Protocol=5-shot2026.06 | 63.1 | |
| U-NorMuonModel Size=1.1B, Evaluation Protocol=5-shot2026.06 | 61.9 | |
| RTNBits=3.02026.07 | 56.1 | |
| NorMuonModel Size=340M, Evaluation Protocol=5-shot2026.06 | 54.9 | |
| U-NorMuonModel Size=340M, Evaluation Protocol=5-shot2026.06 | 54.9 | |
| AuroraModel Size=340M, Evaluation Protocol=5-shot2026.06 | 54.4 | |
| MuonModel Size=340M, Evaluation Protocol=5-shot2026.06 | 51.7 | |
| ADAMS (DENSE all-reduce)Protocol=Zero-shot, Model=GPT-345M, Strategy=DENSE all-reduce2026.07 | 51.54 | |
| ADAMW (DENSE all-reduce)Protocol=Zero-shot, Model=GPT-345M, Strategy=DENSE all-reduce2026.07 | 51.38 | |
| SCAPE (d = 0.1)Protocol=Zero-shot, Model=GPT-345M, d=0.12026.07 | 51.38 | |
| SCAPE (d = 0.01)Protocol=Zero-shot, Model=GPT-345M, d=0.012026.07 | 50.67 |