CSR (Commonsense Reasoning Suite)
67.3CSR AccuracyMultiplication-Only Matrix Inversion Approximation
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Multiplication-Only Matrix Inversion ApproximationModel scale=9B2026.06 | 67.3 | — | — | — | — | — | — | — | |
| Flash Linear AttentionModel scale=9B2026.06 | 67.28 | — | — | — | — | — | — | — | |
| Flash Linear AttentionModel scale=4B2026.06 | 65.48 | — | — | — | — | — | — | — | |
| FLAMODEL=4B, Precision=FP2026.06 | 65.48 | — | — | — | — | — | — | — | |
| Multiplication-Only Matrix Inversion ApproximationModel scale=4B2026.06 | 65.45 | — | — | — | — | — | — | — | |
| FLAMODEL=4B, Precision=W4A162026.06 | 65.05 | — | — | — | — | — | — | — | |
| Multiplication-Only Matrix Inversion ApproximationMODEL=4B, Precision=W4A162026.06 | 65.03 | — | — | — | — | — | — | — | |
| Flash Linear AttentionModel scale=2B2026.06 | 56.99 | — | — | — | — | — | — | — | |
| Multiplication-Only Matrix Inversion ApproximationModel scale=2B2026.06 | 56.93 | — | — | — | — | — | — | — | |
| Multiplication-Only Matrix Inversion ApproximationModel scale=0.8B2026.06 | 51.17 | — | — | — | — | — | — | — | |
| Flash Linear AttentionModel scale=0.8B2026.06 | 51.11 | — | — | — | — | — | — | — | |
| FLAMODEL=0.8B, Precision=FP2026.06 | 51.11 | — | — | — | — | — | — | — | |
| Multiplication-Only Matrix Inversion ApproximationMODEL=0.8B, Precision=W4A162026.06 | 49.21 | — | — | — | — | — | — | — | |
| FLAMODEL=0.8B, Precision=W4A162026.06 | 49.03 | — | — | — | — | — | — | — | |
| GDNNumber of Parameters=360M, Training Tokens=10B2026.01 | — | 46.9 | 51.3 | 64.5 | 25.4 | 31.4 | 47.3 | 62 | |
| GLANumber of Parameters=340M, Training Tokens=10B2026.01 | — | 46 | 50 | 62.9 | 25.5 | 31 | 45.8 | 60.8 | |
| MambaNumber of Parameters=390M, Training Tokens=10B2026.01 | — | 46.4 | 50.5 | 64.1 | 24.9 | 32.4 | 48.3 | 58.2 | |
| Mamba2Number of Parameters=340M, Training Tokens=10B2026.01 | — | 47 | 49.8 | 64.6 | 25.5 | 32 | 49.2 | 61.2 | |
| MHLANumber of Parameters=340M, Training Tokens=10B2026.01 | — | 47.1 | 51.3 | 64.4 | 25.9 | 33.4 | 46.5 | 61.3 | |
| RFID-MoEBackbone=Qwen3-30B-A3B-2507, Ratio=20%, Evaluation Protocol=Zero-shot2026.02 | — | 72 | — | — | — | — | — | — | |
| RFID-MoEBackbone=DeepSeekMoE-16B-Base, Ratio=40%, Evaluation Protocol=Zero-shot2026.02 | — | 62 | — | — | — | — | — | — | |
| RFID-MoEBackbone=Qwen2-57B-A14B, Ratio=40%, Evaluation Protocol=Zero-shot2026.02 | — | 67 | — | — | — | — | — | — | |
| RFID-MoEBackbone=Qwen1.5-MoE-A2.7B, Ratio=40%, Evaluation Protocol=Zero-shot2026.02 | — | 62 | — | — | — | — | — | — | |
| Transformer++Number of Parameters=340M, Training Tokens=10B2026.01 | — | 46.8 | 49.6 | 64.4 | 25.7 | 32.8 | 48.1 | 60.5 |