Zero-shot Language Evaluation on 7-Task Suite (BoolQ, PIQA, HellaSwag, WinoGrande, ARC-E, ARC-C, OBQA) (test)
62.9BoolQ AccuracyMamba-2 FP16
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Mamba-2 FP16Bits=162026.06 | 62.9 | 73.7 | 59.9 | 61.1 | 60.4 | 33.2 | 37.4 | 55.5 | |
| Bi-Mamba 1.3BBits=1, Type=QAT scratch, Train tokens=105B2026.06 | 60 | 68.8 | 47.3 | 55.9 | 48 | 26.3 | 32.2 | 48.4 | |
| BitNet b1.58 1.3BBits=1.58, Type=QAT scratch, Train tokens=100B2026.06 | 56.7 | 68.8 | 37.7 | 55.8 | 54.9 | 24.2 | 19.6 | 45.4 | |
| Ternary MambaBits=1.58, Type=QAT, Train tokens=102M2026.06 | 56 | 69.3 | 47.5 | 56.3 | 48.2 | 25.9 | 33.8 | 48.1 |