Language Understanding on PiQA, ARC, HellaSwag, WinoGrande, MMLU
75.2Aggregate AccuracyLLaMA-3-8B-Lizard
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LLaMA-3-8B-LizardTraining Tokens (B)=0.04, Attention configuration=Linearized (Keep 50% Full Attn.)2025.07 | 75.2 | 73.5 | |
| Mamba2-LLaMA-3Training Tokens (B)=20, Attention configuration=Linearized (Keep 50% Full Attn.)2025.07 | 73.9 | 71 | |
| LLaMA-3-8BTraining Tokens (B)=15000, Attention configuration=Softmax2025.07 | 73.1 | 72 | |
| Zamba-7BTraining Tokens (B)=1000, Attention configuration=Hybrid Softmax2025.07 | 71.8 | 69.5 | |
| StripedHyena-Nous-7BTraining Tokens (B)=–, Attention configuration=Hybrid Softmax2025.07 | 67.8 | 60.8 |