Question Answering on ARC Easy v1 (test)
83.5AccuracyLLaMA-3-8B-Lizard
Evaluation Results
| Method | Links | |
|---|---|---|
| LLaMA-3-8B-LizardTraining Tokens (B)=0.04, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Unbounded2025.07 | 83.5 | |
| Mistral-7B-LizardTraining Tokens (B)=0.04, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Unbounded2025.07 | 83.2 | |
| Mistral-7B-LoLCATsTraining Tokens (B)=0.04, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.07 | 81.7 | |
| LLaMA-3-8B-LoLCATsTraining Tokens (B)=0.04, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.07 | 81.7 | |
| Gemma-7BTraining Tokens (B)=6000, Evaluation Protocol=Zero-shot, Model Architecture Category=Transformer2025.07 | 81.1 | |
| Liger-GLA-Llama-3-8BTraining Tokens (B)=0.02, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.07 | 81.1 | |
| Mistral-7BTraining Tokens (B)=8000*, Evaluation Protocol=Zero-shot, Model Architecture Category=Transformer2025.07 | 80.9 | |
| LLaMA-3-8BTraining Tokens (B)=15000, Evaluation Protocol=Zero-shot, Model Architecture Category=Transformer2025.07 | 80.1 | |
| Liger-GLA-Mistral-7BTraining Tokens (B)=0.02, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.07 | 78.7 | |
| Mamba-7BTraining Tokens (B)=1200, Evaluation Protocol=Zero-shot, Model Architecture Category=Subquadratic2025.07 | 77.5 | |
| RWKV-6-v2.1-7BTraining Tokens (B)=1420, Evaluation Protocol=Zero-shot, Model Architecture Category=Subquadratic2025.07 | 76.8 | |
| Mistral-7B-SUPRATraining Tokens (B)=100, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.07 | 75.9 | |
| TransNormerLLM-7BTraining Tokens (B)=1400, Evaluation Protocol=Zero-shot, Model Architecture Category=Subquadratic2025.07 | 75.4 | |
| Griffin-7BTraining Tokens (B)=300, Evaluation Protocol=Zero-shot, Model Architecture Category=Subquadratic2025.07 | 75.4 | |
| Hawk-7BTraining Tokens (B)=300, Evaluation Protocol=Zero-shot, Model Architecture Category=Subquadratic2025.07 | 74.4 | |
| Mamba2-LLaMA-3-8BTraining Tokens (B)=20, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Unbounded2025.07 | 74.1 |