List your company
ResearchFormerly WizWand

Multi-task Language Understanding on MMLU v1 (test)

66.6AccuracyLLaMA-3-8B
31.96840.95949.9558.941Jul 11, 2025
Updated 6mo ago

Evaluation results

MethodLinks
LLaMA-3-8BTraining Tokens (B)=15000, Evaluation Protocol=5-shot, Model Architecture Category=Transformer2025.0766.6
Gemma-7BTraining Tokens (B)=6000, Evaluation Protocol=5-shot, Model Architecture Category=Transformer2025.0762.9
Mistral-7BTraining Tokens (B)=8000*, Evaluation Protocol=5-shot, Model Architecture Category=Transformer2025.0762.4
LLaMA-3-8B-LizardTraining Tokens (B)=0.04, Evaluation Protocol=5-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Unbounded2025.0761.2
Mistral-7B-LizardTraining Tokens (B)=0.04, Evaluation Protocol=5-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Unbounded2025.0760.8
LLaMA-3-8B-LoLCATsTraining Tokens (B)=0.04, Evaluation Protocol=5-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.0752.8
Mistral-7B-LoLCATsTraining Tokens (B)=0.04, Evaluation Protocol=5-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.0751.4
Liger-GLA-Llama-3-8BTraining Tokens (B)=0.02, Evaluation Protocol=5-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.0743.4
Mamba2-LLaMA-3-8BTraining Tokens (B)=20, Evaluation Protocol=5-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Unbounded2025.0743.2
TransNormerLLM-7BTraining Tokens (B)=1400, Evaluation Protocol=5-shot, Model Architecture Category=Subquadratic2025.0743.1
Griffin-7BTraining Tokens (B)=300, Evaluation Protocol=5-shot, Model Architecture Category=Subquadratic2025.0739.3
Liger-GLA-Mistral-7BTraining Tokens (B)=0.02, Evaluation Protocol=5-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.0736.3
Hawk-7BTraining Tokens (B)=300, Evaluation Protocol=5-shot, Model Architecture Category=Subquadratic2025.0735
Mistral-7B-SUPRATraining Tokens (B)=100, Evaluation Protocol=5-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.0734.2
Mamba-7BTraining Tokens (B)=1200, Evaluation Protocol=5-shot, Model Architecture Category=Subquadratic2025.0733.3