Reasoning and comprehension on MMLU
26.7MMLU Reasoning & Comprehension AccuracyOPT-1.3B FFF
Evaluation Results
| Method | Links | |
|---|---|---|
| OPT-1.3B FFFFew-shot (5-shot)=true, Model Depth (d)=6, Training tokens=26B, Fine-tuning (FT)=true2026.03 | 26.7 | |
| TinyLlama v1.1Few-shot (5-shot)=true, Training tokens=2T2026.03 | 26.58 | |
| OPT-1.3B FFFFew-shot (5-shot)=true, Model Depth (d)=4, Training tokens=26B, Fine-tuning (FT)=true2026.03 | 26.4 | |
| OPT-1.3B FFFew-shot (5-shot)=true, Training configuration=retrained2026.03 | 25.87 | |
| OPT-1.3B FFFFew-shot (5-shot)=true, Model Depth (d)=6, Training tokens=26B, Fine-tuning (FT)=false2026.03 | 25.8 | |
| Pythia-1.0BFew-shot (5-shot)=true, Training tokens=1T2026.03 | 25.7 | |
| OPT-1.3B FFFFew-shot (5-shot)=true, Model Depth (d)=4, Training tokens=26B, Fine-tuning (FT)=false2026.03 | 25.51 | |
| Pythia-1.4BFew-shot (5-shot)=true, Training tokens=1T2026.03 | 25.41 | |
| TinyLlama v1.0Few-shot (5-shot)=true, Training tokens=2T2026.03 | 25.34 | |
| OPT-1.3BFew-shot (5-shot)=true, Training tokens=300B2026.03 | 24.9 |