Commonsense Reasoning on HELLASWAG (test)
95.6AccuracyHuman
Evaluation results
| Method | Links | |
|---|---|---|
| Human2020.04 | 95.6 | |
| UNICORNsetting=Supervised2022.01 | 93.9 | |
| ALUMBackbone=ROBERTA-LARGE2020.04 | 85.6 | |
| ROBERTA_LARGE2020.04 | 85.2 | |
| MT-NLGsetting=Few-shot2022.01 | 82.42 | |
| MT-NLGsetting=Zero-shot2022.01 | 80.24 | |
| MT-NLGsetting=One-shot2022.01 | 80.2 | |
| GPT-3setting=Few-shot2022.01 | 79.3 | |
| Gophersetting=Zero-shot2022.01 | 79.2 | |
| LM-Cocktail2Fine-tune on=Hellaswag2023.11 | 79 | |
| GPT-3setting=Zero-shot2022.01 | 78.9 | |
| LM-Cocktail10Fine-tune on=Hellaswag2023.11 | 78.61 | |
| GPT-3setting=One-shot2022.01 | 78.1 | |
| Fine-tunedFine-tune on=Hellaswag2023.11 | 77.2 | |
| LlamaModel Variant=Base model2023.11 | 71.58 | |
| BERT_LARGE2020.04 | 47.3 | |
| GPT2020.04 | 41.7 | |
| Muon+CWDOptimizer=Muon+CWD, Model Parameters=1B, Training Tokens=100B, Codebase=OLMo2025.10 | 41 | |
| AdamW+CWDOptimizer=AdamW+CWD, Model Parameters=1B, Training Tokens=100B, Codebase=OLMo2025.10 | 40 | |
| MuonOptimizer=Muon, Model Parameters=1B, Training Tokens=100B, Codebase=OLMo2025.10 | 39 | |
| AdamWOptimizer=AdamW, Model Parameters=1B, Training Tokens=100B, Codebase=OLMo2025.10 | 38 |