Commonsense Reasoning on Commonsense Reasoning Suite (MMLU, ARC, PIQA, HellaSwag, OpenBookQA, Winogrande)
63.84Average AccuracyLLaMA-3-8B (GQA)
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| LLaMA-3-8B (GQA)Tokens=15T, KV Mem.=–, Zero-shot=true2026.05 | 63.84 | 46.2 | 65.75 | 80.47 | 76.2 | 45.6 | 68.82 | |
| TransMLATokens=30B, KV Mem.=−71.875%, Zero-shot=true2026.05 | 63.39 | 46.18 | 66.3 | 80.3 | 76.33 | 45 | 66.22 | |
| TransMLATokens=0, KV Mem.=−71.875%, Zero-shot=true2026.05 | 54.13 | 36.38 | 52.84 | 73.83 | 64.34 | 37 | 60.38 | |
| TransGQLATokens=0, KV Mem.=−71.875%, Zero-shot=true2026.05 | 54.13 | 36.38 | 52.84 | 73.83 | 64.34 | 37 | 60.38 |