Multitask Language Understanding on MMLU-Pro (exact_match)
18Exact Match AccuracyBaseline
Evaluation results
| Method | Links | |
|---|---|---|
| BaselineBase Model=Llama 3.2 1B Instruct, Extraction Method=custom-extract2026.03 | 18 | |
| Vocab. TrimmingBase Model=Llama 3.2 1B Instruct, Extraction Method=custom-extract2026.03 | 18 | |
| FlashHeadBase Model=Llama 3.2 1B Instruct, Extraction Method=custom-extract2026.03 | 18 | |
| SVDSoftmaxBase Model=Llama 3.2 1B Instruct, Extraction Method=custom-extract2026.03 | 16 |