Language Understanding on MMLU (Accuracy and Prompt/Response Scores)
82.1AccuracyGPT-4o-mini
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-4o-miniTask Model=GPT-4o-mini2026.03 | 82.1 | 4.388 | 4.709 | 4.5 | |
| Gemini-2.5-FlashTask Model=Gemini-2.5-Flash2026.03 | 80.3 | 4.412 | 4.605 | 4.35 | |
| LLaMA-3.1-8B-ITTask Model=LLaMA-3.1-8B-IT2026.03 | 71.1 | 4.113 | 4.463 | 4.252 | |
| Qwen-2.5-7B-ITTask Model=Qwen-2.5-7B-IT2026.03 | 68.7 | 4.188 | 4.577 | 4.302 | |
| Gemma-2-9B-ITTask Model=Gemma-2-9B-IT2026.03 | 67.3 | 4.18 | 4.431 | 4.153 | |
| Full FTParams (%)=100, Base Model=Llama-3-8B2026.03 | 66.8 | — | — | — | |
| NeuroLoRAParams (%)=0.14, Base Model=Llama-3-8B2026.03 | 66.3 | — | — | — | |
| FlyLoRAParams (%)=0.13, Base Model=Llama-3-8B2026.03 | 65.1 | — | — | — | |
| DoRAParams (%)=0.84, Base Model=Llama-3-8B2026.03 | 64.8 | — | — | — | |
| AdaLoRAParams (%)=0.81, Base Model=Llama-3-8B2026.03 | 64.5 | — | — | — | |
| LoRAParams (%)=0.83, Base Model=Llama-3-8B2026.03 | 64.2 | — | — | — | |
| MoLEParams (%)=0.35, Base Model=Llama-3-8B2026.03 | 63.9 | — | — | — | |
| SingleTrained on=GSM8K, Evaluation Protocol=Without retraining2026.05 | 52.07 | — | — | — | |
| NEXATrained on=GSM8K, Evaluation Protocol=Without retraining2026.05 | 52 | — | — | — | |
| CoTTrained on=GSM8K, Evaluation Protocol=Without retraining2026.05 | 51.27 | — | — | — | |
| AdaFuseBase Model=Llama2-7B2026.03 | 51.15 | — | — | — | |
| MoRALGranularity=layer-wise, Base Model=Llama2-7B2026.03 | 51.1 | — | — | — | |
| PESCGranularity=block-wise, Base Model=Llama2-7B2026.03 | 51.07 | — | — | — | |
| LoRABase Model=Llama2-7B2026.03 | 50.46 | — | — | — | |
| Llama2-7BMode=base, Base Model=Llama2-7B2026.03 | 48.3 | — | — | — | |
| NITPModel Scale=3B2026.05 | 44.95 | — | — | — | |
| NTPModel Scale=3B2026.05 | 43.54 | — | — | — | |
| NITPModel Scale=2B2026.05 | 40.14 | — | — | — | |
| NTPModel Scale=2B2026.05 | 37.96 | — | — | — | |
| NITPModel Scale=0.5B2026.05 | 31.01 | — | — | — | |
| NTPModel Scale=0.5B2026.05 | 30.59 | — | — | — | |
| SmolLM-1.7B-Instructshot=0-shot2026.03 | 27.1 | — | — | — | |
| DATEDGPT-INSTRUCT-2020shot=0-shot2026.03 | 25.7 | — | — | — | |
| GPT2-XLshot=0-shot2026.03 | 25.3 | — | — | — | |
| DATEDGPT-INSTRUCT-2013shot=0-shot2026.03 | 25.3 | — | — | — | |
| DATEDGPT-INSTRUCT-2018shot=0-shot2026.03 | 25.3 | — | — | — | |
| OPT-1.3Bshot=0-shot2026.03 | 25.2 | — | — | — | |
| DATEDGPT-INSTRUCT-2019shot=0-shot2026.03 | 24.9 | — | — | — | |
| TinyLlama-1.1Bshot=0-shot2026.03 | 24.8 | — | — | — | |
| DATEDGPT-INSTRUCT-2014shot=0-shot2026.03 | 24.8 | — | — | — | |
| DATEDGPT-INSTRUCT-2016shot=0-shot2026.03 | 24.8 | — | — | — | |
| DATEDGPT-INSTRUCT-2017shot=0-shot2026.03 | 24.8 | — | — | — | |
| DATEDGPT-INSTRUCT-2021shot=0-shot2026.03 | 24.4 | — | — | — | |
| DATEDGPT-INSTRUCT-2024shot=0-shot2026.03 | 24.3 | — | — | — | |
| DATEDGPT-INSTRUCT-2015shot=0-shot2026.03 | 24.1 | — | — | — | |
| DATEDGPT-INSTRUCT-2023shot=0-shot2026.03 | 23.7 | — | — | — | |
| DATEDGPT-INSTRUCT-2022shot=0-shot2026.03 | 23.5 | — | — | — | |
| Pythia-1Bshot=0-shot2026.03 | 23.1 | — | — | — |