Massive Multitask Language Understanding on MMLU (Accuracy, Delta, and Forgetting Profile)
53AccuracyBase
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BaseBackbone=Qwen2.5-72B2026.04 | 53 | — | — | |
| Standard LoRABackbone=Qwen2.5-72B, Campaign=1, Number of seeds=32026.04 | 53 | 0 | — | |
| AletheiaBackbone=Qwen2.5-72B, Campaign=1, Number of seeds=32026.04 | 53 | 0 | 0 | |
| Standard LoRABackbone=Qwen2.5-14B, Campaign=1, Number of seeds=32026.04 | 52.2 | 0.8 | — | |
| AletheiaBackbone=Qwen2.5-14B, Campaign=1, Number of seeds=32026.04 | 52 | 0.7 | 0.2 | |
| BaseBackbone=Qwen2.5-14B2026.04 | 51.3 | — | — | |
| BaseBackbone=Llama-3.1-8B2026.04 | 50.8 | — | — | |
| Standard LoRABackbone=Qwen2.5-7B, Campaign=1, Number of seeds=32026.04 | 50.3 | 1.3 | — | |
| BaseBackbone=Qwen2.5-7B2026.04 | 49 | — | — | |
| AletheiaBackbone=Llama-3.1-8B, Campaign=1, Number of seeds=32026.04 | 49 | 1.8 | 0.2 | |
| Standard LoRABackbone=Llama-3.1-8B, Campaign=1, Number of seeds=32026.04 | 48.8 | 2 | — | |
| AletheiaBackbone=Qwen2.5-7B, Campaign=1, Number of seeds=32026.04 | 48.2 | 0.8 | 2.2 | |
| BaseBackbone=Qwen2.5-1.5B2026.04 | 46 | — | — | |
| Standard LoRABackbone=Qwen2.5-1.5B, Campaign=1, Number of seeds=32026.04 | 45.2 | 0.8 | — | |
| AletheiaBackbone=Qwen2.5-1.5B, Campaign=1, Number of seeds=32026.04 | 44.2 | 1.8 | 1 | |
| AletheiaBackbone=Mistral-7B, Campaign=1, Number of seeds=32026.04 | 41.3 | 1 | 1.5 | |
| BaseBackbone=Mistral-7B2026.04 | 40.3 | — | — | |
| Standard LoRABackbone=Mistral-7B, Campaign=1, Number of seeds=32026.04 | 39.8 | 0.5 | — | |
| BaseBackbone=Qwen2.5-0.5B2026.04 | 37.3 | — | — | |
| AletheiaBackbone=Qwen2.5-0.5B, Campaign=1, Number of seeds=32026.04 | 36.7 | 0.7 | 0.5 | |
| Standard LoRABackbone=Qwen2.5-0.5B, Campaign=1, Number of seeds=32026.04 | 36.2 | 1.2 | — | |
| AletheiaBackbone=Phi-3.5-mini, Campaign=1, Number of seeds=32026.04 | 26.2 | 3.2 | 0.7 | |
| Standard LoRABackbone=Phi-3.5-mini, Campaign=1, Number of seeds=32026.04 | 25.5 | 2.5 | — | |
| BaseBackbone=TinyLlama-1.1B2026.04 | 23.5 | — | — | |
| AletheiaBackbone=TinyLlama-1.1B, Campaign=1, Number of seeds=32026.04 | 23.5 | 0 | 1.8 | |
| BaseBackbone=Phi-3.5-mini2026.04 | 23 | — | — | |
| BaseBackbone=Llama-3.1-70B2026.04 | 22.7 | — | — | |
| Standard LoRABackbone=Llama-3.1-70B, Campaign=1, Number of seeds=32026.04 | 22.7 | 0 | — | |
| AletheiaBackbone=Llama-3.1-70B, Campaign=1, Number of seeds=32026.04 | 22.7 | 0 | 0 | |
| Standard LoRABackbone=TinyLlama-1.1B, Campaign=1, Number of seeds=32026.04 | 21.7 | 1.8 | — |