Massive Multitask Language Understanding on MMLU (Subject Breakdown Scores)
36.28MMLU Humanities AccuracyCoLA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CoLAzero-shot=true, Params. (%)=0.9406, A=2, B=32025.10 | 36.28 | 32.25 | 40.88 | 46.44 | 38.63 | |
| MASAzero-shot=true, Params. (%)=0.8810, A=5, B=12025.10 | 35.96 | 32.7 | 42.31 | 46.93 | 39.05 | |
| HydraLoRAzero-shot=true, Params. (%)=0.8267, A=1, B=32025.10 | 31.52 | 28.16 | 32.95 | 38.17 | 32.55 | |
| LoRAr=16zero-shot=true, Params. (%)=0.7511, A=1, B=1, Rank=162025.10 | 29.73 | 28.01 | 30.87 | 36.76 | 31.15 | |
| DoRAr=16zero-shot=true, Params. (%)=0.7748, A=1, B=1, Rank=162025.10 | 28.25 | 26.26 | 28.53 | 33.38 | 29 | |
| LoRAr=8zero-shot=true, Params. (%)=0.3770, A=1, B=1, Rank=82025.10 | 26.14 | 24.26 | 25.58 | 28.81 | 26.19 | |
| Prompt Tuningzero-shot=true, Params. (%)=0.00082025.10 | 25.36 | 24.52 | 24.5 | 23.95 | 25.36 | |
| P-Tuningzero-shot=true, Params. (%)=0.05302025.10 | 25.08 | 25.06 | 21.81 | 24.46 | 24.44 | |
| LLaMA3.2-3Bzero-shot=true2025.10 | 24.36 | 21.15 | 21.9 | 23.98 | 23.02 |