Language Understanding on MMLU (out-of-domain)
48.41Logic AccuracyMoRAM
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| MoRAMBackbone=Llama-3.1-8B, Trainable Parameters (M) (added/activated)=41.9/26.2, % Trainable Parameters=0.52%/0.32%2025.06 | 48.41 | 70.09 | 82.46 | 73.53 | 68.18 | 55.53 | 46.08 | 41.96 | 63.7 | |
| LoRA (r = 32)Backbone=Llama-3.1-8B, Rank=32, Trainable Parameters (M)=83.9, % Trainable Parameters=1.03%2025.06 | 42.85 | 69.45 | 81.87 | 72.27 | 66.36 | 55.44 | 45.1 | 39.29 | 63.59 | |
| Llama-3.1-8BBackbone=Llama-3.1-8B2025.06 | 42.06 | 71.06 | 83.63 | 70.17 | 68.18 | 54.84 | 39.22 | 40.18 | 63.45 | |
| LoRA (r = 16)Backbone=Llama-3.1-8B, Rank=16, Trainable Parameters (M)=41.9, % Trainable Parameters=0.52%2025.06 | 41.27 | 68.49 | 80.7 | 72.69 | 66.36 | 54.96 | 44.11 | 38.39 | 63.35 | |
| LoRA (r = 4)Backbone=Llama-3.1-8B, Rank=4, Trainable Parameters (M)=10.5, % Trainable Parameters=0.13%2025.06 | 39.68 | 70.09 | 81.87 | 71.43 | 65.45 | 54.77 | 45.1 | 40.17 | 63.28 | |
| LoRA (r = 8)Backbone=Llama-3.1-8B, Rank=8, Trainable Parameters (M)=21.0, % Trainable Parameters=0.26%2025.06 | 39.68 | 70.74 | 81.87 | 71.85 | 64.54 | 54.17 | 42.16 | 39.29 | 63.03 |