Natural Language Understanding on GLUE (CoLA, MRPC)
87.63CoLA AccuracyMoSLoRA
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MoSLoRABase Model=Qwen2-1.5B, activated parameters=1.31%2026.02 | 87.63 | 88.23 | |
| HydraLoRABase Model=Qwen2-1.5B, activated parameters=2.54%2026.02 | 87.54 | 88.17 | |
| LoRABase Model=Qwen2-1.5B, rank=82026.02 | 87.25 | 87.3 | |
| DoRABase Model=Qwen2-1.5B, rank=82026.02 | 87.15 | 87.19 | |
| MELoRABase Model=Qwen2-1.5B, rank=82026.02 | 86.96 | 86.55 | |
| HypLoRABase Model=Qwen2-1.5B, rank=82026.02 | 85.91 | 87.71 | |
| LoRABase Model=Qwen2-1.5B, rank=increased2026.02 | 83.51 | 86.43 | |
| AdaLoRABase Model=Qwen2-1.5B, rank=82026.02 | 82.45 | 79.77 | |
| HMoRABase Model=Qwen2-1.5B, activated parameters=2.26%2026.02 | 66.63 | 66.49 | |
| BaseBase Model=Qwen2-1.5B2026.02 | 32.41 | 69.16 |