Model Merging on Llama-3.2-1B-Instruct Task Set
0.83Score S(gamma)K-Merge
Evaluation Results
| Method | Links | |
|---|---|---|
| K-MergeLoRA storage budget K=82025.10 | 0.83 | |
| LinearLoRA storage budget K=82025.10 | 0.82 | |
| K-MergeLoRA storage budget K=72025.10 | 0.82 | |
| K-Merge++LoRA storage budget K=62025.10 | 0.82 | |
| K-Merge++LoRA storage budget K=72025.10 | 0.82 | |
| K-Merge++LoRA storage budget K=82025.10 | 0.82 | |
| LinearLoRA storage budget K=72025.10 | 0.81 | |
| K-MergeLoRA storage budget K=62025.10 | 0.81 | |
| K-Merge++LoRA storage budget K=52025.10 | 0.81 | |
| LinearLoRA storage budget K=62025.10 | 0.8 | |
| K-MergeLoRA storage budget K=52025.10 | 0.8 | |
| LinearLoRA storage budget K=52025.10 | 0.79 | |
| K-MergeLoRA storage budget K=42025.10 | 0.77 | |
| K-Merge++LoRA storage budget K=42025.10 | 0.77 | |
| TIESLoRA storage budget K=82025.10 | 0.73 | |
| K-MergeLoRA storage budget K=32025.10 | 0.73 | |
| K-Merge++LoRA storage budget K=32025.10 | 0.73 | |
| LinearLoRA storage budget K=42025.10 | 0.71 | |
| TIESLoRA storage budget K=72025.10 | 0.71 | |
| LinearLoRA storage budget K=32025.10 | 0.69 | |
| K-MergeLoRA storage budget K=22025.10 | 0.69 | |
| K-Merge++LoRA storage budget K=22025.10 | 0.69 | |
| TIESLoRA storage budget K=62025.10 | 0.67 | |
| TIESLoRA storage budget K=52025.10 | 0.65 | |
| K-MergeLoRA storage budget K=12025.10 | 0.63 | |
| K-Merge++LoRA storage budget K=12025.10 | 0.63 | |
| LinearLoRA storage budget K=22025.10 | 0.62 | |
| DARELoRA storage budget K=82025.10 | 0.61 | |
| DARELoRA storage budget K=72025.10 | 0.6 | |
| DARELoRA storage budget K=42025.10 | 0.59 | |
| DARELoRA storage budget K=62025.10 | 0.59 | |
| TIESLoRA storage budget K=42025.10 | 0.58 | |
| DARELoRA storage budget K=52025.10 | 0.58 | |
| OPCMLoRA storage budget K=82025.10 | 0.58 | |
| DARELoRA storage budget K=32025.10 | 0.54 | |
| DARE-TIESLoRA storage budget K=82025.10 | 0.53 | |
| DARELoRA storage budget K=22025.10 | 0.52 | |
| DARE-TIESLoRA storage budget K=72025.10 | 0.52 | |
| OPCMLoRA storage budget K=72025.10 | 0.52 | |
| LinearLoRA storage budget K=12025.10 | 0.51 | |
| TIESLoRA storage budget K=22025.10 | 0.51 | |
| TIESLoRA storage budget K=32025.10 | 0.51 | |
| DARE-TIESLoRA storage budget K=42025.10 | 0.51 | |
| DARE-TIESLoRA storage budget K=52025.10 | 0.51 | |
| DARE-TIESLoRA storage budget K=62025.10 | 0.51 | |
| DARELoRA storage budget K=12025.10 | 0.5 | |
| OPCMLoRA storage budget K=62025.10 | 0.5 | |
| DARE-TIESLoRA storage budget K=32025.10 | 0.49 | |
| DARE-TIESLoRA storage budget K=12025.10 | 0.48 | |
| DARE-TIESLoRA storage budget K=22025.10 | 0.48 | |
| TIESLoRA storage budget K=12025.10 | 0.47 | |
| OPCMLoRA storage budget K=52025.10 | 0.44 | |
| OPCMLoRA storage budget K=32025.10 | 0.4 | |
| OPCMLoRA storage budget K=42025.10 | 0.38 | |
| OPCMLoRA storage budget K=22025.10 | 0.21 | |
| OPCMLoRA storage budget K=12025.10 | 0 |