Instruction Tuning on Data Mix
59.1AccuracyGRADIENTSPACE
Evaluation results
| Method | Links | |
|---|---|---|
| GRADIENTSPACEBackbone Model=LLaMA-2-7B, initialization=SVD init2025.12 | 59.1 | |
| ELREABackbone Model=LLaMA-2-7B2025.12 | 56.2 | |
| K-means + LoRA (Mixture of LoRA)Backbone Model=LLaMA-2-7B2025.12 | 53.3 | |
| FinetuningBackbone Model=LLaMA-2-7B2025.12 | 48.9 | |
| LoRA FinetuningBackbone Model=LLaMA-2-7B2025.12 | 48.1 | |
| Model MergingBackbone Model=LLaMA-2-7B2025.12 | 47.1 | |
| GRADIENTSPACEBackbone Model=LLaMA-3.2-1B, initialization=SVD init2025.12 | 46.4 | |
| ELREABackbone Model=LLaMA-3.2-1B2025.12 | 41.7 | |
| K-means + LoRA (Mixture of LoRA)Backbone Model=LLaMA-3.2-1B2025.12 | 38.8 | |
| Random clustering + LoRABackbone Model=LLaMA-2-7B2025.12 | 38.3 | |
| Model MergingBackbone Model=LLaMA-3.2-1B2025.12 | 34.1 | |
| FinetuningBackbone Model=LLaMA-3.2-1B2025.12 | 33.8 | |
| LoRA FinetuningBackbone Model=LLaMA-3.2-1B2025.12 | 33.6 | |
| Zero-shot prompting (Pretrained)Backbone Model=LLaMA-2-7B2025.12 | 33.4 | |
| Random clustering + LoRABackbone Model=LLaMA-3.2-1B2025.12 | 26.9 | |
| Zero-shot prompting (Pretrained)Backbone Model=LLaMA-3.2-1B2025.12 | 20.1 |