Instruction Fine-tuning on Alpaca-52K (train)
65.88Average ScoreFull-finetuning
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Full-finetuningBackbone=LLaMA2-7B, LLMs branch=16-16-16, low-rank branch=16-16-162025.02 | 65.88 | 2.4 | 58.56 | |
| QLORABackbone=LLaMA2-7B, LLMs branch=4-16-16, low-rank branch=16-16-162025.02 | 65.69 | 3.2 | 10.73 | |
| GSQ-TuningBackbone=LLaMA2-7B, LLMs branch=4-6-6, low-rank branch=6-6-62025.02 | 65.39 | 3.4 | 5.97 | |
| LLaMA2-7BLLMs branch=16-16-None, low-rank branch=w/o2025.02 | 64.13 | 0 | 13.2 |