Knowledge-Preserved Adaptation on Combined Suite TriviaQA, NQ open, WebQS, HumanEval, MBPP
21.91Average ScoreCorDA
Evaluation results
| Method | Links | |
|---|---|---|
| CorDA#Params=320M, Fine-tuning Task=Code, Rank (r)=1282024.06 | 21.91 | |
| PiSSA#Params=320M, Fine-tuning Task=Code, Rank (r)=1282024.06 | 21.54 | |
| LoRA#Params=320M, Fine-tuning Task=Code, Rank (r)=1282024.06 | 21.5 | |
| Full fine-tuning#Params=6738M, Fine-tuning Task=Code2024.06 | 18.46 |