Language Generation on Vicuna (test)
19.4ROUGE-LTeacher
Evaluation results
| Method | Links | |
|---|---|---|
| TeacherModel=LLaMA, #Params=13B2026.03 | 19.4 | |
| VQAEModel=LLaMA, #Params=7B2026.03 | 18.7 | |
| KDModel=LLaMA, #Params=7B2026.03 | 18.4 | |
| SeqKDModel=LLaMA, #Params=7B2026.03 | 18.1 | |
| SFT w/o KDModel=LLaMA, #Params=7B2026.03 | 17.5 | |
| KDModel=GPT-2, #Params=760M2026.03 | 16.9 | |
| TeacherModel=GPT-2, #Params=1.5B2026.03 | 16.3 | |
| VQAEModel=GPT-2, #Params=760M2026.03 | 16.3 | |
| SFT w/o KDModel=GPT-2, #Params=760M2026.03 | 16.1 | |
| SeqKDModel=GPT-2, #Params=760M2026.03 | 15.9 | |
| VQAEModel=GPT-2, #Params=120M2026.03 | 15.2 | |
| SFT w/o KDModel=GPT-2, #Params=120M2026.03 | 14.7 | |
| SeqKDModel=GPT-2, #Params=120M2026.03 | 14.3 | |
| KDModel=GPT-2, #Params=120M2026.03 | 13.4 |