Natural Language Generation and Understanding on S_NLG and S_NLU (test)
84.17Average PerformanceTeacher
Evaluation results
| Method | Links | |
|---|---|---|
| TeacherStudent Model=LLaMA-68M, Teacher Model=LLaMA-7B2024.02 | 84.17 | |
| TeacherStudent Model=LLaMA-68M, Teacher Model=LLaMA-1.1B2024.02 | 75.23 | |
| TeacherStudent Model=Pythia-410M, Teacher Model=Pythia-2.8B2024.02 | 73.5 | |
| TeacherStudent Model=Pythia-410M, Teacher Model=Pythia-1.4B2024.02 | 67.86 | |
| Supervised KD + ATKDStudent Model=Pythia-410M, Teacher Model=Pythia-1.4B2024.02 | 61.81 | |
| Supervised KD + ATKDStudent Model=Pythia-410M, Teacher Model=Pythia-2.8B2024.02 | 61.22 | |
| Supervised KDStudent Model=Pythia-410M, Teacher Model=Pythia-1.4B2024.02 | 60.66 | |
| Supervised KDStudent Model=Pythia-410M, Teacher Model=Pythia-2.8B2024.02 | 59.91 | |
| Reverse KD + ATKDStudent Model=Pythia-410M, Teacher Model=Pythia-2.8B2024.02 | 57.94 | |
| Reverse KD + ATKDStudent Model=Pythia-410M, Teacher Model=Pythia-1.4B2024.02 | 57.05 | |
| Reverse KDStudent Model=Pythia-410M, Teacher Model=Pythia-1.4B2024.02 | 55.92 | |
| Reverse KDStudent Model=Pythia-410M, Teacher Model=Pythia-2.8B2024.02 | 54.67 | |
| Supervised KD + ATKDStudent Model=LLaMA-68M, Teacher Model=LLaMA-1.1B2024.02 | 31.19 | |
| Supervised KD + ATKDStudent Model=LLaMA-68M, Teacher Model=LLaMA-7B2024.02 | 30.19 | |
| Supervised KDStudent Model=LLaMA-68M, Teacher Model=LLaMA-1.1B2024.02 | 30.06 | |
| Supervised KDStudent Model=LLaMA-68M, Teacher Model=LLaMA-7B2024.02 | 27.94 | |
| Reverse KD + ATKDStudent Model=LLaMA-68M, Teacher Model=LLaMA-7B2024.02 | 26.99 | |
| Reverse KD + ATKDStudent Model=LLaMA-68M, Teacher Model=LLaMA-1.1B2024.02 | 26.73 | |
| Reverse KDStudent Model=LLaMA-68M, Teacher Model=LLaMA-1.1B2024.02 | 26.15 | |
| Reverse KDStudent Model=LLaMA-68M, Teacher Model=LLaMA-7B2024.02 | 25.94 |