5-shot Performance on CEval, MMLU, GSM8K, Arc-E, Arc-C (test)
79.86CEvalQwen-14B
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen-14BFew-shot=5-shot, Model Size=14B2024.07 | 79.86 | 66.3 | 69.14 | 89.24 | 82.25 | 77.36 | |
| DDK (Ours)Few-shot=5-shot, Model Size=1.8B, Strategy=DDK2024.07 | 65.38 | 47.59 | 55.19 | 76.64 | 57.01 | 60.36 | |
| Qwen-1.5 (Student) + TEDFew-shot=5-shot, Model Size=1.8B, Strategy=TED2024.07 | 62.11 | 45.47 | 49.43 | 74.94 | 55.47 | 57.48 | |
| Qwen-1.5 (Student) + MiniLLMFew-shot=5-shot, Model Size=1.8B, Strategy=MiniLLM2024.07 | 62.03 | 45.41 | 49.28 | 75.02 | 54.87 | 57.32 | |
| Qwen-1.5 (Student)Few-shot=5-shot, Model Size=1.8B2024.07 | 61.96 | 45.59 | 38.4 | 72.16 | 52.11 | 54.04 | |
| Qwen-1.5 (Student) + KDFew-shot=5-shot, Model Size=1.8B, Strategy=KD2024.07 | 61.66 | 44.28 | 50.26 | 73.87 | 54.69 | 56.95 | |
| Qwen-1.5 (Student) + CPTFew-shot=5-shot, Model Size=1.8B, Strategy=CPT2024.07 | 60.92 | 45.6 | 43.36 | 73.1 | 52.28 | 55.05 |