General Language Understanding on C-Eval (val)
78.68AccuracyQwen-1.5 14B (Teacher)
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen-1.5 14B (Teacher)Model Scale=14B, Distillation Strategy=None, Evaluation Protocol=Zero-shot2024.07 | 78.68 | |
| Qwen-1.5 1.8B + DDKModel Scale=1.8B, Distillation Strategy=DDK, Evaluation Protocol=Zero-shot2024.07 | 63.75 | |
| Qwen-1.5 1.8B + TEDModel Scale=1.8B, Distillation Strategy=TED, Evaluation Protocol=Zero-shot2024.07 | 62.04 | |
| Qwen-1.5 1.8B + MiniLLMModel Scale=1.8B, Distillation Strategy=MiniLLM, Evaluation Protocol=Zero-shot2024.07 | 61.66 | |
| Qwen-1.5 1.8B + CPT & DoReMiModel Scale=1.8B, Distillation Strategy=CPT & DoReMi, Evaluation Protocol=Zero-shot2024.07 | 61.44 | |
| Qwen-1.5 1.8B + KDModel Scale=1.8B, Distillation Strategy=Knowledge Distillation, Evaluation Protocol=Zero-shot2024.07 | 61.29 | |
| Qwen-1.5 1.8B + CPTModel Scale=1.8B, Distillation Strategy=CPT, Evaluation Protocol=Zero-shot2024.07 | 60.13 | |
| PHSATraining Tokens=100B, Inference Top-K=N/A, Training Top-K=22026.01 | 59.96 | |
| Qwen-1.5 1.8B (Student)Model Scale=1.8B, Distillation Strategy=None, Evaluation Protocol=Zero-shot2024.07 | 59.66 | |
| InfLLM v2Training Tokens=100B, Inference Top-K=N/A, Training Top-K=22026.01 | 59.36 | |
| PHSATraining Tokens=100B, Inference Top-K=N/A, Training Top-K=42026.01 | 59.14 | |
| DenseTraining Tokens=100B, Inference Top-K=22026.01 | 58.99 | |
| PHSATraining Tokens=100B, Inference Top-K=2, Training Top-K=22026.01 | 58.99 | |
| PHSATraining Tokens=100B, Inference Top-K=4, Training Top-K=42026.01 | 58.84 | |
| DenseTraining Tokens=100B, Inference Top-K=N/A2026.01 | 58.77 | |
| InfLLM v2Training Tokens=100B, Inference Top-K=N/A, Training Top-K=42026.01 | 58.77 | |
| InfLLM v2Training Tokens=100B, Inference Top-K=2, Training Top-K=22026.01 | 58.4 | |
| InfLLM v2Training Tokens=100B, Inference Top-K=4, Training Top-K=42026.01 | 58.25 |