Scientific QA on GPQA (Average accuracy)
41.94Average AccuracyTeacher
Evaluation results
| Method | Links | |
|---|---|---|
| TeacherBackbone=Llama3.1-70B, Training Method=Supervised Fine-Tuning (SFT), Training Dataset=Full Tulu 32026.06 | 41.94 | |
| SeqKDBackbone=Llama3.1-8B, Training Method=Two-stage KD strategy (SeqKD), Training Dataset=Full Tulu 32026.06 | 30.59 | |
| GKDBackbone=Llama3.1-8B, Training Method=Two-stage KD strategy (GKD), Training Dataset=Full Tulu 32026.06 | 30.22 | |
| SFTBackbone=Llama3.1-8B, Training Method=Supervised Fine-Tuning (SFT), Training Dataset=Full Tulu 32026.06 | 30.04 |