Model Calibration on Fineweb-edu 1.0 (test)
0.002ECERandom Sampling KD (Ours 12)
Evaluation results
| Method | Links | |
|---|---|---|
| Random Sampling KD (Ours 12)Unique Tokens=122025.03 | 0.002 | |
| FullKDMethod=FullKD2025.03 | 0.002 | |
| CEStudent Size=3B, Teacher Size=8B2025.03 | 0.003 | |
| Random Sampling KD (Ours 12+)Unique Tokens=12, Adaptive Weighting=true2025.03 | 0.017 | |
| Top-K 50K=502025.03 | 0.018 | |
| Top-K 12K=122025.03 | 0.047 | |
| LMSIEval.=cross2026.07 | 0.095 | |
| N-OPSDEval.=cross2026.07 | 0.177 | |
| TTRLEval.=cross2026.07 | 0.178 | |
| N-OPSDEval.=in2026.07 | 0.191 | |
| Qwen3-4B baseEval.=base2026.07 | 0.246 | |
| TTRLEval.=in2026.07 | 0.248 | |
| LMSIEval.=in2026.07 | 0.255 |