Speculative Decoding on Fineweb-edu 1.0 (test)
0.735Speculative Accept RateRandom Sampling KD (Ours 12+)
Evaluation results
| Method | Links | |
|---|---|---|
| Random Sampling KD (Ours 12+)Unique Tokens=12, Adaptive Weighting=true2025.03 | 0.735 | |
| FullKD2025.03 | 0.734 | |
| Random Sampling KD (Ours 12)Unique Tokens=122025.03 | 0.732 | |
| Top-K 50K=502025.03 | 0.731 | |
| Top-K 12K=122025.03 | 0.73 | |
| CE2025.03 | 0.711 |