Language Modeling on WikiText (PPL, Latency, and KL Divergence)
0KL DivergenceOracle
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| OracleTransfer Pair=MistralLite→Mistral-7B, Network Bandwidth (Bnet)=200 Gbps2026.06 | 0 | 4.15 | 0 | 331.9 | 1 | |
| SCDTransfer Pair=MistralLite→Mistral-7B, Network Bandwidth (Bnet)=200 Gbps2026.06 | 0.1105 | 4.65 | 87.31 | 125.2 | 2.65 | |
| DroidSpeak-styleTransfer Pair=MistralLite→Mistral-7B, Network Bandwidth (Bnet)=200 Gbps2026.06 | 0.1459 | 5.33 | 127.34 | 198.7 | 1.67 | |
| SCD (Reuse-only)Transfer Pair=MistralLite→Mistral-7B, Network Bandwidth (Bnet)=200 Gbps2026.06 | 0.1661 | 6.11 | 69.02 | 118.2 | 2.81 | |
| Raw KV (BF16)Transfer Pair=MistralLite→Mistral-7B, Network Bandwidth (Bnet)=200 Gbps2026.06 | 3.1209 | 100 | 115.02 | 145.5 | 2.28 | |
| Quantized (4-bit)Transfer Pair=MistralLite→Mistral-7B, Network Bandwidth (Bnet)=200 Gbps2026.06 | 6.1887 | 100 | 28.76 | 59.2 | 5.6 |