Language Modeling on Dream 7B
69.2Throughput (tokens/sec)TSPD
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| TSPDKV cache=true, Generation Length=2562026.05 | 69.2 | 7.6 | 75 | |
| F-dLLMKV cache=true, Generation Length=2562026.05 | 48.2 | 5.3 | 74.8 | |
| CDKV cache=true, Generation Length=2562026.05 | 42.8 | 4.7 | 74.5 | |
| TSPDKV cache=false, Generation Length=2562026.05 | 28.2 | 3.1 | 75.3 | |
| Learn2PDKV cache=false, Generation Length=2562026.05 | 19.1 | 2.1 | 73.4 | |
| ProphetKV cache=false, Generation Length=2562026.05 | 15.1 | 1.7 | 74.9 | |
| VanillaKV cache=true, Generation Length=2562026.05 | 9.1 | 1 | 75 |