Language Modeling on Wiki2 and C4 (test)
5.69Perplexity (Wiki2)STAR-KV
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| STAR-KVModel=LLaMA-2-13B, Compression (%)=602026.06 | 5.69 | 8.22 | 6.96 | |
| BaselineModel=LLaMA-2-13B, Compression (%)=02026.06 | 5.71 | 8.19 | 6.95 | |
| STAR-KVModel=LLaMA-2-13B, Compression (%)=752026.06 | 5.89 | 8.66 | 7.28 | |
| BaselineModel=Mistral-7B-Inst., Compression (%)=02026.06 | 5.94 | 9.72 | 7.83 | |
| PaluModel=LLaMA-2-13B, Compression (%)=602026.06 | 6.48 | 9.69 | 8.09 | |
| PaluModel=Mistral-7B-Inst., Compression (%)=602026.06 | 7.07 | 12.93 | 10 | |
| STAR-KVModel=Mistral-7B-Inst., Compression (%)=602026.06 | 8.01 | 10.81 | 9.41 |