Language Modeling on Wiki2 and C4
5.12Perplexity (Wiki2)LLaMA-2-7B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LLaMA-2-7BCompression Rate=0%2026.06 | 5.12 | 7.04 | 6.08 | |
| STAR-KVBase Model=LLaMA-2-7B, Compression Rate=60%2026.06 | 5.49 | 7.45 | 6.47 | |
| LLaMA2Size=7B, Bit=162025.08 | 5.5 | 7.3 | 6.4 | |
| PaluBase Model=LLaMA-2-7B, Compression Rate=50%2026.06 | 5.63 | 8.38 | 7.01 | |
| STAR-KVBase Model=LLaMA-2-7B, Compression Rate=75%2026.06 | 5.86 | 8.02 | 6.94 | |
| ReCalKVBase Model=LLaMA-2-7B, Compression Rate=70%2026.06 | 6.75 | 13.63 | 11.32 | |
| STAR-KVBase Model=LongChat-7B-v1.5, Compression Rate=60%2026.06 | 6.83 | 9.98 | 8.41 | |
| LongChat-7B-v1.5Compression Rate=0%2026.06 | 6.86 | 9.93 | 8.4 | |
| STAR-KVBase Model=LongChat-7B-v1.5, Compression Rate=75%2026.06 | 7.34 | 10.44 | 8.89 | |
| PaluBase Model=LongChat-7B-v1.5, Compression Rate=50%2026.06 | 7.37 | 11.67 | 9.52 | |
| LLaMA-3-8B-InstCompression Rate=0%2026.06 | 7.74 | 12.61 | 10.18 | |
| MoSSize=7B, Bit=1.012025.08 | 7.9 | 9.8 | 8.8 | |
| STAR-KVBase Model=LLaMA-3-8B-Inst, Compression Rate=60%2026.06 | 8.52 | 13.51 | 11.01 | |
| BinaryLLMSize=7B, Bit=1.012025.08 | 8.7 | 9.7 | 9.2 | |
| ReCalKVBase Model=LongChat-7B-v1.5, Compression Rate=70%2026.06 | 9.01 | 13.63 | 11.32 | |
| FBI-LLMSize=7B, Bit=1.012025.08 | 9.1 | 10.5 | 10.3 | |
| PaluBase Model=LLaMA-3-8B-Inst, Compression Rate=50%2026.06 | 9.43 | 17.37 | 13.4 | |
| OneBit-LLaMA2Size=7B, Bit=1.012025.08 | 9.7 | 11.1 | 10.4 | |
| STAR-KVBase Model=LLaMA-3-8B-Inst, Compression Rate=75%2026.06 | 10.2 | 15.46 | 12.83 |