Natural Language Generation on UltraChat
36.8BLEUADORE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ADOREBackbone=Llama-2 7B, Context length=4,000, KV cache size=1922024.07 | 36.8 | 28.8 | 63.5 | |
| Full AttentionBackbone=Llama-2 7B, Context length=4,0002024.07 | 35.6 | 29.2 | 63.4 | |
| Strided AttentionBackbone=Llama-2 7B, Context length=4,000, KV cache size=1922024.07 | 28 | 24.8 | 57.5 | |
| H2O(Rebuilt)Backbone=Llama-2 7B, Context length=4,000, KV cache size=1922024.07 | 27.6 | 26.9 | 61.5 | |
| Window AttentionBackbone=Llama-2 7B, Context length=4,000, KV cache size=1922024.07 | 26.7 | 28 | 61.4 | |
| H2OBackbone=Llama-2 7B, Context length=4,000, KV cache size=1922024.07 | 26.4 | 25.3 | 60.3 | |
| StreamingLLMBackbone=Llama-2 7B, Context length=4,000, KV cache size=1922024.07 | 23.9 | 26 | 59.6 | |
| KV CompressionBackbone=Llama-2 7B, Context length=4,000, KV cache size=1922024.07 | 21.1 | 23.2 | 56.9 |