Context Management on Long-context (test)
1mTokensSnapKV
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SnapKVModel=Llama-3-8B-it-262k, Settings=Ktok = 10242026.03 | 1 | 0.208 | 41.78 | |
| TokenSelectModel=Llama-3.1-8B-it, Settings=Ktok = 10242026.03 | 1 | — | 39.43 | |
| H2OModel=Llama-3.1-8B-it, Settings=Kcom = 1%2026.03 | 1 | 0.123 | 65.81 | |
| RetrievalAttnModel=Llama-3-8B-it-262k, Settings=Ktok = 20482026.03 | 2 | 0.026 | 32.22 | |
| OmniKVModel=Llama-3-8B-it-262k, Settings=Ktok = 6.7%2026.03 | 8 | — | 45.83 | |
| LLMLingua-2Model=Llama-3.1-8B-it, Settings=Kcom = 25%2026.03 | 32 | — | 18.08 | |
| UT-ACA (Update: Sub.8)Model=Qwen2-7B-it, Settings=Kmax = 162026.03 | 119 | 0.059 | 37.08 | |
| UT-ACA (Update: Sub.16)Model=Llama-3.1-8B-it, Settings=Kmax = 322026.03 | 123 | 0.069 | 45.23 | |
| InfLLMModel=Llama-3.1-8B-it, Settings=K = 82026.03 | 128 | 0.064 | 25.81 | |
| InfLLMModel=Qwen2-7B-it, Settings=K = 82026.03 | 128 | 0.061 | 37.77 | |
| UT-ACA (Update: Sub.16)Model=Llama-3.1-8B-it, Settings=Kmax = 482026.03 | 218 | 0.075 | 62.17 | |
| InfLLMModel=Llama-3.1-8B-it, Settings=K = 162026.03 | 256 | 0.072 | 51.23 | |
| InfLLMModel=Qwen2-7B-it, Settings=K = 162026.03 | 256 | 0.064 | 44.39 | |
| UT-ACA (Update: Sub.8)Model=Qwen2-7B-it, Settings=Kmax = 322026.03 | 302 | 0.066 | 48.33 | |
| UT-ACA (Update: Sub.16)Model=Llama-3.1-8B-it, Settings=Kmax = 642026.03 | 344 | 0.082 | 70.48 | |
| UT-ACA (Update: Sub.8)Model=Qwen2-7B-it, Settings=Kmax = 482026.03 | 432 | 0.079 | 56.64 | |
| UT-ACA (Update: Sub.16)Model=Llama-3.1-8B-it, Settings=Kmax = 962026.03 | 498 | 0.097 | 74.61 | |
| InfLLMModel=Llama-3.1-8B-it, Settings=K = 322026.03 | 512 | 0.08 | 72.26 | |
| InfLLMModel=Qwen2-7B-it, Settings=K = 322026.03 | 512 | 0.077 | 54.72 |