List your company
ResearchFormerly WizWand

Long-context Language Modeling on RULER (4K Context)

95.3AccuracyFull KV
16.36436.85757.3577.843Oct 9, 2025
Updated 5mo ago

Evaluation results

MethodLinks
Full KVBackbone=LLaMA-3.1-8B, KV Budget (%)=1002025.1095.3
H2O + OBCACHE-V&KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=402025.1082
H2O + OBCACHE-KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=402025.1081.3
TOVA + OBCACHE-V&KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=Avg.2025.1076.7
H2O + OBCACHE-VBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=402025.1076
H2O + OBCACHE-V&KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=302025.1075.6
H2O + OBCACHE-KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=302025.1075.4
H2OBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=402025.1075.3
TOVABackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=Avg.2025.1074.5
H2O + OBCACHE-VBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=302025.1068.6
H2O + OBCACHE-V&KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=Avg.2025.1067.8
H2O + OBCACHE-KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=Avg.2025.1067.6
H2O + OBCACHE-V&KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=202025.1067.5
H2OBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=302025.1067.1
H2O + OBCACHE-KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=202025.1065.9
TOVA + OBCACHE-V&KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=102025.1065.5
TOVABackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=102025.1061.8
H2O + OBCACHE-VBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=Avg.2025.1059.9
H2OBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=Avg.2025.1057.5
H2O + OBCACHE-VBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=202025.1057.1
AdaKV + OBCACHE-V&KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Agnostic, KV Budget (%)=Avg.2025.1055.2
H2OBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=202025.1054.2
H2O + OBCACHE-KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=102025.1047.7
H2O + OBCACHE-V&KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=102025.1046.3
H2O + OBCACHE-VBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=102025.1037.7
H2OBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Aware, KV Budget (%)=102025.1033.5
SnapKV + OBCACHE-KBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Agnostic, KV Budget (%)=102025.1024.1
AdaKVBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Agnostic, KV Budget (%)=102025.1021.5
SnapKVBackbone=LLaMA-3.1-8B, Evaluation Protocol=Q-Agnostic, KV Budget (%)=102025.1019.4