Commonsense Question Answering on CSQA LONGGENBENCH
73.5Accuracy (n=15)Full Attention
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Full AttentionBackbone=Llama-3.1-8B-Instruct, Decoding scheme=greedy decoding2025.08 | 73.5 | 74.1 | 71.9 | 73.2 | |
| QuestBackbone=Llama-3.1-8B-Instruct, KV cache budget=0.15, Retrospective window size (w)=2, Decoding scheme=greedy decoding2025.08 | 70.9 | 53.5 | 46.1 | 56.8 | |
| RetroAttentionBackbone=Llama-3.1-8B-Instruct, KV cache budget=0.15, Retrospective window size (w)=2, Decoding scheme=greedy decoding2025.08 | 68.8 | 59.1 | 53 | 60.3 | |
| TOVABackbone=Llama-3.1-8B-Instruct, KV cache budget=0.15, Retrospective window size (w)=2, Decoding scheme=greedy decoding2025.08 | 6.7 | 6.3 | 11.1 | 8.1 | |
| StreamingLLMBackbone=Llama-3.1-8B-Instruct, KV cache budget=0.15, Retrospective window size (w)=2, Decoding scheme=greedy decoding2025.08 | 0.8 | 1.1 | 4 | 2 |