Multiple Choice Question Answering on LongBench v2 (test)
44.07Accuracy (Easy, Short)Dense
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| DenseBackbone=Llama3.1, Attention Type=Full Dense Attention2025.06 | 44.07 | 28.41 | 31.11 | 32.23 | 25.98 | 25.4 | 30.42 | |
| top-kBackbone=Llama3.1, sparse token budget=1024, Attention Type=Exact top-k attention2025.06 | 40.68 | 25 | 33.33 | 29.75 | 25.2 | 23.81 | 28.63 | |
| HATABackbone=Llama3.1, sparse token budget=10242025.06 | 38.98 | 27.27 | 35.56 | 29.75 | 26.77 | 25.4 | 29.62 |