Long-Context Retrieval on RULER (4K-128K Sweep with Latency)
96.74Retrieval Accuracy (4K)LLaMA3.1-8B-Instruct
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| LLaMA3.1-8B-InstructModel=LLaMA3.1-8B-Instruct, Attention=Full, Max Context=128K2025.12 | 96.74 | 94.03 | 92.02 | 84.17 | 81.32 | 76.89 | 87.52 | 1,282.05 | |
| MInferenceModel=LLaMA3.1-8B-Instruct, Max Context=128K2025.12 | 96.54 | 94.06 | 91.37 | 85.79 | 83.03 | 54.13 | 84.15 | 837.09 | |
| TCA-AttentionModel=LLaMA3.1-8B-Instruct, Max Context=128K2025.12 | 96.31 | 95.38 | 93.92 | 86.38 | 82.89 | 77.46 | 88.72 | 453.22 | |
| XAttentionModel=LLaMA3.1-8B-Instruct, Max Context=128K2025.12 | 96.15 | 93.95 | 93.71 | 90.9 | 83.35 | 72.57 | 88.44 | 504 | |
| MInferenceModel=Qwen2.5-7B-Instruct, Max Context=128K2025.12 | 96.08 | 94.92 | 91.69 | 89.92 | 70.46 | 51.62 | 82.45 | 771.61 | |
| Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct, Attention=Full, Max Context=128K2025.12 | 96 | 94.85 | 91.77 | 89.85 | 70.38 | 52.92 | 82.63 | 1,101.01 | |
| TCA-AttentionModel=Qwen2.5-7B-Instruct, Max Context=128K2025.12 | 96 | 94.77 | 91.92 | — | 69.85 | 52.31 | 82.49 | 396.72 | |
| FlexPrefillModel=LLaMA3.1-8B-Instruct, Max Context=128K2025.12 | 95.99 | 93.67 | 92.73 | 88.14 | 81.14 | 74.67 | 87.72 | 1,021.47 | |
| FlexPrefillModel=Qwen2.5-7B-Instruct, Max Context=128K2025.12 | 95.62 | 94.31 | 92 | 88.23 | 70.23 | 52.15 | 82.09 | 892.55 | |
| XAttentionModel=Qwen2.5-7B-Instruct, Max Context=128K2025.12 | 95.45 | 92.91 | 92.04 | 88.84 | 68.84 | 55.36 | 82.24 | 441.36 |