Long-context evaluation (Speed & Accuracy) on RULER 128K
65.03RULER ScoreFullAttention
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| FullAttentionBackbone=Llama3.1-70B-Instruct, Number of attention heads=64, Sequence length=128K2025.09 | 65.03 | 91.84 | 1 | |
| ProxyAttnBackbone=Llama3.1-70B-Instruct, Number of attention heads=64, Sequence length=128K2025.09 | 62.23 | 44.23 | 2.08 | |
| MInferenceBackbone=Llama3.1-70B-Instruct, Number of attention heads=64, Sequence length=128K2025.09 | 60.33 | 82.9 | 1.11 | |
| XAttentionBackbone=Llama3.1-70B-Instruct, Number of attention heads=64, Sequence length=128K2025.09 | 58.59 | 60.57 | 1.52 |