Efficiency Evaluation on Long-context sequences 8K to 60K
14.85Avg TTFTHyLRA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| HyLRASequence Length=8K, Attention Mechanism=HyLRA2026.01 | 14.85 | 1,145 | 10,309 | 0.93 | |
| Jump3Sequence Length=8K, Attention Mechanism=Jump32026.01 | 15.27 | 1,229 | 11,068 | 1 | |
| Full AttentionSequence Length=8K, Attention Mechanism=Full2026.01 | 16.81 | 1,225 | 11,033 | — | |
| Jump3Sequence Length=16K, Attention Mechanism=Jump32026.01 | 25.69 | 839 | 7,553 | 1.15 | |
| HyLRASequence Length=16K, Attention Mechanism=HyLRA2026.01 | 26.23 | 771 | 6,946 | 1.06 | |
| Full AttentionSequence Length=16K, Attention Mechanism=Full2026.01 | 26.55 | 728 | 6,555 | — | |
| Jump3Sequence Length=30K, Attention Mechanism=Jump32026.01 | 37.25 | 544 | 4,627 | 1.21 | |
| HyLRASequence Length=30K, Attention Mechanism=HyLRA2026.01 | 37.76 | 514 | 4,374 | 1.14 | |
| Full AttentionSequence Length=30K, Attention Mechanism=Full2026.01 | 38.23 | 450 | 3,826 | — | |
| Jump3Sequence Length=60K, Attention Mechanism=Jump32026.01 | 53.98 | 273 | 4,730 | 1.54 | |
| HyLRASequence Length=60K, Attention Mechanism=HyLRA2026.01 | 56.62 | 258 | 4,128 | 1.45 | |
| Full AttentionSequence Length=60K, Attention Mechanism=Full2026.01 | 57.69 | 177 | 2,835 | — |