Long-context understanding on LongBench (Specific QA/Retrieval Metrics)
30.94MFQATidalDecode
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| TidalDecodeBackbone=Llama-3-8B-Instruct-Gradient-1048k, Budget=40962026.02 | 30.94 | 6.19 | 13.85 | 14.4 | 13.71 | 19.48 | 86.3 | 78 | 32.86 | |
| Full AttentionBackbone=Llama-3-8B-Instruct-Gradient-1048k2026.02 | 30.76 | 5.52 | 14.56 | 13.32 | 11.5 | 19.43 | 86.56 | 77 | 32.33 | |
| LycheeDecodeBackbone=Llama-3-8B-Instruct-Gradient-1048k, Budget=40962026.02 | 30.11 | 5.85 | 14.39 | 12.86 | 12.66 | 19.3 | 86.78 | 82.58 | 33.07 | |
| QuestBackbone=Llama-3-8B-Instruct-Gradient-1048k, Budget=40962026.02 | 28.92 | 3.74 | 13.63 | 12.83 | 12.15 | 19.36 | 85.91 | 72.5 | 31.13 | |
| TidalDecodeBackbone=Llama-3-8B-Instruct-Gradient-1048k, Budget=10242026.02 | 28.57 | 7.63 | 11.11 | 13.56 | 9.82 | 20.37 | 79.78 | 75.17 | 30.75 | |
| LycheeDecodeBackbone=Llama-3-8B-Instruct-Gradient-1048k, Budget=10242026.02 | 28.28 | 6.12 | 14.89 | 14.42 | 12.81 | 19.05 | 82.69 | 69.92 | 31.02 | |
| QuestBackbone=Llama-3-8B-Instruct-Gradient-1048k, Budget=10242026.02 | 26.21 | 4.08 | 12.19 | 12.61 | 10.75 | 19.56 | 83.47 | 63.84 | 29.09 | |
| Full AttentionBackbone=Qwen3-8B2026.02 | 25.84 | 3.43 | 10.96 | 11.97 | 11.74 | 20.9 | 90.21 | 89.08 | 33.02 | |
| LycheeDecodeBackbone=Qwen3-8B, Budget=40962026.02 | 24.9 | 3.32 | 10.88 | 12.74 | 11.68 | 20.71 | 90.34 | 93.25 | 33.48 | |
| SeerAttention-RBackbone=Qwen3-8B, Budget=40962026.02 | 24.85 | 3.3 | 11.15 | 12.42 | 11.35 | 20.61 | 90.19 | 93.17 | 33.38 | |
| LycheeDecodeBackbone=Qwen3-8B, Budget=10242026.02 | 24.26 | 3.14 | 10.45 | 11.05 | 12 | 19.81 | 86.64 | 91.71 | 32.38 | |
| SeerAttention-RBackbone=Qwen3-8B, Budget=10242026.02 | 23.91 | 2.97 | 10.28 | 11.88 | 11.28 | 19.04 | 87.5 | 86.79 | 31.71 | |
| DuoAttention*Backbone=Llama-3-8B-Instruct-Gradient-1048k, Budget=1024, double retrieval head budget=true2026.02 | 23.88 | 6.27 | 10.44 | 10.41 | 7.48 | 19 | 80.61 | 47.17 | 25.66 | |
| DuoAttention*Backbone=Llama-3-8B-Instruct-Gradient-1048k, Budget=4096, double retrieval head budget=true2026.02 | 23.74 | 6.63 | 13.8 | 13.67 | 10.4 | 17.93 | 86.03 | 61 | 29.15 | |
| TidalDecodeBackbone=Qwen3-8B, Budget=40962026.02 | 23.57 | 2.99 | 10.79 | 11.47 | 11.31 | 20.01 | 88.94 | 85 | 31.76 | |
| DuoAttentionBackbone=Llama-3-8B-Instruct-Gradient-1048k, Budget=40962026.02 | 22.27 | 7.16 | 13.93 | 12.74 | 10.73 | 17.93 | 83.76 | 34.75 | 25.41 | |
| TidalDecodeBackbone=Qwen3-8B, Budget=10242026.02 | 21.32 | 2.73 | 9.96 | 10.48 | 9.97 | 19.27 | 80.4 | 83.43 | 29.7 | |
| DuoAttentionBackbone=Llama-3-8B-Instruct-Gradient-1048k, Budget=10242026.02 | 19.02 | 7.36 | 8.6 | 9.68 | 8.77 | 17.75 | 41.92 | 13.25 | 15.79 |