Long-context language understanding on RULER 128k
89.49AccuracyRetroInfer
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RetroInferModel=Qwen2.5-14B2025.11 | 89.49 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LiteCache + HATAModel=Qwen2.5-14B2025.11 | 88.94 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-14BModel=Qwen2.5-14B2025.11 | 88.85 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HATA AlgoModel=Qwen2.5-14B2025.11 | 88.47 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| InfiniGenModel=Qwen2.5-14B2025.11 | 85.26 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HATA AlgoModel=Llama3-8B2025.11 | 73.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LiteCache + HATAModel=Llama3-8B2025.11 | 73.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama3-8BModel=Llama3-8B2025.11 | 72.96 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RetroInferModel=Llama3-8B2025.11 | 72.73 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| InfiniGenModel=Llama3-8B2025.11 | 69.37 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FusedKVContext Window=128k, Model Parameters=1.5B2025.12 | — | 42 | 71.3 | 64 | 85 | 4 | 5.25 | 40.25 | 42 | 90 | 87 | 27 | 16 | 10.2 | |
| FusedKV-LiteContext Window=128k, Model Parameters=1.5B2025.12 | — | 42.31 | 78.3 | 75.67 | 91 | 18 | 0.75 | 28.5 | 98 | 97 | 18 | 20 | 19 | 5.8 | |
| VanillaContext Window=128k, Model Parameters=1.5B2025.12 | — | 49.11 | 89 | 78.67 | 94 | 63 | 16 | 29 | 27 | 90 | 82 | 28 | 21 | 12.8 | |
| YOCOContext Window=128k, Model Parameters=1.5B2025.12 | — | 17.32 | 25 | 76 | 8 | 31 | 1.5 | 0 | 47 | 0 | 0 | 21 | 14 | 1.6 |