Long-Context Retrieval on RULER 4K
92.97ScoreHybrid-KSA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Hybrid-KSASetting=CPT setting2026.04 | 92.97 | — | — | — | |
| Full AttentionSetting=CPT setting2026.04 | 92.88 | — | — | — | |
| KSASetting=CPT setting2026.04 | 91.55 | — | — | — | |
| Hybrid-SWASetting=CPT setting2026.04 | 91.3 | — | — | — | |
| Hybrid-LinearSetting=CPT setting2026.04 | 86.39 | — | — | — | |
| Hybrid-SCASetting=CPT setting2026.04 | 86.02 | — | — | — | |
| Hybrid-KSAsetting=Train-from-scratch2026.04 | 80.65 | — | — | — | |
| Hybrid-GDNsetting=Train-from-scratch2026.04 | 79.83 | — | — | — | |
| Hybrid-SCAsetting=Train-from-scratch2026.04 | 77.72 | — | — | — | |
| Fullsetting=Train-from-scratch2026.04 | 76.08 | — | — | — | |
| Hybrid-SWAsetting=Train-from-scratch2026.04 | 74.54 | — | — | — | |
| KSAsetting=Train-from-scratch2026.04 | 70.44 | — | — | — | |
| fp16Backbone=Qwen3-8B, n=50, config=fp162026.05 | — | 73 | 60 | 100 | |
| HQMQ+Med3Backbone=Qwen3-8B, n=50, bits=4.89, config=HQMQ s96 r6 + Med3x2026.05 | — | 72 | 68 | 100 | |
| naive int4Backbone=Qwen3-8B, n=50, config=naive int42026.05 | — | 43 | 26 | 23 |