Many-shot in-context learning on Long-context benchmarks
74.2ICL Performance (8k Context)FlexPrefill
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| FlexPrefillBackbone=Qwen 2.5 7B2026.03 | 74.2 | 78.5 | 82.3 | 84.2 | 85.2 | |
| CLPBackbone=Qwen 2.5 7B2026.03 | 73.6 | 79.4 | 81.7 | 84.5 | 87.4 | |
| L2A (Ours)Backbone=Qwen 2.5 7B2026.03 | 71.4 | 79.6 | 83.5 | 87 | 88.6 | |
| FlexPrefillBackbone=Qwen3 8B2026.03 | 71.2 | 75.8 | 80.2 | 83.7 | 85.1 | |
| CLPBackbone=Qwen3 8B2026.03 | 70.8 | 74.7 | 79.1 | 82.8 | 85.4 | |
| SelfExtendBackbone=Qwen 2.5 7B2026.03 | 70.2 | 75 | 78 | 81.2 | 81.8 | |
| BaseBackbone=Qwen 2.5 7B2026.03 | 70.1 | 75.1 | 78.4 | 78.5 | 71.5 | |
| DCABackbone=Qwen 2.5 7B2026.03 | 70 | 75.6 | 78.7 | 81.2 | 82.4 | |
| NSABackbone=Qwen 2.5 7B2026.03 | 66.6 | 71 | 74.2 | 78.2 | 78.6 | |
| BaseBackbone=Qwen3 8B2026.03 | 65.5 | 68.9 | 70.1 | 68.8 | 9.3 | |
| DCABackbone=Qwen3 8B2026.03 | 65.4 | 68.5 | 70.5 | 72.9 | 76.2 | |
| L2ABackbone=Qwen3 8B2026.03 | 64.4 | 67.3 | 74.4 | 80.2 | 85.7 | |
| S^2-attnBackbone=Qwen 2.5 7B2026.03 | 61.8 | 67 | 71.5 | 76.7 | 77 | |
| NSABackbone=Qwen 2.5 1.5B2026.03 | 61.5 | 65 | 65.4 | 65.3 | 65 | |
| DCABackbone=Qwen 2.5 1.5B2026.03 | 54.7 | 64.8 | 69.8 | 74.2 | 76.3 | |
| L2ABackbone=Qwen 2.5 1.5B2026.03 | 54.6 | 65.8 | 72.3 | 77 | 80.3 | |
| BaseBackbone=Qwen 2.5 1.5B2026.03 | 54.5 | 65.2 | 69.4 | 68.1 | 43.6 | |
| SelfExtendBackbone=Qwen 2.5 1.5B2026.03 | 54.4 | 65.2 | 70.2 | 73.6 | 74.4 | |
| S^2-attnBackbone=Qwen 2.5 1.5B2026.03 | 54.2 | 62.4 | 68.8 | 73.8 | 72.8 | |
| CLPBackbone=Qwen 2.5 1.5B2026.03 | 49.5 | 60.2 | 66.2 | 72.3 | 77.4 | |
| FlexPrefillBackbone=Qwen 2.5 1.5B2026.03 | 39.2 | 53 | 63 | 68.9 | 72 |