Long-context language understanding on LongBench (Custom Metric Set)
47.1QPGPT-3.5-Turbo
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-3.5-TurboCompression Type=Full-Context (No Compression)2026.05 | 47.1 | 25.1 | 56.5 | 55.2 | 20.6 | 25.7 | 25.1 | 17.3 | 63 | 91.7 | 39.4 | 53.5 | 58.5 | 62.5 | |
| RAGPCompression Type=Text-based Compression, Compression Ratio=3-4x2026.05 | 46.7 | 23.9 | 48.9 | 54.9 | 23.1 | 25.4 | 25.1 | 17.2 | 70 | 91.8 | 41.5 | 53.5 | 58 | 58.8 | |
| Qwen2.5-7B-Instruct-1MCompression Type=Full-Context (No Compression)2026.05 | 45.3 | 25.6 | 61 | 49.8 | 23 | 30 | 18.6 | 12.1 | 59.4 | 86.9 | 36.5 | 42 | 29.8 | 21.7 | |
| Qwen3-8BCompression Type=Full-Context (No Compression)2026.05 | 44.7 | 26.1 | 63.2 | 52.9 | 19.6 | 26.9 | 23.9 | 16.2 | 70.5 | 88 | 36.2 | 47.4 | 40.9 | 44.9 | |
| LLaMA-3.1-8B-InstructCompression Type=Full-Context (No Compression)2026.05 | 44.6 | 26.3 | 62.2 | 55 | 23.3 | 32.4 | 24.2 | 16.2 | 19.3 | 89.1 | 7.6 | 0 | 42.8 | 46.4 | |
| GLM-4-9B-Chat-1MCompression Type=Full-Context (No Compression)2026.05 | 43.8 | 26.7 | 63 | 53.6 | 22.8 | 27.6 | 21 | 12.2 | 61.5 | 90.1 | 39.2 | 28.7 | 55.6 | 59.5 | |
| GlyphCompression Type=Visual-based Compression, Compression Ratio=3-4x2026.05 | 40.6 | 28.5 | 37.2 | 45.9 | 19.8 | 25.5 | 21.5 | 12.4 | 82.6 | 88.5 | 32.5 | 44.4 | 60.8 | 48.9 |