Context Compression on BenchPress short-context (test)
56.41EM (4x Single Context)Mean-Pooling
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Mean-PoolingBackbone=Qwen3-8B2025.10 | 56.41 | — | 55.04 | 47.95 | 49.02 | 34.72 | 33.15 | — | |
| Mean-PoolingBackbone=Qwen3-4B2025.10 | 55.25 | — | 53.77 | 45.43 | 45.86 | 30.91 | 28.38 | — | |
| Mean-PoolingBackbone=Gemma2-2B2025.10 | 54.2 | — | 52.77 | 45.88 | 45.68 | 32.41 | 30.83 | — | |
| Compression-Tokens (Bidirectional)Backbone=Qwen3-8B2025.10 | 53.44 | — | 53.99 | 44.92 | 47.15 | 33.6 | 34.27 | — | |
| Compression-Tokens (Causal)Backbone=Qwen3-8B2025.10 | 51.59 | — | 50.01 | 41.26 | 42.99 | 34.21 | 32.5 | — | |
| Mean-PoolingBackbone=Qwen3-1.7B2025.10 | 51.28 | — | 48.77 | 40.45 | 39.04 | 25.15 | 22.01 | — | |
| Compression-Tokens (Bidirectional)Backbone=Qwen3-4B2025.10 | 51.06 | — | 52.37 | 42.53 | 45.14 | 28.84 | 29.51 | — | |
| Compression-Tokens (Causal)Backbone=Qwen3-4B2025.10 | 49.66 | — | 46.26 | 40.05 | 39.59 | 30.07 | 28.84 | — | |
| Compression-Tokens (Bidirectional)Backbone=Gemma2-2B2025.10 | 49.43 | — | 49.4 | 40.89 | 42.7 | 31.79 | 30.43 | — | |
| PCC LargeBackbone=Llama3.1-8B2025.10 | 49.34 | — | — | 36.64 | — | 27.92 | — | — | |
| PCC LiteBackbone=GPT2-Large & Llama3.1-8B2025.10 | 48.81 | — | — | 38.38 | — | 25.94 | — | — | |
| Mean-PoolingBackbone=Llama3.2-1B2025.10 | 47.97 | — | 45.45 | 33.15 | 36.93 | 22.89 | 22.7 | — | |
| Compression-Tokens (Causal)Backbone=Gemma2-2B2025.10 | 47.9 | — | 45.98 | 39.57 | 39.74 | 32.02 | 29.66 | — | |
| Compression-Tokens (Bidirectional)Backbone=Qwen3-1.7B2025.10 | 46.45 | — | 46.72 | 36.62 | 38.93 | 24.31 | 24.33 | — | |
| Mean-PoolingBackbone=Qwen3-0.6B2025.10 | 45.82 | — | 43.07 | 32.5 | 33.09 | 19.05 | 16.07 | — | |
| Compression-Tokens (Bidirectional)Backbone=Llama3.2-1B2025.10 | 43.46 | — | 42.96 | 35.04 | 35.46 | 24.91 | 24.56 | — | |
| Compression-Tokens (Causal)Backbone=Llama3.2-1B2025.10 | 41.84 | — | 39.03 | 33.73 | 33.38 | 24.11 | 24.64 | — | |
| Compression-Tokens (Bidirectional)Backbone=Qwen3-0.6B2025.10 | 40.98 | — | 41.92 | 30.92 | 33.64 | 18.82 | 18.55 | — | |
| Compression-Tokens (Causal)Backbone=Qwen3-0.6B2025.10 | 39.66 | — | 36.76 | 27.99 | 29 | 18.23 | 18.2 | — | |
| Compression-Tokens (Causal)Backbone=Qwen3-1.7B2025.10 | 36.66 | — | 41.64 | 35.49 | 34.64 | 24.27 | 24.04 | — | |
| ICAEBackbone=Mistral-7B2025.10 | 24.94 | — | — | — | — | — | — | — | |
| Gemma2-2BBackbone=Gemma2-2B2025.10 | — | 57.63 | — | — | — | — | — | 15 | |
| Llama3.2-1BBackbone=Llama3.2-1B2025.10 | — | 51.67 | — | — | — | — | — | 9.47 | |
| LLMLingua2Backbone=Qwen3-8B2025.10 | — | — | 30.02 | — | 16.16 | — | 16.06 | — | |
| Qwen3-0.6BBackbone=Qwen3-0.6B2025.10 | — | 50.85 | — | — | — | — | — | 4.78 | |
| Qwen3-1.7BBackbone=Qwen3-1.7B2025.10 | — | 55.19 | — | — | — | — | — | 9.07 | |
| Qwen3-4BBackbone=Qwen3-4B2025.10 | — | 58.87 | — | — | — | — | — | 13.74 | |
| Qwen3-8BBackbone=Qwen3-8B2025.10 | — | 59.82 | — | — | — | — | — | 16.19 |