Question Answering on HotpotQA (test) (F1, EM, CR, TP)
65.49Exact Match (EM)ATACompressor
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ATACompressorBackbone=Qwen-2-7B2026.02 | 65.49 | 80.23 | 23.81 | 3.63 | |
| ATACompressorBackbone=LLaMA-2-7B2026.02 | 62.65 | 78.44 | 24.15 | 3.86 | |
| QGCBackbone=Qwen-2-7B2026.02 | 51.5 | 72.36 | 13.98 | 1.83 | |
| QGCBackbone=LLaMA-2-7B2026.02 | 45.12 | 68.18 | 14.58 | 1.95 | |
| 500CompressorBackbone=Qwen-2-7B2026.02 | 42.15 | 67.4 | 23.16 | 3.51 | |
| LongLLMLinguaBackbone=Qwen-2-7B2026.02 | 40.05 | 64.6 | 4.67 | 1.28 | |
| Original-ContextBackbone=Qwen-2-7B2026.02 | 39.73 | 59.88 | 1 | 1.24 | |
| ICAEBackbone=Qwen-2-7B2026.02 | 39.68 | 65.35 | 23.16 | 3.6 | |
| 500CompressorBackbone=LLaMA-2-7B2026.02 | 39.6 | 64.3 | 23.16 | 3.69 | |
| ICAEBackbone=LLaMA-2-7B2026.02 | 37.7 | 62.08 | 23.16 | 3.76 | |
| LongLLMLinguaBackbone=LLaMA-2-7B2026.02 | 37.3 | 62.8 | 4.22 | 1.31 | |
| Selective-ContextBackbone=Qwen-2-7B2026.02 | 37.08 | 53.7 | 3.73 | 1.24 | |
| Original-ContextBackbone=LLaMA-2-7B2026.02 | 36.2 | 53.71 | 1 | 1.21 | |
| Selective-ContextBackbone=LLaMA-2-7B2026.02 | 36.05 | 51.65 | 3.52 | 1.21 | |
| AutoCompressorBackbone=LLaMA-2-7B2026.02 | 32 | 59.64 | 12.23 | 2.95 | |
| PPO + PRS_shortBackbone=Qwen2.5-3B-Instruct2025.12 | 32 | — | — | — | |
| GRPO + PRS_shortBackbone=Qwen2.5-3B-Instruct2025.12 | 31 | — | — | — | |
| VSPO + PRS_shortBackbone=Qwen2.5-3B-Instruct2025.12 | 31 | — | — | — | |
| SFTBackbone=Qwen2.5-3B-Instruct2025.12 | 18.6 | — | — | — | |
| Closed-bookBackbone=Qwen-2-7B2026.02 | 10.3 | 30.58 | — | 5.64 | |
| UntrainedBackbone=Qwen2.5-3B-Instruct2025.12 | 6.22 | — | — | — | |
| Closed-bookBackbone=LLaMA-2-7B2026.02 | 4.8 | 22.8 | — | 6.37 |