Question Answering on SearchQA
78EMComprExIT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ComprExITBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.02 | 78 | 84.4 | |
| ICAEBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.02 | 76.43 | 82.89 | |
| Prompt tuningBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Prompt tuning, Context Setting=w/ context2026.02 | 72.04 | 78.42 | |
| ComprExITBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.02 | 71.91 | 78.93 | |
| SeCoBackbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 69.77 | 78.22 | |
| 500xBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.02 | 69.26 | 76.28 | |
| SeCoBackbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 68.6 | 76.66 | |
| SeCoBackbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 68 | 75.67 | |
| ICAEBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.02 | 67.4 | 74.43 | |
| SeCoBackbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 67.29 | 75.59 | |
| Prompt tuningBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Prompt tuning, Context Setting=w/ context2026.02 | 64.03 | 70.64 | |
| COMIBackbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 62.88 | 70.65 | |
| 500x (EPL)Backbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 59.45 | 71.93 | |
| COMIBackbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 59.05 | 66.69 | |
| 500x (EPL)Backbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 58.28 | 70.56 | |
| 500x (DPL)Backbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 56.65 | 71.69 | |
| SAC (EPL)Backbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 56.5 | 69.66 | |
| SAC (EPL)Backbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 56.41 | 69.45 | |
| 500x (DPL)Backbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 55.91 | 69.28 | |
| ICAE (EPL)Backbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 53.56 | 66.95 | |
| RAMBackbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 51.47 | 61.69 | |
| RAMBackbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 49.83 | 56.26 | |
| COMIBackbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 49.27 | 55.96 | |
| 500x (EPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 48.49 | 62.25 | |
| SAC (EPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 47.66 | 61.47 | |
| 500x (DPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 47.34 | 61 | |
| RAMBackbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 46.8 | 56.83 | |
| SAC (EPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 46.73 | 60.56 | |
| 500x (DPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 46.63 | 59.93 | |
| COMIBackbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 46.5 | 52.61 | |
| ICAE (EPL)Backbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 46.4 | 60.19 | |
| RAMBackbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 46.13 | 52.36 | |
| 500x (EPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 44.75 | 58.16 | |
| ICAE (EPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 44.17 | 57.67 | |
| ICAE (EPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 44.13 | 58.21 | |
| DOPAModel=Qwen3-1.7B2026.04 | 40.95 | 52.56 | |
| KNNModel=Qwen3-1.7B2026.04 | 39.35 | 51.96 | |
| DOPAModel=LLaMA3.2-3B2026.04 | 34.5 | 53.72 | |
| KNNModel=LLaMA3.2-3B2026.04 | 33.1 | 52.46 | |
| Zero-shotBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/ context2026.02 | 29.4 | 37.26 | |
| Zero-shotBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/o context2026.02 | 25.43 | 30.35 | |
| 500xBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.02 | 17.44 | 25.59 | |
| Zero-shotBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/ context2026.02 | 5.83 | 12.39 | |
| BeaconBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.02 | 5.24 | 9.13 | |
| Zero-shotBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/o context2026.02 | 4.04 | 6.64 | |
| BeaconBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.02 | 1.9 | 9.86 |