Multi-Doc Question Answering on LongBench-E
49.6F1 ScoreTeacher Model (w/ Context)
Evaluation results
| Method | Links | |
|---|---|---|
| Teacher Model (w/ Context)Backbone=Qwen3-1.7B, Compression=no compression2025.10 | 49.6 | |
| Mean-PoolingBackbone=Qwen3-1.7B, Compression Ratio=4x2025.10 | 45.9 | |
| Compression-Tokens (Bidirectional)Backbone=Qwen3-1.7B, Compression Ratio=4x2025.10 | 43 | |
| Mean-PoolingBackbone=Qwen3-1.7B, Compression Ratio=16x2025.10 | 41.4 | |
| Compression-Tokens (Causal)Backbone=Qwen3-1.7B, Compression Ratio=4x2025.10 | 40.9 | |
| Compression-Tokens (Bidirectional)Backbone=Qwen3-1.7B, Compression Ratio=16x2025.10 | 38.2 | |
| Mean-PoolingBackbone=Qwen3-1.7B, Compression Ratio=128x2025.10 | 36 | |
| Compression-Tokens (Causal)Backbone=Qwen3-1.7B, Compression Ratio=16x2025.10 | 32.5 | |
| Compression-Tokens (Bidirectional)Backbone=Qwen3-1.7B, Compression Ratio=128x2025.10 | 32.1 | |
| Compression-Tokens (Causal)Backbone=Qwen3-1.7B, Compression Ratio=128x2025.10 | 31.6 | |
| LLMLingua2Backbone=Qwen3-1.7B, Compression Ratio=4x2025.10 | 29.4 | |
| PCC LargeBackbone=Llama3.1-8B, Compression Ratio=4x2025.10 | 28 | |
| LLMLingua2Backbone=Qwen3-1.7B, Compression Ratio=16x2025.10 | 22.4 | |
| LLMLingua2Backbone=Qwen3-1.7B, Compression Ratio=128x2025.10 | 21.8 | |
| Teacher Model (w/o Context)Backbone=Qwen3-1.7B, Compression=no compression2025.10 | 21.7 | |
| PCC LargeBackbone=Llama3.1-8B, Compression Ratio=16x2025.10 | 11.4 | |
| PCC LargeBackbone=Llama3.1-8B, Compression Ratio=128x2025.10 | 7.2 |