Single-Doc Question Answering on LongBench-E
43.3F1 ScoreTeacher Model (w/ Context)
Evaluation results
| Method | Links | |
|---|---|---|
| Teacher Model (w/ Context)Backbone=Qwen3-1.7B, Compression=no compression2025.10 | 43.3 | |
| Mean-PoolingBackbone=Qwen3-1.7B, Compression Ratio=4x2025.10 | 39.7 | |
| Compression-Tokens (Bidirectional)Backbone=Qwen3-1.7B, Compression Ratio=4x2025.10 | 35.9 | |
| Compression-Tokens (Causal)Backbone=Qwen3-1.7B, Compression Ratio=4x2025.10 | 33.3 | |
| Mean-PoolingBackbone=Qwen3-1.7B, Compression Ratio=16x2025.10 | 32.5 | |
| Compression-Tokens (Bidirectional)Backbone=Qwen3-1.7B, Compression Ratio=16x2025.10 | 30.5 | |
| Mean-PoolingBackbone=Qwen3-1.7B, Compression Ratio=128x2025.10 | 24.2 | |
| LLMLingua2Backbone=Qwen3-1.7B, Compression Ratio=4x2025.10 | 20.7 | |
| Compression-Tokens (Causal)Backbone=Qwen3-1.7B, Compression Ratio=16x2025.10 | 19.5 | |
| Compression-Tokens (Bidirectional)Backbone=Qwen3-1.7B, Compression Ratio=128x2025.10 | 19.1 | |
| Compression-Tokens (Causal)Backbone=Qwen3-1.7B, Compression Ratio=128x2025.10 | 17.9 | |
| PCC LargeBackbone=Llama3.1-8B, Compression Ratio=4x2025.10 | 17.2 | |
| LLMLingua2Backbone=Qwen3-1.7B, Compression Ratio=16x2025.10 | 12.5 | |
| Teacher Model (w/o Context)Backbone=Qwen3-1.7B, Compression=no compression2025.10 | 10.1 | |
| LLMLingua2Backbone=Qwen3-1.7B, Compression Ratio=128x2025.10 | 8.9 | |
| PCC LargeBackbone=Llama3.1-8B, Compression Ratio=16x2025.10 | 5.6 | |
| PCC LargeBackbone=Llama3.1-8B, Compression Ratio=128x2025.10 | 2.4 |