Question Answering on LongBench HotpotQA (F1)
63.58F1 ScoreCORE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| COREGenerator LLM=Qwen2.5-14B-Instruct, Compressor model=Qwen2.5-1.5B-Instruct2025.08 | 63.58 | 126 | |
| Full ContextGenerator LLM=Qwen2.5-14B-Instruct2025.08 | 62.22 | 9,151 | |
| LongLLMLinguaGenerator LLM=Qwen2.5-14B-Instruct, Compressor model=Qwen2.5-1.5B-Instruct2025.08 | 57.69 | 907 | |
| No ContextGenerator LLM=Qwen2.5-14B-Instruct2025.08 | 51.13 | 0 | |
| RandomContext Budget (B)=20482026.02 | 28.4 | — | |
| CurvPruneContext Budget (B)=20482026.02 | 27.8 | — | |
| BM25Context Budget (B)=20482026.02 | 27.3 | — | |
| BM25+TexContext Budget (B)=20482026.02 | 27 | — | |
| Sel. CtxContext Budget (B)=20482026.02 | 20.1 | — | |
| LLMLinguaContext Budget (B)=20482026.02 | 18.8 | — | |
| RecencyContext Budget (B)=20482026.02 | 12.2 | — | |
| Head+TailContext Budget (B)=20482026.02 | 8.2 | — |