Question Answering on NarrativeQA (F1, EM)
33.61F1QRRanker
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| QRRankerMethod Category=QRRanker Series, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.02 | 33.61 | 16.04 | |
| RAMBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.02 | 32.47 | 1,964 | |
| RAMBackbone=Qwen3-4B-Instruct, Compression=4x2026.02 | 31.92 | 1,945 | |
| RAMBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.02 | 31.57 | 1,880 | |
| QRHeads-4BMethod Category=QRRanker Series, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.02 | 31.4 | 14.7 | |
| RAMBackbone=Qwen3-4B-Instruct, Compression=8x2026.02 | 31.22 | 1,992 | |
| Qwen3-Reranker-4B (trained)Method Category=Reranking Methods, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.02 | 30.51 | 13.52 | |
| ProvenceBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.02 | 30.11 | 1,513 | |
| ProvenceBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.02 | 29.15 | 1,410 | |
| Qwen3-Reranker-4B (vanilla)Method Category=Reranking Methods, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.02 | 29.1 | 12.58 | |
| SFT-Embedding-8BMethod Category=Embedding Methods, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.02 | 28.48 | 12.11 | |
| Activation BeaconBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.02 | 28.29 | 1,861 | |
| Activation BeaconBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.02 | 26.8 | 1,795 | |
| Qwen3-Embedding-8BMethod Category=Embedding Methods, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.02 | 26.3 | 11.01 | |
| LLMLingua-2-largeBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.02 | 24.93 | 1,259 | |
| ProvenceBackbone=Qwen3-4B-Instruct, Compression=4x2026.02 | 24.46 | 1,100 | |
| ProvenceBackbone=Qwen3-4B-Instruct, Compression=8x2026.02 | 24.34 | 1,071 | |
| EXITBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.02 | 24.17 | 1,100 | |
| LLMLingua-2-largeBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.02 | 22.46 | 1,128 | |
| Original PromptBackbone=Qwen3-4B-Instruct2026.02 | 20.06 | 855 | |
| Original PromptBackbone=LLaMA-3.1-8B-Instruct2026.02 | 19.26 | 639 | |
| LongLLMLinguaBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.02 | 19.17 | 715 | |
| LLMLingua-2-largeBackbone=Qwen3-4B-Instruct, Compression=4x2026.02 | 19.03 | 789 | |
| EXITBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.02 | 18.62 | 686 | |
| LLMLingua-2-largeBackbone=Qwen3-4B-Instruct, Compression=8x2026.02 | 15.99 | 620 | |
| LongLLMLinguaBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.02 | 15.84 | 452 | |
| EXITBackbone=Qwen3-4B-Instruct, Compression=4x2026.02 | 15.59 | 564 | |
| ICAEBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.02 | 12.46 | 470 | |
| ICAEBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.02 | 12.41 | 441 | |
| LongLLMLinguaBackbone=Qwen3-4B-Instruct, Compression=4x2026.02 | 12.2 | 282 | |
| ICAEBackbone=Qwen3-4B-Instruct, Compression=8x2026.02 | 11.75 | 245 | |
| ICAEBackbone=Qwen3-4B-Instruct, Compression=4x2026.02 | 11.72 | 253 | |
| EXITBackbone=Qwen3-4B-Instruct, Compression=8x2026.02 | 11.53 | 282 | |
| LongLLMLinguaBackbone=Qwen3-4B-Instruct, Compression=8x2026.02 | 9.76 | 188 | |
| Closed-bookBackbone=Qwen3-4B-Instruct2026.02 | 9.65 | 47 | |
| Closed-bookBackbone=LLaMA-3.1-8B-Instruct2026.02 | 9 | 103 |