List your company
ResearchFormerly WizWand

Question Answering on NarrativeQA (F1, EM)

33.61F1QRRanker
8.015614.660321.30527.9497Feb 2, 2026Feb 3, 2026Feb 5, 2026Feb 7, 2026Feb 8, 2026Feb 10, 2026Feb 12, 2026
Updated 8mo ago

Evaluation results

MethodLinks
QRRankerMethod Category=QRRanker Series, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.0233.6116.04
RAMBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.0232.471,964
RAMBackbone=Qwen3-4B-Instruct, Compression=4x2026.0231.921,945
RAMBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.0231.571,880
QRHeads-4BMethod Category=QRRanker Series, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.0231.414.7
RAMBackbone=Qwen3-4B-Instruct, Compression=8x2026.0231.221,992
Qwen3-Reranker-4B (trained)Method Category=Reranking Methods, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.0230.5113.52
ProvenceBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.0230.111,513
ProvenceBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.0229.151,410
Qwen3-Reranker-4B (vanilla)Method Category=Reranking Methods, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.0229.112.58
SFT-Embedding-8BMethod Category=Embedding Methods, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.0228.4812.11
Activation BeaconBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.0228.291,861
Activation BeaconBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.0226.81,795
Qwen3-Embedding-8BMethod Category=Embedding Methods, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.0226.311.01
LLMLingua-2-largeBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.0224.931,259
ProvenceBackbone=Qwen3-4B-Instruct, Compression=4x2026.0224.461,100
ProvenceBackbone=Qwen3-4B-Instruct, Compression=8x2026.0224.341,071
EXITBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.0224.171,100
LLMLingua-2-largeBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.0222.461,128
Original PromptBackbone=Qwen3-4B-Instruct2026.0220.06855
Original PromptBackbone=LLaMA-3.1-8B-Instruct2026.0219.26639
LongLLMLinguaBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.0219.17715
LLMLingua-2-largeBackbone=Qwen3-4B-Instruct, Compression=4x2026.0219.03789
EXITBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.0218.62686
LLMLingua-2-largeBackbone=Qwen3-4B-Instruct, Compression=8x2026.0215.99620
LongLLMLinguaBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.0215.84452
EXITBackbone=Qwen3-4B-Instruct, Compression=4x2026.0215.59564
ICAEBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.0212.46470
ICAEBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.0212.41441
LongLLMLinguaBackbone=Qwen3-4B-Instruct, Compression=4x2026.0212.2282
ICAEBackbone=Qwen3-4B-Instruct, Compression=8x2026.0211.75245
ICAEBackbone=Qwen3-4B-Instruct, Compression=4x2026.0211.72253
EXITBackbone=Qwen3-4B-Instruct, Compression=8x2026.0211.53282
LongLLMLinguaBackbone=Qwen3-4B-Instruct, Compression=8x2026.029.76188
Closed-bookBackbone=Qwen3-4B-Instruct2026.029.6547
Closed-bookBackbone=LLaMA-3.1-8B-Instruct2026.029103