List your company
ResearchFormerly WizWand

Open Domain Question Answering on 2WikiMultihopQA

59.3EMDecomp
23.47232.773542.07551.3765Dec 20, 2022Jun 1, 2023Nov 11, 2023Apr 22, 2024Oct 2, 2024Mar 14, 2025Aug 24, 2025
Updated 5mo ago

Evaluation results

MethodLinks
Decomp2022.1259.3—
IRCoT QABackbone=GPT3 (175B)2022.1257.768
CORE (3B)Downstream LLM=Qwen2.5-14B-Instruct, Compression setting=Compression of top 10 docs (with compressor trained on top 5 docs), # tok=482025.0840.7146.33
SelfAsk2022.1240.1—
CORE (3B)Downstream LLM=Qwen2.5-14B-Instruct, Compression setting=Compression of top 5 docs, # tok=402025.0839.0544.74
RECOMP (3B)Downstream LLM=Qwen2.5-14B-Instruct, Compression setting=Compression of top 10 docs (with compressor trained on top 5 docs), # tok=502025.0833.1238.69
RECOMP (3B)Downstream LLM=Qwen2.5-14B-Instruct, Compression setting=Compression of top 5 docs, # tok=422025.0832.7438.06
Top10 DocumentsDownstream LLM=Qwen2.5-14B-Instruct, Compression setting=None, Retrieval setting=Top10, # tok=15312025.0831.0436.75
Top5 DocumentsDownstream LLM=Qwen2.5-14B-Instruct, Compression setting=None, Retrieval setting=Top5, # tok=7662025.0829.6435.21
Deepseek-V3 (671B)Downstream LLM=Qwen2.5-14B-Instruct, Compression setting=Compression of top 10 docs (with compressor trained on top 5 docs), # tok=402025.082934.64
Top3 DocumentsDownstream LLM=Qwen2.5-14B-Instruct, Compression setting=None, Retrieval setting=Top3, # tok=4602025.0827.8933.58
Top1 DocumentDownstream LLM=Qwen2.5-14B-Instruct, Compression setting=None, Retrieval setting=Top1, # tok=1532025.0826.7931.87
No RetrievalDownstream LLM=Qwen2.5-14B-Instruct, Compression setting=None, # tok=02025.0826.1129.51
Deepseek-V3 (671B)Downstream LLM=Qwen2.5-14B-Instruct, Compression setting=Compression of top 5 docs, # tok=452025.0825.0730.31
llama3.2-3BDownstream LLM=Qwen2.5-14B-Instruct, Compression setting=Compression of top 10 docs (with compressor trained on top 5 docs), # tok=592025.0824.9130.11
llama3.2-3BDownstream LLM=Qwen2.5-14B-Instruct, Compression setting=Compression of top 5 docs, # tok=602025.0824.8529.95