List your company
ResearchFormerly WizWand

End-to-end Question Answering on 2WikiMultiHopQA (test val)

35.44EMSetR-CoT & IRI
29.56431.089532.61534.1405Jul 9, 2025
Updated 8mo ago

Evaluation results

MethodLinks
SetR-CoT & IRIRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=SET SELECTION, # of Passages=2.912025.0735.4430.35
SetR-CoTRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=SET SELECTION, # of Passages=2.882025.0735.3430.34
SetR-Selection onlyRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=SET SELECTION, # of Passages=3.412025.0734.8429.4
RankGPT (gpt-4o)Retrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.0734.3628.06
RankZephyrRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.0732.6826.59
RankVicunaRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.0732.6626.85
RankLlamaRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.0732.2425.78
SetR-Selection onlyRetrieval (First-stage)=BM25, Ranking Strategy=SET SELECTION, # of Passages=2.952025.0732.2224.2
SetR-CoT & IRIRetrieval (First-stage)=BM25, Ranking Strategy=SET SELECTION, # of Passages=2.632025.0732.1724.22
SetR-CoTRetrieval (First-stage)=BM25, Ranking Strategy=SET SELECTION, # of Passages=2.482025.0732.0724.43
bge-reranker-largeRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.0731.9225.47
RankGPT (gpt-4o)Retrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.0731.7123.31
FirstMistralRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.0731.4325.31
bge-large-en-v1.5Retrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RETRIEVAL ONLY, # of Passages=52025.0731.1725.22
RankVicunaRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.0730.4622.42
RankLlamaRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.0730.321.91
RankZephyrRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.0730.2922.34
bge-reranker-largeRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.0730.1621.84
FirstMistralRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.0730.1521.97
BM25Retrieval (First-stage)=BM25, Ranking Strategy=RETRIEVAL ONLY, # of Passages=52025.0729.7921.79