List your company
ResearchFormerly WizWand

End-to-end Question Answering on HotpotQA (test val)

36.68EMSetR-Selection only
26.311229.003131.69534.3869Jul 9, 2025
Updated 8mo ago

Evaluation results

MethodLinks
SetR-Selection onlyRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=SET SELECTION, # of Passages=3.412025.0736.6837.84
SetR-CoT & IRIRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=SET SELECTION, # of Passages=2.912025.0736.6238.11
SetR-CoTRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=SET SELECTION, # of Passages=2.882025.0736.4638.2
RankGPT (gpt-4o)Retrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.0733.8534.45
bge-reranker-largeRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.0732.4833.24
SetR-CoT & IRIRetrieval (First-stage)=BM25, Ranking Strategy=SET SELECTION, # of Passages=2.632025.0732.230.57
RankVicunaRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.0732.0832.83
RankLlamaRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.0731.8832.95
RankZephyrRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.0731.8332.97
SetR-Selection onlyRetrieval (First-stage)=BM25, Ranking Strategy=SET SELECTION, # of Passages=2.952025.0731.6130.55
RankGPT (gpt-4o)Retrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.0730.8929.24
SetR-CoTRetrieval (First-stage)=BM25, Ranking Strategy=SET SELECTION, # of Passages=2.482025.0730.7930.12
FirstMistralRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.0730.131.07
bge-large-en-v1.5Retrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RETRIEVAL ONLY, # of Passages=52025.0730.0730.97
bge-reranker-largeRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.0729.7128.08
RankLlamaRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.0729.4827.82
RankZephyrRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.0728.9627.76
RankVicunaRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.0728.6927.31
BM25Retrieval (First-stage)=BM25, Ranking Strategy=RETRIEVAL ONLY, # of Passages=52025.0726.925.86
FirstMistralRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.0726.7126.1