End-to-end Question Answering on 2WikiMultiHopQA (test val)
35.44EMSetR-CoT & IRI
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SetR-CoT & IRIRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=SET SELECTION, # of Passages=2.912025.07 | 35.44 | 30.35 | |
| SetR-CoTRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=SET SELECTION, # of Passages=2.882025.07 | 35.34 | 30.34 | |
| SetR-Selection onlyRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=SET SELECTION, # of Passages=3.412025.07 | 34.84 | 29.4 | |
| RankGPT (gpt-4o)Retrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.07 | 34.36 | 28.06 | |
| RankZephyrRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.07 | 32.68 | 26.59 | |
| RankVicunaRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.07 | 32.66 | 26.85 | |
| RankLlamaRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.07 | 32.24 | 25.78 | |
| SetR-Selection onlyRetrieval (First-stage)=BM25, Ranking Strategy=SET SELECTION, # of Passages=2.952025.07 | 32.22 | 24.2 | |
| SetR-CoT & IRIRetrieval (First-stage)=BM25, Ranking Strategy=SET SELECTION, # of Passages=2.632025.07 | 32.17 | 24.22 | |
| SetR-CoTRetrieval (First-stage)=BM25, Ranking Strategy=SET SELECTION, # of Passages=2.482025.07 | 32.07 | 24.43 | |
| bge-reranker-largeRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.07 | 31.92 | 25.47 | |
| RankGPT (gpt-4o)Retrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.07 | 31.71 | 23.31 | |
| FirstMistralRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.07 | 31.43 | 25.31 | |
| bge-large-en-v1.5Retrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RETRIEVAL ONLY, # of Passages=52025.07 | 31.17 | 25.22 | |
| RankVicunaRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.07 | 30.46 | 22.42 | |
| RankLlamaRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.07 | 30.3 | 21.91 | |
| RankZephyrRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.07 | 30.29 | 22.34 | |
| bge-reranker-largeRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.07 | 30.16 | 21.84 | |
| FirstMistralRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.07 | 30.15 | 21.97 | |
| BM25Retrieval (First-stage)=BM25, Ranking Strategy=RETRIEVAL ONLY, # of Passages=52025.07 | 29.79 | 21.79 |