End-to-end Question Answering on MuSiQue (test val)
10.79EMSetR-CoT & IRI
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SetR-CoT & IRIRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=SET SELECTION, # of Passages=2.912025.07 | 10.79 | 15.43 | |
| SetR-Selection onlyRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=SET SELECTION, # of Passages=3.412025.07 | 10.38 | 15.28 | |
| SetR-CoTRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=SET SELECTION, # of Passages=2.882025.07 | 9.76 | 14.31 | |
| RankGPT (gpt-4o)Retrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.07 | 9.43 | 13.25 | |
| bge-reranker-largeRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.07 | 8.06 | 12.5 | |
| SetR-Selection onlyRetrieval (First-stage)=BM25, Ranking Strategy=SET SELECTION, # of Passages=2.952025.07 | 8.02 | 11.07 | |
| RankZephyrRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.07 | 8.02 | 11.72 | |
| RankVicunaRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.07 | 7.78 | 11.35 | |
| RankLlamaRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.07 | 7.61 | 11.77 | |
| bge-large-en-v1.5Retrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RETRIEVAL ONLY, # of Passages=52025.07 | 7.44 | 10.78 | |
| SetR-CoTRetrieval (First-stage)=BM25, Ranking Strategy=SET SELECTION, # of Passages=2.482025.07 | 7.03 | 10.87 | |
| RankGPT (gpt-4o)Retrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.07 | 6.91 | 9.98 | |
| RankZephyrRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.07 | 6.78 | 10.03 | |
| SetR-CoT & IRIRetrieval (First-stage)=BM25, Ranking Strategy=SET SELECTION, # of Passages=2.632025.07 | 6.62 | 10.57 | |
| FirstMistralRetrieval (First-stage)=bge-large-en-v1.5, Ranking Strategy=RERANKING, # of Passages=52025.07 | 6.53 | 10.64 | |
| bge-reranker-largeRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.07 | 6.12 | 10 | |
| RankLlamaRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.07 | 6.04 | 9.26 | |
| RankVicunaRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.07 | 5.99 | 9.03 | |
| BM25Retrieval (First-stage)=BM25, Ranking Strategy=RETRIEVAL ONLY, # of Passages=52025.07 | 5.46 | 8.22 | |
| FirstMistralRetrieval (First-stage)=BM25, Ranking Strategy=RERANKING, # of Passages=52025.07 | 5.29 | 8.42 |