Multi-hop Question Answering on MuSiQue (F1, ROUGHL, EM)
72F1 ScoreDIRECT QUERY
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DIRECT QUERYPRIVATE=NO, Backbone=GPT-4O2023.12 | 72 | 72 | 56 | |
| CONFUSIONPROMPTPRIVATE=YES, Backbone=GPT-4O2023.12 | 68 | 68 | 54 | |
| DIRECT QUERYPRIVATE=NO, Backbone=GPT-4-TURBO2023.12 | 66 | 66 | 50 | |
| CONFUSIONPROMPTPRIVATE=YES, Backbone=GPT-4-TURBO2023.12 | 63 | 63 | 50 | |
| CONFUSIONPROMPTPRIVATE=YES, Backbone=GPT-3.5-TURBO2023.12 | 61 | 61 | 45 | |
| DIRECT QUERYPRIVATE=NO, Backbone=GPT-3.5-TURBO2023.12 | 60 | 61 | 44 | |
| SAGEvariant=+2 round, Avg. Rank=2.52026.05 | 53.1 | — | 34.2 | |
| SAGEvariant=+ IRCoT, Avg. Rank=3.32026.05 | 52.6 | — | 35.2 | |
| SAGEvariant=+1 round, Avg. Rank=4.22026.05 | 52.3 | — | 32.2 | |
| GFM-RAG + IRCoTAvg. Rank=3.72026.05 | 49.2 | — | 36.6 | |
| LLAMA2-7BPRIVATE=YES2023.12 | 48 | 48 | 32 | |
| SAGEAvg. Rank=7.52026.05 | 47.3 | — | 28 | |
| VICUNA-13BPRIVATE=YES2023.12 | 42 | 42 | 23 | |
| PropRAGAvg. Rank=3.82026.05 | 41.3 | — | 38.9 | |
| HippoRAG 2Avg. Rank=4.72026.05 | 40.5 | — | 33.9 | |
| GFM-RAGAvg. Rank=7.82026.05 | 40.4 | — | 30.2 | |
| SubgraphRAGAvg. Rank=7.52026.05 | 39.2 | — | 33 | |
| HippoRAG + IRCoTAvg. Rank=11.32026.05 | 33.3 | — | 21.9 | |
| ColBERTv2 + IRCoTAvg. Rank=14.42026.05 | 30.5 | — | 19.1 | |
| HippoRAGAvg. Rank=13.82026.05 | 29.8 | — | 19.2 | |
| GraphRAGAvg. Rank=18.02026.05 | 29.5 | — | 13.4 | |
| RAPTORAvg. Rank=15.82026.05 | 28.9 | — | 17.6 | |
| FLAREAvg. Rank=12.72026.05 | 28.4 | — | 16.2 | |
| LightRAGAvg. Rank=16.72026.05 | 27.5 | — | 18.1 | |
| ColBERTv2Avg. Rank=17.32026.05 | 26.4 | — | 15.5 | |
| G-RetrieverAvg. Rank=18.22026.05 | 25.9 | — | 18 | |
| Adaptive-RAGAvg. Rank=14.32026.05 | 25.6 | — | 13.8 | |
| GTRAvg. Rank=19.72026.05 | 25.2 | — | 15.1 | |
| BM25Avg. Rank=15.52026.05 | 23.6 | — | 19.5 | |
| ContrieverAvg. Rank=20.52026.05 | 21.6 | — | 16.3 | |
| PARAPHRASERPRIVATE=YES, Backbone=GPT-3.5-TURBO2023.12 | 8 | 8 | 3 | |
| PARAPHRASERPRIVATE=YES, Backbone=GPT-4O2023.12 | 7 | 7 | 4 | |
| PARAPHRASERPRIVATE=YES, Backbone=GPT-4-TURBO2023.12 | 6 | 6 | 3 | |
| TEXT2TEXTPRIVATE=YES, Backbone=GPT-4-TURBO2023.12 | 3 | 3 | 1 | |
| TEXT2TEXTPRIVATE=YES, Backbone=GPT-3.5-TURBO2023.12 | 2 | 2 | 1 | |
| TEXT2TEXTPRIVATE=YES, Backbone=GPT-4O2023.12 | 2 | 2 | 1 |