Answer Extraction and Supporting Sentence Prediction on HotpotQA fullwiki (dev)
66.56Answer EMHopRetriever-plus
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| HopRetriever-plusSetting=fullwiki2020.12 | 66.56 | 79.21 | 56.02 | 81.81 | 42.01 | 68.97 | |
| HopRetrieverSetting=fullwiki2020.12 | 62.07 | 75.18 | 52.53 | 78.92 | 37.81 | 64.5 | |
| PathRetrieverSetting=fullwiki2020.12 | 60.49 | 73.3 | 49.16 | 76.05 | 35.82 | 61.43 | |
| Semantic RetrievalSetting=fullwiki2020.12 | 46.41 | 58.7 | 39.86 | 71.53 | 26.53 | 49 | |
| AISOPre-trained Model Scale=large2021.09 | 45.9 | 72.5 | — | — | — | — | |
| TPRRPre-trained Model Scale=large, Source=Official Leaderboard2021.09 | 45.3 | 71.4 | — | — | — | — | |
| MDRVariant=+2021.09 | 42.1 | 66.3 | — | — | — | — | |
| HopRetriever-plusVariant=+2021.09 | 42 | 69 | — | — | — | — | |
| AISOPre-trained Model Scale=base2021.09 | 40.2 | 66.9 | — | — | — | — | |
| HopRetriever2021.09 | 37.8 | 64.5 | — | — | — | — | |
| Cognitive Graph QASetting=fullwiki2020.12 | 37.55 | 49.4 | 23.11 | 58.52 | 12.18 | 35.28 | |
| GRRVariant=+2021.09 | 35.8 | 61.4 | — | — | — | — | |
| Semantic Retrieval2021.09 | 26.6 | 49.2 | — | — | — | — | |
| CogQA2021.09 | 12.2 | 35.3 | — | — | — | — | |
| MUPPET2021.09 | 11.8 | 27.6 | — | — | — | — |