Answer Extraction and Supporting Sentence Prediction on HotpotQA fullwiki (test)
67.5Answer EMAISO
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| AISOPre-trained Model Scale=large2021.09 | 67.5 | 80.5 | — | — | — | — | |
| TPRRPre-trained Model Scale=large, Source=Official Leaderboard2021.09 | 67 | 79.5 | — | — | — | — | |
| IRRR++Source=Official Leaderboard2021.09 | 66.3 | 79.9 | — | — | — | — | |
| EBSPre-trained Model Scale=large, Source=Official Leaderboard2021.09 | 66.2 | 79.3 | — | — | — | — | |
| HopRetriever-plusSetting=fullwiki2020.12 | 64.83 | 77.81 | 56.08 | 81.79 | 40.95 | 67.75 | |
| HopRetriever-plusVariant=+2021.09 | 64.8 | 77.8 | — | — | — | — | |
| DDRQA2021.09 | 62.5 | 75.9 | — | — | — | — | |
| MDRVariant=+2021.09 | 62.3 | 75.3 | — | — | — | — | |
| AISOPre-trained Model Scale=large2021.09 | 61.2 | 86 | — | — | — | — | |
| HopRetrieverSetting=fullwiki2020.12 | 60.83 | 73.93 | 53.07 | 79.26 | 38 | 63.91 | |
| HopRetriever2021.09 | 60.8 | 73.9 | — | — | — | — | |
| PathRetrieverSetting=fullwiki2020.12 | 60.04 | 72.96 | 49.08 | 76.41 | 35.35 | 61.18 | |
| GRRVariant=+2021.09 | 60 | 73 | — | — | — | — | |
| Semantic Retrieval + HGNSetting=fullwiki2020.12 | 59.74 | 71.41 | 51.03 | 77.37 | 37.92 | 62.26 | |
| TPRRPre-trained Model Scale=large, Source=Official Leaderboard2021.09 | 59.4 | 84.3 | — | — | — | — | |
| MDRVariant=+2021.09 | 57.5 | 80.9 | — | — | — | — | |
| EBSPre-trained Model Scale=large, Source=Official Leaderboard2021.09 | 57.3 | 84 | — | — | — | — | |
| IRRR++Source=Official Leaderboard2021.09 | 57.2 | 82.6 | — | — | — | — | |
| HopRetriever-plusVariant=+2021.09 | 56.1 | 81.8 | — | — | — | — | |
| HopRetriever2021.09 | 53.1 | 79.3 | — | — | — | — | |
| Transformer-XHSetting=fullwiki2020.12 | 51.6 | 64.07 | 40.91 | 71.42 | 26.14 | 51.29 | |
| DDRQA2021.09 | 51 | 78.9 | — | — | — | — | |
| GRRVariant=+2021.09 | 49.1 | 76.4 | — | — | — | — | |
| Semantic RetrievalSetting=fullwiki2020.12 | 45.32 | 57.34 | 38.67 | 70.83 | 25.14 | 47.6 | |
| Semantic Retrieval2021.09 | 45.3 | 57.3 | — | — | — | — | |
| AISOPre-trained Model Scale=large2021.09 | 44.9 | 72 | — | — | — | — | |
| TPRRPre-trained Model Scale=large, Source=Official Leaderboard2021.09 | 44.4 | 70.8 | — | — | — | — | |
| IRRR++Source=Official Leaderboard2021.09 | 43.1 | 69.8 | — | — | — | — | |
| DrKITSetting=fullwiki2020.12 | 42.13 | 51.72 | 37.05 | 59.84 | 24.69 | 42.8 | |
| EBSPre-trained Model Scale=large, Source=Official Leaderboard2021.09 | 42 | 70 | — | — | — | — | |
| MDRVariant=+2021.09 | 41.8 | 66.6 | — | — | — | — | |
| HopRetriever-plusVariant=+2021.09 | 41 | 67.8 | — | — | — | — | |
| Semantic Retrieval2021.09 | 38.7 | 70.8 | — | — | — | — | |
| HopRetriever2021.09 | 38 | 63.9 | — | — | — | — | |
| GoldEn Retriever2021.09 | 37.9 | 49.8 | — | — | — | — | |
| Cognitive Graph QASetting=fullwiki2020.12 | 37.12 | 48.87 | 22.82 | 57.69 | 12.42 | 34.92 | |
| CogQA2021.09 | 37.1 | 48.9 | — | — | — | — | |
| DDRQA2021.09 | 36 | 63.9 | — | — | — | — | |
| GRRVariant=+2021.09 | 35.4 | 61.2 | — | — | — | — | |
| GoldEn Retriever2021.09 | 30.7 | 64.6 | — | — | — | — | |
| MUPPET2021.09 | 30.6 | 40.3 | — | — | — | — | |
| DecompRCSetting=fullwiki2020.12 | 30 | 40.65 | — | — | — | — | |
| Semantic Retrieval2021.09 | 25.1 | 47.6 | — | — | — | — | |
| CogQA2021.09 | 22.8 | 57.7 | — | — | — | — | |
| GoldEn Retriever2021.09 | 18 | 39.1 | — | — | — | — | |
| MUPPET2021.09 | 16.7 | 47.3 | — | — | — | — | |
| CogQA2021.09 | 12.4 | 34.9 | — | — | — | — | |
| MUPPET2021.09 | 10.9 | 27 | — | — | — | — |