Question Answering on HotpotQA distractor setting (test)
82.2Answer F1HGN-large
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| HGN-largeSize=large2020.04 | 82.2 | — | — | 88.5 | — | 74.2 | — | |
| HGNContext encoding backbone=RoBERTa-large2019.11 | 82.19 | 69.22 | 62.76 | 88.47 | 47.11 | 74.21 | — | |
| GSAN-largeSize=large2020.04 | 81.6 | — | — | 88.7 | — | 73.9 | — | |
| Longformer-largeSize=large2020.04 | 81.3 | — | — | 88.3 | — | 73.2 | — | |
| Longformer2019.11 | 81.25 | 68 | 63.09 | 88.34 | 45.91 | 73.16 | — | |
| C2F Reader2019.11 | 81.24 | 67.98 | 60.81 | 87.63 | 44.67 | 72.73 | — | |
| Quark (dev)Evaluation Split=dev2020.04 | 81.2 | — | — | 87 | — | 72.3 | — | |
| C2F Reader2020.04 | 81.2 | — | — | 87.6 | — | 72.8 | — | |
| ETC-large+Size=large2020.04 | 81.2 | — | — | 89.1 | — | 73.6 | — | |
| ETCScale=large2019.11 | 81.18 | 68.12 | 63.25 | 89.09 | 46.4 | 73.62 | — | |
| TAP 2 (ensemble)Ensemble=true2020.04 | 79.8 | — | — | 86.7 | — | 70.7 | — | |
| SAEScale=large2019.11 | 79.62 | 66.92 | 61.53 | 86.86 | 45.36 | 71.45 | — | |
| SAE2020.04 | 79.6 | — | — | 86.7 | — | 71.4 | — | |
| EPS+BERT2019.11 | 79.05 | 65.79 | 58.5 | 86.26 | 42.47 | 70.48 | — | |
| TAP22019.11 | 78.59 | 64.99 | 55.47 | 85.57 | 39.77 | 69.12 | — | |
| P-BERT2019.11 | 74.16 | 61.18 | 51.38 | 82.76 | 35.42 | 63.79 | — | |
| ChainEx2019.11 | 74.11 | 61.2 | — | — | — | — | — | |
| LQR-Net2019.11 | 73.78 | 60.2 | 56.21 | 84.09 | 36.56 | 63.68 | — | |
| BERT Plusbackbone=BERT2019.05 | 69.9 | 56 | 42.3 | 80.6 | 26.9 | 58.1 | — | |
| DFGNEntity Recognition Model=BERT NER2019.05 | 69.69 | 56.31 | 51.5 | 81.62 | 33.62 | 59.82 | — | |
| DFGN2019.11 | 69.69 | 56.31 | 51.5 | 81.62 | 33.62 | 59.82 | — | |
| DecompRC2019.11 | 69.63 | 55.2 | — | — | — | — | — | |
| DFGN + BERTbackbone=BERT2019.05 | 68.5 | 55.2 | 49.9 | 81.1 | 31.9 | 58.2 | — | |
| DFGNEntity Recognition Model=Stanford CoreNLP NER2019.05 | 68.49 | 55.17 | 49.85 | 81.06 | 31.87 | 58.23 | — | |
| QFE2019.05 | 68.1 | 53.9 | 57.8 | 84.5 | 34.6 | 59.6 | — | |
| QFE*Status=unpublished2019.05 | 68.06 | 53.86 | 57.75 | 84.49 | 34.63 | 59.61 | — | |
| QFE2019.11 | 68.06 | 53.86 | 57.75 | 84.49 | 34.63 | 59.61 | — | |
| GRN*Status=unpublished2019.05 | 66.71 | 52.92 | 52.37 | 84.11 | 31.77 | 58.47 | — | |
| GRN2019.05 | 66.7 | 52.9 | 52.4 | 84.1 | 31.8 | 58.5 | — | |
| KGNNSetting=distractor2019.11 | 65.75 | 50.81 | 38.74 | 76.79 | 22.4 | 52.82 | — | |
| Baseline ModelSource=Yang et al. (2018)2019.05 | 59.02 | 45.6 | 20.32 | 64.49 | 10.83 | 40.16 | — | |
| Yang et al. (2018)Setting=distractor2019.11 | 59.02 | 45.6 | 20.32 | 64.49 | 10.83 | 40.16 | — | |
| Baseline Model2019.11 | 59.02 | 45.6 | 20.32 | 64.49 | 10.83 | 40.16 | — | |
| Baseline2019.05 | 59 | 45.6 | 20.3 | 64.5 | 10.8 | 40.2 | — | |
| AGARBackbone=InternVL3.5-8B2026.06 | — | — | — | — | — | — | 32.8 | |
| AGLABackbone=InternVL3.5-8B2026.06 | — | — | — | — | — | — | 30.2 | |
| CGRBackbone=InternVL3.5-8B2026.06 | — | — | — | — | — | — | 19.8 | |
| Plain VQABackbone=InternVL3.5-8B2026.06 | — | — | — | — | — | — | 31.5 | |
| RandomBackbone=InternVL3.5-8B2026.06 | — | — | — | — | — | — | 31.6 | |
| UniformBackbone=InternVL3.5-8B2026.06 | — | — | — | — | — | — | 31.9 | |
| VARBackbone=InternVL3.5-8B2026.06 | — | — | — | — | — | — | 25.9 | |
| VEA-BrBackbone=InternVL3.5-8B2026.06 | — | — | — | — | — | — | 29.4 | |
| VEA-ConBackbone=InternVL3.5-8B2026.06 | — | — | — | — | — | — | 29.6 |