Machine Reading Comprehension on Molweni (test)
64.3EMHuman performance
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Human performance2021.04 | 64.3 | 80.2 | |
| Human performance2021.10 | 64.3 | 80.2 | |
| Human performance2022.10 | 64.3 | 80.2 | |
| QuISGBackbone=ELECTRA2022.10 | 59.32 | 72.86 | |
| Enhanced Speaker-aware ModelBackbone=ELECTRA2021.09 | 58.6 | 72.2 | |
| ELECTRA_ourBackbone=ELECTRA2021.09 | 58 | 72.9 | |
| SelfSuperBackbone=ELECTRA2022.10 | 58 | 72.9 | |
| Our BaselinesBackbone=ELECTRA2021.09 | 57.9 | 71 | |
| Speaker EmbeddingBackbone=ELECTRA2021.09 | 57.9 | 71.1 | |
| MDFNBackbone=ELECTRA2021.09 | 57.9 | 71.1 | |
| ELECTRA (Reimpl.)Backbone=ELECTRA2022.10 | 57.85 | 72.17 | |
| Public BaselinesBackbone=ELECTRA2021.09 | 57.3 | 70.4 | |
| ELECTRA_baselineBackbone=ELECTRA2021.09 | 56.8 | 70.6 | |
| Speaker EmbeddingBackbone=BERTwwm2021.09 | 56 | 68.3 | |
| Enhanced Speaker-aware ModelBackbone=BERTwwm2021.09 | 56 | 69.1 | |
| MDFNBackbone=BERTwwm2021.09 | 55.8 | 68.7 | |
| Multi-taskBackbone=BERT-wwm2021.10 | 54.9 | 68.4 | |
| Li et al. (2020)Backbone=BERT-wwm2021.10 | 54.7 | 67.7 | |
| Public BaselinesBackbone=BERTwwm2021.09 | 54.7 | 67.6 | |
| Our BaselinesBackbone=BERTwwm2021.09 | 53.9 | 67.5 | |
| QA-onlyBackbone=BERT-wwm2021.10 | 53.8 | 67.5 | |
| Enhanced Speaker-aware ModelBackbone=BERTlarge2021.09 | 52.9 | 66.9 | |
| Speaker EmbeddingBackbone=BERTlarge2021.09 | 52.4 | 65.7 | |
| Our BaselinesBackbone=BERTlarge2021.09 | 52 | 65.6 | |
| Li et al. (2020)Backbone=BERT-large2021.10 | 51.8 | 65.5 | |
| Public BaselinesBackbone=BERTlarge2021.09 | 51.8 | 65.5 | |
| MDFNBackbone=BERTlarge2021.09 | 51.7 | 65.6 | |
| Multi-taskBackbone=BERT-large2021.10 | 50.6 | 64.9 | |
| Enhanced Speaker-aware ModelBackbone=BERTbase2021.09 | 49.7 | 64.4 | |
| QA-onlyBackbone=BERT-large2021.10 | 49.6 | 64 | |
| BERT_ourBackbone=BERT2021.09 | 49.2 | 64 | |
| SelfSuperBERTBackbone=BERT2022.10 | 49.2 | 64 | |
| MDFNBackbone=BERTbase2021.09 | 48.4 | 62.4 | |
| Speaker EmbeddingBackbone=BERTbase2021.09 | 47.9 | 61.5 | |
| Multi-taskBackbone=BERT-base2021.10 | 47.1 | 61.3 | |
| DADgraph2021.04 | 46.5 | 61.5 | |
| BERT_DADGraphBackbone=BERT2021.09 | 46.5 | 61.5 | |
| DADGraphBackbone=BERT2022.10 | 46.5 | 61.5 | |
| QA-onlyBackbone=BERT-base2021.10 | 46.2 | 59.2 | |
| BERT_our baselineBackbone=BERT2021.09 | 45.8 | 60.2 | |
| DialogueGCN2021.04 | 45.7 | 61 | |
| Our BaselinesBackbone=BERTbase2021.09 | 45.7 | 58.8 | |
| DialogueRNN2021.04 | 45.4 | 60.9 | |
| BERT2021.04 | 45.3 | 58 | |
| BERT_public baselineBackbone=BERT2021.09 | 45.3 | 58 | |
| Li et al. (2020)Backbone=BERT-base2021.10 | 45.3 | 58 | |
| Public BaselinesBackbone=BERTbase2021.09 | 45.3 | 58 | |
| DocQA2021.04 | 42.5 | 56 | |
| BiDAF2021.04 | 22.9 | 39.8 |