Pronoun Disambiguation on WSC (test)
78.8Accuracy (Single)ALBERT-xxlarge
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| ALBERT-xxlargeSetup=Zero-shot2021.04 | 78.8 | — | — | — | — | — | — | 57.61 | |
| ALBERT-xxlargeSetup=Zero-shot2021.04 | 77.4 | — | — | — | — | — | — | 54.79 | |
| RoBERTa-largeSetup=Zero-shot2021.04 | 73.91 | — | — | — | — | — | — | 47.83 | |
| RoBERTa-largeSetup=Zero-shot2021.04 | 71.23 | — | — | — | — | — | — | 42.47 | |
| RoBERTa-baseSetup=Zero-shot2021.04 | 63.04 | — | — | — | — | — | — | 27.17 | |
| BERT-largeSetup=Zero-shot2021.04 | 61.41 | — | — | — | — | — | — | 23.91 | |
| BERT-largeSetup=Zero-shot2021.04 | 60.27 | — | — | — | — | — | — | 21.92 | |
| RoBERTa-baseSetup=Zero-shot2021.04 | 60.27 | — | — | — | — | — | — | 21.92 | |
| BERT-baseSetup=Zero-shot2021.04 | 56.52 | — | — | — | — | — | — | 15.22 | |
| ALBERT-baseSetup=Zero-shot2021.04 | 55.48 | — | — | — | — | — | — | 12.33 | |
| ALBERT-baseSetup=Zero-shot2021.04 | 55.43 | — | — | — | — | — | — | 13.04 | |
| BERT-baseSetup=Zero-shot2021.04 | 54.79 | — | — | — | — | — | — | 12.33 | |
| randomSetup=Zero-shot2021.04 | 50 | — | — | — | — | — | — | 25 | |
| randomSetup=Zero-shot2021.04 | 50 | — | — | — | — | — | — | 25 | |
| BERT_CS_largeFormat=MCQA2019.08 | — | 75.5 | 73.7 | 86.5 | 74.8 | 73.3 | 86.3 | — | |
| BERT_largeTraining Objective=MTP (Masked Token Prediction)2019.08 | — | 70.3 | 70.8 | 67.6 | 73.3 | 70.1 | 59.5 | — | |
| BERT_largeFormat=MCQA (Multi-choice Question-Answering)2019.08 | — | 71.4 | 69.9 | 81.1 | 71.8 | 64.9 | 82.4 | — | |
| Ensemble 14 LMs2019.08 | — | 63.7 | 60.6 | 83.8 | 63.4 | 53.4 | 44.3 | — | |
| GEEPBackbone=RoBERTa-base, Data=Gender-neutral dataset2021.10 | — | 52.7 | — | — | — | — | — | — | |
| Knowledge Hunter2019.08 | — | 57.1 | 58.3 | 50 | 58.8 | 58.8 | 90.1 | — | |
| Kocijan et al. (2019)Training Objective=MTP2019.08 | — | 72.2 | 71.6 | 75.7 | 74.8 | 72.5 | 61.1 | — | |
| RoBERTaBackbone=RoBERTa-base, Training=Pre-trained without further training2021.10 | — | 50.1 | — | — | — | — | — | — | |
| Ruan et al. (2019)2019.08 | — | 71.1 | 69.5 | 81.1 | 74.1 | 72.5 | 66.4 | — | |
| SPPABackbone=RoBERTa-base, Data=Gender-neutral dataset, Training=Full fine-tuning2021.10 | — | 50.9 | — | — | — | — | — | — |