Pronoun Disambiguation on Winograd Schema Challenge
90.1AccuracyRoBERTa-WG
Evaluation results
| Method | Links | |
|---|---|---|
| RoBERTa-WGSupervision Type=Supervised, Finetuning Dataset=WinoGrande, Zero-shot Transfer=True2021.05 | 90.1 | |
| CorefRoBERTa LARGESupervision Type=Supervised, Zero-shot Transfer=True2021.05 | 83.2 | |
| RoBERTa-DPRSupervision Type=Supervised, Finetuning Dataset=DPR, Zero-shot Transfer=True2021.05 | 83.1 | |
| MNPPSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 79.5 | |
| AMS-CSQA-DPRSupervision Type=Supervised, Finetuning Dataset=CSQA, DPR, Zero-shot Transfer=True2021.05 | 75.5 | |
| UDSSM-MASKEDWIKI-DPRSupervision Type=Supervised, Finetuning Dataset=MASKEDWIKI, DPR, Zero-shot Transfer=True2021.05 | 75.1 | |
| BERT-MASKEDWIKI-DPRSupervision Type=Supervised, Finetuning Dataset=MASKEDWIKI, DPR, Zero-shot Transfer=True2021.05 | 72.5 | |
| BERT-WIKICREM-DPRSupervision Type=Supervised, Finetuning Dataset=WIKICREM, DPR, Zero-shot Transfer=True2021.05 | 71.8 | |
| CorefBERT LARGESupervision Type=Supervised, Zero-shot Transfer=True2021.05 | 71.4 | |
| BERT_NSP-DPRSupervision Type=Supervised, Finetuning Dataset=DPR, Zero-shot Transfer=True2021.05 | 71.1 | |
| WSC Know. HuntingSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 71.1 | |
| GPT-2Supervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 70.7 | |
| CSSSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 69.6 | |
| Ensemble LMsSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 63.7 | |
| BERT-WIKICREAMSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 63.4 | |
| UDSSMSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 62.4 | |
| BERT-MASKEDWIKISupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 61.9 | |
| Maximum Attention Score (MAS)model=BERT2019.05 | 60.3 | |
| Modified Skip-GramSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 60.3 | |
| BERT Inner AttentionSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 60.3 | |
| Know. Hunter2019.05 | 57.1 | |
| Bi-LSTM-DPRSupervision Type=Supervised, Finetuning Dataset=DPR, Zero-shot Transfer=True2021.05 | 56 | |
| Single LM2019.05 | 54.5 | |
| Transformer2019.05 | 54.1 | |
| USSM + Supervised DeepNet + KB2019.05 | 52.8 | |
| USSM + KB2019.05 | 52 | |
| Random guess2019.05 | 50 |