Coreference Resolution on Winograd WSC273 (test)
90.1AccuracyFine-tuned SOTA
Evaluation results
| Method | Links | |
|---|---|---|
| Fine-tuned SOTAlearning_setting=fine-tuned2020.05 | 90.1 | |
| PaLM 540BNumber of shots=02022.10 | 90.1 | |
| GPT-3learning_setting=one-shot2020.05 | 89.7 | |
| GPT-3learning_setting=few-shot2020.05 | 88.6 | |
| GPT-3learning_setting=zero-shot2020.05 | 88.3 | |
| GPT-3 (Davinci)Number of shots=02022.10 | 88.3 | |
| GLM-130BNumber of shots=02022.10 | 84.3 | |
| RoBERTa-LMBackbone size=LARGE2020.04 | 83.2 | |
| CorefRoBERTaBackbone size=LARGE2020.04 | 83.2 | |
| BERT (ASER++ & WscR)Category=External Knowledge Enhanced Methods, Knowledge Source=ASER++ & WscR2021.06 | 74.1 | |
| BERT (ASER & WscR)Category=External Knowledge Enhanced Methods, Knowledge Source=ASER & WscR2021.06 | 72.5 | |
| CorefBERTBackbone size=LARGE2020.04 | 71.4 | |
| BERT (WscR)Category=External Knowledge Enhanced Methods, Knowledge Source=WscR2021.06 | 71.4 | |
| BERT-LMBackbone size=LARGE2020.04 | 70 | |
| WikiCREMBackbone size=LARGE2020.04 | 70 | |
| BERT (ASER++)Category=External Knowledge Enhanced Methods, Knowledge Source=ASER++ (Proposed)2021.06 | 66.2 | |
| BERT (ASER)Category=External Knowledge Enhanced Methods, Knowledge Source=ASER2021.06 | 64.5 | |
| longdocTraining=ON2022.08 | 64.2 | |
| CorefBERTBackbone size=BASE2020.04 | 64.1 | |
| longdocTraining=PC2022.08 | 63.5 | |
| longdocTraining=ON + PS 60K, Speaker features (S)=true, Pseudo-singletons (PS)=60K2022.08 | 63.1 | |
| longdocTraining=ON, Speaker features (S)=true2022.08 | 62.7 | |
| longdocTraining=Joint + PS 60K, Speaker features (S)=true, Pseudo-singletons (PS)=60K2022.08 | 62 | |
| BERT (w/o finetuning)Category=Language Model-based Methods, Finetuning=False2021.06 | 61.9 | |
| LM (Ensemble)Category=Language Model-based Methods, Variant=Ensemble2021.06 | 61.5 | |
| longdocTraining=Joint, Speaker features (S)=true2022.08 | 61.3 | |
| BERT-LMBackbone size=BASE2020.04 | 61.2 | |
| longdocTraining=ON, Speaker features (S)=true, Genre features (G)=true2022.08 | 60.9 | |
| Knowledge HuntingCategory=Pure Knowledge-based Methods2021.06 | 57.3 | |
| String MatchCategory=Pure Knowledge-based Methods2021.06 | 56.6 | |
| LM (Single)Category=Language Model-based Methods, Variant=Single2021.06 | 54.5 | |
| OPT 175BNumber of shots=02022.10 | 52.9 | |
| BLOOM 176BNumber of shots=02022.10 | 49.1 | |
| longdocTraining=LB02022.08 | 31 |