Opinion Role Labeling on MPQA Target 2.0 (test)
62.34O-R F1 (hd)SyPtrTrans
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| SyPtrTransBERT=Yes2021.10 | 62.34 | — | — | — | — | 52.02 | 63.22 | 55.23 | |
| PtrTransBERT=Yes2021.10 | 61.65 | — | — | — | — | 50.42 | 62.25 | 53.66 | |
| SPANOMBERT=Yes2021.10 | 61.2 | — | — | — | — | 49.88 | 62.04 | 53.27 | |
| SyPtrTransPre-trained Embeddings=w/ BERT2021.10 | 59.48 | — | — | 65.28 | 51.62 | 44.04 | — | — | |
| SPANOM+SyntPre-trained Embeddings=w/ BERT, Syntax Integration=true2021.10 | 58.46 | — | — | — | 50.46 | 41.82 | — | — | |
| PtrTransPre-trained Embeddings=w/ BERT2021.10 | 58.28 | — | — | 63.9 | 50.11 | 41.96 | — | — | |
| SPANOMPre-trained Embeddings=w/ BERT2021.10 | 58.24 | — | — | 63.71 | 49.89 | 41.1 | — | — | |
| SyPtrTransBERT=No2021.10 | 57.12 | — | — | — | — | 43.63 | 58.29 | 47.68 | |
| PtrTransBERT=No2021.10 | 56.88 | — | — | — | — | 42.82 | 57.91 | 46.96 | |
| SPANOMBERT=No2021.10 | 55.62 | — | — | — | — | 41.65 | 56.47 | 45.09 | |
| SyPtrTransPre-trained Embeddings=w/o BERT2021.10 | 54.67 | — | — | 59.87 | 44.41 | 35.03 | — | — | |
| PtrTransPre-trained Embeddings=w/o BERT2021.10 | 53.19 | — | — | 58.14 | 43.66 | 33.23 | — | — | |
| SPANOMPre-trained Embeddings=w/o BERT2021.10 | 52.9 | — | — | 58.06 | 43.12 | 32.42 | — | — | |
| TransitionPre-trained Embeddings=w/o BERT2021.10 | 47.02 | — | — | 53.04 | — | 31.45 | — | — | |
| BiLSTM+CRFPre-trained Embeddings=w/o BERT2021.10 | 46.72 | — | — | 52.65 | — | 30.07 | — | — | |
| BERT+CRFBERT=Yes2021.10 | 46.62 | — | — | — | — | 34.29 | 55.52 | 50.39 | |
| BiLSTM+CRFBERT=No2021.10 | — | — | — | — | — | — | 58.22 | 54.98 | |
| Previous SOTA2020.05 | — | 81.59 | 73.16 | — | — | — | — | — | |
| RoBERTaModel Size=base2020.05 | — | 80.23 | 72.19 | — | — | — | — | — | |
| RoBERTaModel Size=large2020.05 | — | 81.74 | 75.87 | — | — | — | — | — | |
| RoBERTa + SKEPModel Size=base2020.05 | — | 82.77 | 74.82 | — | — | — | — | — | |
| RoBERTa + SKEPModel Size=large2020.05 | — | 83.59 | 77.41 | — | — | — | — | — | |
| TransitionBERT=No2021.10 | — | — | — | — | — | — | 60.93 | 56.44 |