Semantic Role Labeling on CoNLL 2005 (WSJ)
95.5F1 ScoreDEEPSTRUCT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DEEPSTRUCTmode=multi-task2022.05 | 95.5 | — | — | — | |
| DEEPSTRUCTtraining protocol=multi-task2022.05 | 95.5 | — | — | — | |
| DEEPSTRUCTmode=w/ finetune2022.05 | 95.2 | — | — | — | |
| DEEPSTRUCTtraining protocol=w/ finetune2022.05 | 95.2 | — | — | — | |
| OursSyntax Category=syntax-agnostic, Backbone=RoBERTa2021.09 | 90 | 90.4 | 89.7 | — | |
| Zhang et al.Syntax Category=syntax-agnostic, Backbone=RoBERTa2021.09 | 89.6 | 89.6 | 89.7 | — | |
| TANLTraining Setting=multi-dataset2021.01 | 89.4 | — | — | — | |
| TANLTraining Setting=standard2021.01 | 89.3 | — | — | — | |
| OursSyntax Category=syntax-agnostic, Backbone=BERT2021.09 | 89.3 | 89.7 | 89 | — | |
| TANL2022.05 | 89.3 | — | — | — | |
| TANL2022.05 | 89.3 | — | — | — | |
| TANLTraining Setting=multi-task2021.01 | 89.1 | — | — | — | |
| TANLtraining protocol=multitask2022.05 | 89.1 | — | — | — | |
| Zhou et al.Syntax Category=syntax-aware, Backbone=BERT2021.09 | 88.9 | 89 | 88.8 | — | |
| Mohammadshahi and HendersonSyntax Category=syntax-aware, Backbone=BERT2021.09 | 88.9 | 89.1 | 88.7 | — | |
| BERT SRLTraining Setting=standard2021.01 | 88.8 | — | — | — | |
| Shi and LinSyntax Category=syntax-agnostic, Backbone=BERT2021.09 | 88.8 | 88.6 | 89 | — | |
| Shi and Lin, 20192022.05 | 88.8 | — | — | — | |
| BERT SRL2022.05 | 88.8 | — | — | — | |
| Xia et al.Syntax Category=syntax-aware, Backbone=RoBERTa2021.09 | 88.6 | 88.4 | 88.8 | — | |
| ELMO-SPANModel Type=Ensemble2018.10 | 88.5 | — | — | — | |
| Marcheggiani and TitovSyntax Category=syntax-aware, Backbone=RoBERTa2021.09 | 88 | 87.7 | 88.1 | — | |
| Jindal et al.Syntax Category=syntax-agnostic, Backbone=BERT2021.09 | 87.9 | 88.7 | 88 | — | |
| Li et al.Syntax Category=syntax-agnostic2021.09 | 87.7 | 87.9 | 87.5 | — | |
| ELMO-SPANModel Type=Single2018.10 | 87.6 | — | — | — | |
| He+ 18Model Type=Single2018.10 | 87.4 | — | — | — | |
| Dep and SpanTraining Setting=standard2021.01 | 86.3 | — | — | — | |
| Dep and Span2022.05 | 86.3 | — | — | — | |
| Tan+ 18Model Type=Ensemble2018.10 | 86.1 | — | — | — | |
| Tan+ 18Model Type=Single2018.10 | 84.8 | — | — | — | |
| He+ 17Model Type=Ensemble2018.10 | 84.6 | — | — | — | |
| Strubell+ 18Model Type=Single2018.10 | 83.9 | — | — | — | |
| He+ 17Model Type=Single2018.10 | 83.1 | — | — | — | |
| Zhou+ 15Model Type=Single2018.10 | 82.8 | — | — | — | |
| FitzGerald+ 15Model Type=Ensemble2018.10 | 80.3 | — | — | — | |
| Toutanova+ 08Model Type=Ensemble2018.10 | 80.3 | — | — | — | |
| Täckström+ 15Model Type=Single2018.10 | 79.9 | — | — | — | |
| Toutanova+ 08Model Type=Single2018.10 | 79.7 | — | — | — | |
| FitzGerald+ 15Model Type=Single2018.10 | 79.4 | — | — | — | |
| Punyakanok+ 08Model Type=Single2018.10 | 79.4 | — | — | — | |
| Punyakanok+ 08Model Type=Ensemble2018.10 | 79.4 | — | — | — | |
| Full Fine-tuningBase Model=RoBERTa-large2023.03 | — | — | — | 90.2 | |
| P-tuningBase Model=RoBERTa-large2023.03 | — | — | — | 76.8 | |
| P-tuning V2Base Model=RoBERTa-large2023.03 | — | — | — | 89.2 | |
| P-tuning V2 (reproduced)Base Model=RoBERTa-large, reproduced=true2023.03 | — | — | — | 89.4 | |
| Zero-init.Base Model=RoBERTa-large2023.03 | — | — | — | 89.6 |