Span-based Semantic Role Labeling on CoNLL 2005 (Out-of-domain (Brown))
85.1F1 ScoreOurs
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| OursSyntax Category=syntax-agnostic, Backbone=RoBERTa2021.09 | 85.1 | 86.4 | 83.8 | |
| OursSyntax Category=syntax-agnostic, Backbone=BERT2021.09 | 84.7 | 85.9 | 83.5 | |
| Zhang et al.Syntax Category=syntax-agnostic, Backbone=RoBERTa2021.09 | 83.7 | 83.8 | 83.6 | |
| SynG2G-TrSA=true, Evaluation Setting=given predicate, Pre-training=true2021.04 | 83.21 | 83.93 | 82.5 | |
| Mohammadshahi and HendersonSyntax Category=syntax-aware, Backbone=BERT2021.09 | 83.2 | 83.9 | 82.5 | |
| Xia et al.Syntax Category=syntax-aware, Backbone=RoBERTa2021.09 | 83.2 | 83.1 | 83.3 | |
| BERT-LSTM-largeEnsemble=false2019.04 | 82 | 81.9 | 82.1 | |
| Shi and LinSyntax Category=syntax-agnostic, Backbone=BERT2021.09 | 82 | 81.9 | 82.1 | |
| Paolini et al.Syntax Category=syntax-agnostic, Backbone=T52021.09 | 82 | — | — | |
| Jia et al. (2022)SA=false, Evaluation Setting=given predicate, Pre-training=true2021.04 | 81.9 | — | — | |
| Jindal et al. (2020)SA=false, Evaluation Setting=given predicate, Pre-training=true2021.04 | 81.44 | 81.52 | 81.36 | |
| Zhou et al.Syntax Category=syntax-aware, Backbone=BERT2021.09 | 81.4 | 81.9 | 81 | |
| BERT-LSTM-baseEnsemble=false2019.04 | 80.9 | 80.7 | 81.2 | |
| Marcheggiani and TitovSyntax Category=syntax-aware, Backbone=RoBERTa2021.09 | 80.6 | 80.5 | 80.7 | |
| SynG2G-TrSA=true, Evaluation Setting=end-to-end, Pre-training=true2021.04 | 80.53 | 80.01 | 81.07 | |
| Li et al. (2019)Ensemble=false2019.04 | 80.5 | 80.6 | 80.4 | |
| Li et al.Syntax Category=syntax-agnostic2021.09 | 80.5 | 80.6 | 80.4 | |
| Li et al. (2019)SA=false, Evaluation Setting=given predicate, Pre-training=true2021.04 | 80.5 | 80.6 | 80.4 | |
| He et al. (2018a)Ensemble=false2019.04 | 80.4 | — | — | |
| He et al. (2018a)SA=false, Evaluation Setting=given predicate, Pre-training=true2021.04 | 80.4 | — | — | |
| Jindal et al.Syntax Category=syntax-agnostic, Backbone=BERT2021.09 | 80.2 | 80.3 | 80.1 | |
| Zhang et al. (2021)SA=false, Evaluation Setting=given predicate, Pre-training=true2021.04 | 80.2 | 80.3 | 80.1 | |
| Ouchi et al. (2018)Ensemble=true2019.04 | 79.6 | 81 | 78.4 | |
| Ouchi et al. (2018)Ensemble=false2019.04 | 78.7 | 79.9 | 77.5 | |
| Ouchi et al. (2018)SA=false, Evaluation Setting=given predicate, Pre-training=true2021.04 | 78.7 | 79.9 | 77.5 | |
| Strubell et al. (2018)†SA=true, Evaluation Setting=end-to-end, Pre-training=true2021.04 | 78.25 | 79.02 | 77.49 | |
| SynG2G-Tr (w/o BERT)SA=true, Evaluation Setting=given predicate, Pre-training=false2021.04 | 77.45 | 77.73 | 77.18 | |
| Strubell et al. (2018)†SA=true, Evaluation Setting=given predicate, Pre-training=false2021.04 | 76.54 | 76.65 | 76.44 | |
| Strubell et al. (2018)Ensemble=false2019.04 | 76.5 | 76.7 | 76.4 | |
| Li et al. (2019)SA=false, Evaluation Setting=end-to-end, Pre-training=true2021.04 | 76.4 | 74.7 | 78.1 | |
| He et al. (2018a)SA=false, Evaluation Setting=end-to-end, Pre-training=true2021.04 | 76.1 | 73.9 | 78.4 | |
| Xia et al. (2020)SA=true, Evaluation Setting=given predicate, Pre-training=false2021.04 | 75.86 | 76.3 | 75.42 | |
| SynG2G-Tr (w/o BERT)SA=true, Evaluation Setting=end-to-end, Pre-training=false2021.04 | 75.26 | 73.92 | 76.65 | |
| Strubell et al. (2018)SA=true, Evaluation Setting=end-to-end, Pre-training=false2021.04 | 74.66 | 75.8 | 73.54 | |
| Xia et al. (2020)SA=true, Evaluation Setting=end-to-end, Pre-training=false2021.04 | 74.19 | 73.47 | 74.92 | |
| Tan et al. (2017)SA=false, Evaluation Setting=given predicate, Pre-training=false2021.04 | 74.1 | 73.5 | 74.6 | |
| He et al. (2018a)SA=false, Evaluation Setting=given predicate, Pre-training=false2021.04 | 73.7 | — | — | |
| He et al. (2017)SA=false, Evaluation Setting=end-to-end, Pre-training=false2021.04 | 73.6 | 74.9 | 72.4 | |
| Ouchi et al. (2018)SA=false, Evaluation Setting=given predicate, Pre-training=false2021.04 | 73.1 | 76 | 70.4 | |
| Xia et al. (2019)SA=true, Evaluation Setting=end-to-end, Pre-training=false2021.04 | 72.9 | 73.7 | 72 | |
| He et al. (2018a)SA=false, Evaluation Setting=end-to-end, Pre-training=false2021.04 | 70.8 | 69.7 | 71.9 |