List your company
ResearchFormerly WizWand

Span-based Semantic Role Labeling on CoNLL 2005 (Out-of-domain (Brown))

85.1F1 ScoreOurs
70.22874.08977.9581.811Apr 10, 2019Sep 5, 2019Jan 31, 2020Jun 27, 2020Nov 22, 2020Apr 19, 2021Sep 14, 2021
Updated 8mo ago

Evaluation results

MethodLinks
OursSyntax Category=syntax-agnostic, Backbone=RoBERTa2021.0985.186.483.8
OursSyntax Category=syntax-agnostic, Backbone=BERT2021.0984.785.983.5
Zhang et al.Syntax Category=syntax-agnostic, Backbone=RoBERTa2021.0983.783.883.6
SynG2G-TrSA=true, Evaluation Setting=given predicate, Pre-training=true2021.0483.2183.9382.5
Mohammadshahi and HendersonSyntax Category=syntax-aware, Backbone=BERT2021.0983.283.982.5
Xia et al.Syntax Category=syntax-aware, Backbone=RoBERTa2021.0983.283.183.3
BERT-LSTM-largeEnsemble=false2019.048281.982.1
Shi and LinSyntax Category=syntax-agnostic, Backbone=BERT2021.098281.982.1
Paolini et al.Syntax Category=syntax-agnostic, Backbone=T52021.0982——
Jia et al. (2022)SA=false, Evaluation Setting=given predicate, Pre-training=true2021.0481.9——
Jindal et al. (2020)SA=false, Evaluation Setting=given predicate, Pre-training=true2021.0481.4481.5281.36
Zhou et al.Syntax Category=syntax-aware, Backbone=BERT2021.0981.481.981
BERT-LSTM-baseEnsemble=false2019.0480.980.781.2
Marcheggiani and TitovSyntax Category=syntax-aware, Backbone=RoBERTa2021.0980.680.580.7
SynG2G-TrSA=true, Evaluation Setting=end-to-end, Pre-training=true2021.0480.5380.0181.07
Li et al. (2019)Ensemble=false2019.0480.580.680.4
Li et al.Syntax Category=syntax-agnostic2021.0980.580.680.4
Li et al. (2019)SA=false, Evaluation Setting=given predicate, Pre-training=true2021.0480.580.680.4
He et al. (2018a)Ensemble=false2019.0480.4——
He et al. (2018a)SA=false, Evaluation Setting=given predicate, Pre-training=true2021.0480.4——
Jindal et al.Syntax Category=syntax-agnostic, Backbone=BERT2021.0980.280.380.1
Zhang et al. (2021)SA=false, Evaluation Setting=given predicate, Pre-training=true2021.0480.280.380.1
Ouchi et al. (2018)Ensemble=true2019.0479.68178.4
Ouchi et al. (2018)Ensemble=false2019.0478.779.977.5
Ouchi et al. (2018)SA=false, Evaluation Setting=given predicate, Pre-training=true2021.0478.779.977.5
Strubell et al. (2018)†SA=true, Evaluation Setting=end-to-end, Pre-training=true2021.0478.2579.0277.49
SynG2G-Tr (w/o BERT)SA=true, Evaluation Setting=given predicate, Pre-training=false2021.0477.4577.7377.18
Strubell et al. (2018)†SA=true, Evaluation Setting=given predicate, Pre-training=false2021.0476.5476.6576.44
Strubell et al. (2018)Ensemble=false2019.0476.576.776.4
Li et al. (2019)SA=false, Evaluation Setting=end-to-end, Pre-training=true2021.0476.474.778.1
He et al. (2018a)SA=false, Evaluation Setting=end-to-end, Pre-training=true2021.0476.173.978.4
Xia et al. (2020)SA=true, Evaluation Setting=given predicate, Pre-training=false2021.0475.8676.375.42
SynG2G-Tr (w/o BERT)SA=true, Evaluation Setting=end-to-end, Pre-training=false2021.0475.2673.9276.65
Strubell et al. (2018)SA=true, Evaluation Setting=end-to-end, Pre-training=false2021.0474.6675.873.54
Xia et al. (2020)SA=true, Evaluation Setting=end-to-end, Pre-training=false2021.0474.1973.4774.92
Tan et al. (2017)SA=false, Evaluation Setting=given predicate, Pre-training=false2021.0474.173.574.6
He et al. (2018a)SA=false, Evaluation Setting=given predicate, Pre-training=false2021.0473.7——
He et al. (2017)SA=false, Evaluation Setting=end-to-end, Pre-training=false2021.0473.674.972.4
Ouchi et al. (2018)SA=false, Evaluation Setting=given predicate, Pre-training=false2021.0473.17670.4
Xia et al. (2019)SA=true, Evaluation Setting=end-to-end, Pre-training=false2021.0472.973.772
He et al. (2018a)SA=false, Evaluation Setting=end-to-end, Pre-training=false2021.0470.869.771.9