Semantic Role Labeling on CoNLL 2012 (test)
88.59F1 ScoreFei et al. (2021a) ++ROBERTa
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Fei et al. (2021a) ++ROBERTaPredicates=With pre-identified, Backbone=RoBERTa2025.06 | 88.59 | 88.09 | 88.83 | — | |
| Llama3-8B (Fine-tune)Predicates=With pre-identified, Model Type=Fine-tuned LLM2025.06 | 88.12 | 88.19 | 88.06 | — | |
| Zhang et al. (2022) + BERTPredicates=With pre-identified, Backbone=BERT2025.06 | 87.66 | 87.52 | 87.79 | — | |
| Fei et al. (2021a)Predicates=With pre-identified2025.06 | 86.2 | 86.51 | 85.92 | — | |
| Llama3-8B (Fine-tune)Predicates=Without pre-identified, Model Type=Fine-tuned LLM2025.06 | 85.61 | 85.64 | 85.59 | — | |
| Span-based SRLGold predicates=true, ELMo=true2018.05 | 85.5 | — | — | — | |
| Zhang et al. (2022) + BERTPredicates=Without pre-identified, Backbone=BERT2025.06 | 85.45 | 84.53 | 86.41 | — | |
| He et al. + ELMoenhancement=ELMo2018.02 | 84.6 | — | — | — | |
| Peters et al. (2018)Gold predicates=true, ELMo=true2018.05 | 84.6 | — | — | — | |
| DEEPATT (FFN, Ensemble)Nonlinearity=FFN, model_type=Ensemble2017.12 | 83.9 | 83.3 | 84.5 | 69.3 | |
| Tan et al. (2018)Gold predicates=true, Product of Experts (POE)=true2018.05 | 83.9 | — | — | — | |
| Zhang et al. (2022)Predicates=With pre-identified2025.06 | 83.66 | 83.02 | 84.31 | — | |
| He et al. (2017)model_type=Ensemble2017.12 | 83.4 | 83.5 | 83.3 | 68.5 | |
| He et al.evaluation_protocol=ensemble2018.02 | 83.4 | — | — | — | |
| He et al. (2017)Gold predicates=true, Product of Experts (POE)=true2018.05 | 83.4 | — | — | — | |
| LISAEmbeddings=ELMo, External Parse=D&M2018.04 | 83.38 | 84.14 | 82.64 | — | |
| SAEmbeddings=ELMo2018.04 | 83.28 | 84.39 | 82.21 | — | |
| LISAEmbeddings=ELMo2018.04 | 83.12 | 83.97 | 82.29 | — | |
| Span-based SRL (Ours)Embeddings=ELMo, Evaluation Protocol=End-to-End2018.05 | 82.9 | 81.9 | 84 | — | |
| He et al.Embeddings=ELMo2018.04 | 82.9 | 81.9 | 84 | — | |
| ChatGPT+SimCSE KNNPredicates=With pre-identified, Reference=Sun et al. (2023)2025.06 | 82.8 | — | — | — | |
| DEEPATT (FFN)Nonlinearity=FFN, model_type=Single2017.12 | 82.7 | 81.9 | 83.6 | 67.5 | |
| Tan et al. (2018)Gold predicates=true2018.05 | 82.7 | — | — | — | |
| LISAEmbeddings=GloVe, External Parse=D&M2018.04 | 82.33 | 83.3 | 81.38 | — | |
| Span-based SRLGold predicates=true, ELMo=false2018.05 | 82.1 | — | — | — | |
| He et al. (2017)model_type=Single2017.12 | 81.7 | 81.8 | 81.6 | 66 | |
| He et al.evaluation_protocol=single2018.02 | 81.7 | — | — | — | |
| He et al. (2017)Gold predicates=true2018.05 | 81.7 | — | — | — | |
| DEEPATT (RNN)Nonlinearity=RNN, model_type=Single2017.12 | 81.5 | 80.9 | 82.2 | 65.7 | |
| He et al.implementation=reproduced by this paper2018.02 | 81.4 | — | — | — | |
| Zhou and Xu (2015)2017.12 | 81.3 | — | — | — | |
| Zhou and Xu2018.02 | 81.3 | — | — | — | |
| SAEmbeddings=GloVe2018.04 | 81.26 | 82.55 | 80.02 | — | |
| Zhang et al. (2022)Predicates=Without pre-identified2025.06 | 81.21 | 79.27 | 83.24 | — | |
| DEEPATT (CNN)Nonlinearity=CNN, model_type=Single2017.12 | 81.2 | 79.8 | 82.6 | 66.1 | |
| Zhou and Xu (2015)Gold predicates=true2018.05 | 81.1 | — | — | — | |
| LISAEmbeddings=GloVe2018.04 | 80.7 | 81.86 | 79.56 | — | |
| FitzGerald et al. (2015)model_type=Ensemble, variant=Struct.2017.12 | 80.1 | 81.2 | 79 | 62.6 | |
| FitzGerald et al. (2015)Gold predicates=true, Product of Experts (POE)=true2018.05 | 80.1 | — | — | — | |
| Span-based SRL (Ours)Embeddings=GloVe, Evaluation Protocol=End-to-End2018.05 | 79.8 | 79.4 | 80.1 | — | |
| He et al.Embeddings=GloVe2018.04 | 79.8 | 79.4 | 80.1 | — | |
| Täckström et al. (2015)model_type=Ensemble, variant=Struct.2017.12 | 79.4 | 80.6 | 78.2 | 61.8 | |
| He et al. (2017)Model Variant=Product of Experts (POE), Evaluation Protocol=End-to-End2018.05 | 78.4 | 80.2 | 76.6 | — | |
| Pradhan et al. (2013)revision=Revised2017.12 | 77.5 | 78.5 | 76.6 | 55.8 | |
| Pradhan et al.2018.02 | 77.5 | — | — | — | |
| He et al. (2017)Evaluation Protocol=End-to-End2018.05 | 76.8 | 78.6 | 75.1 | — | |
| Llama3-8B (Frozen)Predicates=With pre-identified, Model Type=Frozen LLM2025.06 | 18.91 | 21.91 | 16.64 | — | |
| Llama3-8B (Frozen)Predicates=Without pre-identified, Model Type=Frozen LLM2025.06 | 6.88 | 6.29 | 7.59 | — | |
| Llama2-7B+3-shotPredicates=With pre-identified, Reference=Cheng et al. (2024)2025.06 | 3.13 | 1.83 | 10.67 | — |