# Semantic Role Labeling on CoNLL WSJ 2005 (dev) Source: [Semantic Role Labeling on CoNLL WSJ 2005 (dev)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:42:58.982Z Evaluation of Semantic Role Labeling performance on the development set of the CoNLL WSJ 2005 dataset, typically measured using F1 score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Semantic Role Labeling | | Dataset (including any stated split) | CoNLL WSJ 2005 (dev) | | Metric | F1 Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 19 of 19 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [F1 Score](): Higher is better; unit: PERCENT. - [Precision](): Higher is better; unit: PERCENT. - [Recall](): Higher is better; unit: PERCENT. ## Recorded results | Method | F1 Score | Result date | Report paper | | --- | --- | --- | --- | | LISA + Gold | 89.25 | Not recorded | 68fa65151e209131cea081fa | | Span-based SRL | 87.4 | Not recorded | 68fa64be1e209131cea06361 | | Tan et al. (2018) | 86.1 | Not recorded | 68fa64be1e209131cea06361 | | Span-based SRL (Ours) | 85.3 | Not recorded | 68fa64be1e209131cea06361 | | LISA + D&M | 85.27 | Not recorded | 68fa65151e209131cea081fa | | Tan et al. (2018) | 84.8 | Not recorded | 68fa64be1e209131cea06361 | | He et al. (2017) | 84.6 | Not recorded | 68fa64be1e209131cea06361 | | Span-based SRL | 83.9 | Not recorded | 68fa64be1e209131cea06361 | | He et al. (2018) | 83.9 | Not recorded | 68fa65151e209131cea081fa | | LISA | 83.67 | Not recorded | 68fa65151e209131cea081fa | | He et al. (2017) | 83.1 | Not recorded | 68fa64be1e209131cea06361 | | Tan et al. (2018) | 83.1 | Not recorded | 68fa65151e209131cea081fa | | SA | 82.97 | Not recorded | 68fa65151e209131cea081fa | | Zhou and Xu (2015) | 82.8 | Not recorded | 68fa64be1e209131cea06361 | | Yang and Mitchell (2017) | 81.9 | Not recorded | 68fa64be1e209131cea06361 | | Span-based SRL (Ours) | 81.6 | Not recorded | 68fa64be1e209131cea06361 | | He et al. (2017) | 81.5 | Not recorded | 68fa64be1e209131cea06361 | | He et al. (2017) | 80.3 | Not recorded | 68fa64be1e209131cea06361 | | FitzGerald et al. (2015) | 80.3 | Not recorded | 68fa64be1e209131cea06361 | ### LISA + Gold F1 Score: 89.25. Result ID: 698c9b5e0b614d12cff1e61d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"predicates":"gold","syntax":"Gold"}. This record does not assert independent reproduction. ### Span-based SRL F1 Score: 87.4. Result ID: 698c8a440b614d12cff17273. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Gold predicates":true,"ELMo":true}. This record does not assert independent reproduction. ### Tan et al. (2018) F1 Score: 86.1. Result ID: 698c8a440b614d12cff17274. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Gold predicates":true,"Product of Experts (POE)":true}. This record does not assert independent reproduction. ### Span-based SRL (Ours) F1 Score: 85.3. Result ID: 698c8a440b614d12cff17263. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Embeddings":"ELMo","Evaluation Protocol":"End-to-End"}. This record does not assert independent reproduction. ### LISA + D&M F1 Score: 85.27. Result ID: 698c9b5e0b614d12cff1e61c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"predicates":"gold","syntax":"D&M"}. This record does not assert independent reproduction. ### Tan et al. (2018) F1 Score: 84.8. Result ID: 698c8a440b614d12cff17278. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Gold predicates":true}. This record does not assert independent reproduction. ### He et al. (2017) F1 Score: 84.6. Result ID: 698c8a440b614d12cff17275. [Reporting paper]() Recorded evaluation configuration: {"Gold predicates":true,"Product of Experts (POE)":true}. This record does not assert independent reproduction. ### Span-based SRL F1 Score: 83.9. Result ID: 698c8a440b614d12cff17277. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Gold predicates":true,"ELMo":false}. This record does not assert independent reproduction. ### He et al. (2018) F1 Score: 83.9. Result ID: 698c9b5e0b614d12cff1e618. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"predicates":"gold"}. This record does not assert independent reproduction. ### LISA F1 Score: 83.67. Result ID: 698c9b5e0b614d12cff1e61b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"predicates":"gold","syntax":"none"}. This record does not assert independent reproduction. ### He et al. (2017) F1 Score: 83.1. Result ID: 698c8a440b614d12cff17279. [Reporting paper]() Recorded evaluation configuration: {"Gold predicates":true}. This record does not assert independent reproduction. ### Tan et al. (2018) F1 Score: 83.1. Result ID: 698c9b5e0b614d12cff1e619. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"predicates":"gold"}. This record does not assert independent reproduction. ### SA F1 Score: 82.97. Result ID: 698c9b5e0b614d12cff1e61a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"predicates":"gold","syntax":"none"}. This record does not assert independent reproduction. ### Zhou and Xu (2015) F1 Score: 82.8. Result ID: 698c8a440b614d12cff1727b. [Reporting paper]() Recorded evaluation configuration: {"Gold predicates":true}. This record does not assert independent reproduction. ### Yang and Mitchell (2017) F1 Score: 81.9. Result ID: 698c8a440b614d12cff1727a. [Reporting paper]() Recorded evaluation configuration: {"Gold predicates":true}. This record does not assert independent reproduction. ### Span-based SRL (Ours) F1 Score: 81.6. Result ID: 698c8a440b614d12cff17265. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Embeddings":"GloVe","Evaluation Protocol":"End-to-End"}. This record does not assert independent reproduction. ### He et al. (2017) F1 Score: 81.5. Result ID: 698c8a440b614d12cff17264. [Reporting paper]() Recorded evaluation configuration: {"Model Variant":"Product of Experts (POE)","Evaluation Protocol":"End-to-End"}. This record does not assert independent reproduction. ### He et al. (2017) F1 Score: 80.3. Result ID: 698c8a440b614d12cff17266. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"End-to-End"}. This record does not assert independent reproduction. ### FitzGerald et al. (2015) F1 Score: 80.3. Result ID: 698c8a440b614d12cff17276. [Reporting paper]() Recorded evaluation configuration: {"Gold predicates":true,"Product of Experts (POE)":true}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.