Relation Extraction on SCIERC (test)
90.3F1 ScoreRELA
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| RELASemantic Information=true, Correlation Information=true2022.12 | 90.3 | — | — | — | — | — | — | — | |
| IRE-RoBERTaSemantic Information=false, Correlation Information=false2022.12 | 88.9 | — | — | — | — | — | — | — | |
| IRE-BERTSemantic Information=false, Correlation Information=false2022.12 | 88.8 | — | — | — | — | — | — | — | |
| BART-RESemantic Information=true, Correlation Information=true2022.12 | 88.8 | — | — | — | — | — | — | — | |
| BART-DCSemantic Information=true, Correlation Information=false2022.12 | 88.6 | — | — | — | — | — | — | — | |
| BART-DSSemantic Information=true, Correlation Information=false2022.12 | 88.3 | — | — | — | — | — | — | — | |
| BART-DBSemantic Information=false, Correlation Information=true2022.12 | 88.1 | — | — | — | — | — | — | — | |
| LUKESemantic Information=false, Correlation Information=false2022.12 | 87.7 | — | — | — | — | — | — | — | |
| MTBSemantic Information=false, Correlation Information=false2022.12 | 87.4 | — | — | — | — | — | — | — | |
| REBELSemantic Information=true, Correlation Information=false2022.12 | 86.3 | — | — | — | — | — | — | — | |
| TriMFEvaluation protocol=boundary, Backbone=SciBERT2021.01 | 52.44 | 52.63 | 52.32 | — | — | — | — | — | |
| SpERTEvaluation protocol=boundary, Backbone=SciBERT2021.01 | 50.84 | 53.4 | 48.54 | — | — | — | — | — | |
| DYGIE++Backbone=BERT-Base2019.09 | 48.4 | — | — | — | — | — | — | — | |
| DYGIE++Evaluation protocol=boundary2021.01 | 48.4 | — | — | — | — | — | — | — | |
| DYGIE2019.04 | 41.6 | — | — | — | — | — | — | — | |
| DYGIEBackbone=ELMo2019.09 | 41.6 | — | — | — | — | — | — | — | |
| DyGIEEvaluation protocol=boundary2021.01 | 41.6 | — | — | — | — | — | — | — | |
| SCIIElearning=Multi-task2018.08 | 39.3 | 47.6 | 33.5 | — | — | — | — | — | |
| Luan et al.2019.04 | 39.3 | — | — | — | — | — | — | — | |
| SciIEEvaluation protocol=boundary2021.01 | 39.3 | 47.6 | 33.5 | — | — | — | — | — | |
| E2E Rel+ELMOFeatures=ELMO2018.08 | 36.6 | 38.4 | 34.9 | — | — | — | — | — | |
| E2E Rel(Pipeline)setting=Pipeline2018.08 | 35.9 | 37.8 | 34.2 | — | — | — | — | — | |
| E2E Relsetting=End-to-end2018.08 | 34.1 | 37.1 | 32.2 | — | — | — | — | — | |
| BERT-BaseEvaluation Protocol=Frozen2019.03 | — | — | — | 72.74 | — | — | — | — | |
| BERT-BaseEvaluation Protocol=Finetune2019.03 | — | — | — | 78.71 | — | — | — | — | |
| DYGIEEncoder=ELMo, Representation Type=T-Concat2021.09 | — | — | — | — | — | 41.6 | — | — | |
| Luan et al. (2018)Encoder=LSTM + ELMo, Additional data usage=coreference2020.10 | — | — | — | — | 64.2 | 39.3 | — | — | |
| Luan et al. (2019)Encoder=LSTM + ELMo, Cross-sentence information usage=true, Additional data usage=coreference2020.10 | — | — | — | — | 65.2 | 41.6 | — | — | |
| PFNPre-trained embedding=SciBERT, Match setting=exact match2021.08 | — | — | — | — | — | — | — | 38.4 | |
| PL-MarkerEncoder=SciBERT, Representation Type=Solid & Levitated Marker, Cross-sentence information=true2021.09 | — | — | — | — | — | 53.2 | 41.6 | — | |
| PUREContext Setting=single-sentence, Encoder=SciBERT2020.10 | — | — | — | — | 66.6 | 48.2 | 35.6 | — | |
| PUREContext Setting=cross-sentence, Encoder=SciBERT2020.10 | — | — | — | — | 68.9 | 50.1 | 36.8 | — | |
| PUREPre-trained embedding=SciBERT, Match setting=exact match, Setting=single-sentence2021.08 | — | — | — | — | — | — | — | 35.6 | |
| PURE-AEncoder=SciBERT, Representation Type=Levitated Marker2021.09 | — | — | — | — | — | 48.1 | — | — | |
| PURE-FEncoder=SciBERT, Representation Type=Solid Marker2021.09 | — | — | — | — | — | 50.1 | 36.8 | — | |
| SCIBERTEvaluation Protocol=Frozen2019.03 | — | — | — | 75.25 | — | — | — | — | |
| SCIBERTEvaluation Protocol=Finetune2019.03 | — | — | — | 79.97 | — | — | — | — | |
| SPEPre-trained embedding=SciBERT, Match setting=exact match2021.08 | — | — | — | — | — | — | — | 34.6 | |
| TriMFEncoder=SciBERT, Representation Type=T-Concat2021.09 | — | — | — | — | — | 52.4 | — | — | |
| UniREEncoder=SciBERT, Representation Type=T-Concat2021.09 | — | — | — | — | — | — | 36.9 | — | |
| Wadden et al. (DYGIE++)Encoder=SciBERT, Additional data usage=coreference2020.10 | — | — | — | — | 67.5 | 48.4 | — | — |