Relation Extraction on ACE05 (test)
73F1 ScorePL-Marker
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| PL-MarkerEncoder=ALBERT_XXL, Representation Type=Solid & Levitated Marker2021.09 | 73 | — | — | — | — | — | — | 71.1 | — | — | |
| PUREContext Setting=cross-sentence, Encoder=ALBERT-xxlarge-v12020.10 | 69.4 | — | — | — | — | — | — | 67 | 90.9 | — | |
| PURE-FEncoder=ALBERT_XXL, Representation Type=Solid Marker2021.09 | 69.4 | — | — | — | — | — | — | 67 | — | — | |
| PUREContext Setting=single-sentence, Encoder=ALBERT-xxlarge-v12020.10 | 69 | — | — | — | — | — | — | 65.6 | 89.7 | — | |
| PL-MarkerEncoder=BERT_BASE, Representation Type=Solid & Levitated Marker, Cross-sentence information=true2021.09 | 69 | — | — | — | — | — | — | 66.5 | — | — | |
| HySPASpace Complexity=O(n), Time Complexity=O(n), Backbone=RoBERTa2021.06 | 68.2 | — | — | — | — | — | — | — | — | — | |
| HySPASpace Complexity=O(n), Time Complexity=O(n), Backbone=ALBERT2021.06 | 68 | — | — | — | — | — | — | — | — | — | |
| PUREContext Setting=cross-sentence, Encoder=BERT-base2020.10 | 67.7 | — | — | — | — | — | — | 64.8 | 90.1 | — | |
| PURE-FEncoder=BERT_BASE, Representation Type=Solid Marker2021.09 | 67.7 | — | — | — | — | — | — | 64.8 | — | — | |
| Table-Sequence EncodersAveraging=micro-averaged2020.10 | 67.6 | — | — | — | — | — | — | 64.3 | — | — | |
| Wang and LuEncoder=ALBERT-xxlarge-v12020.10 | 67.6 | — | — | — | — | — | — | 64.3 | 89.5 | — | |
| TabSeqSpace Complexity=O(n²), Time Complexity=O(n), Backbone=ALBERT-xxlarge2021.06 | 67.6 | — | — | — | — | — | — | — | — | — | |
| TableSeqEncoder=ALBERT_XXL, Representation Type=T-Concat2021.09 | 67.6 | — | — | — | — | — | — | 64.3 | — | — | |
| Lin et al.Encoder=BERT-large2020.10 | 67.5 | — | — | — | — | — | — | — | 88.8 | — | |
| OneIESpace Complexity=O(n), Time Complexity=O(n²)2021.06 | 67.5 | — | — | — | — | — | — | — | — | — | |
| OneIEEncoder=BERT_L, Representation Type=T-Concat2021.09 | 67.5 | — | — | — | — | — | — | — | — | — | |
| PUREContext Setting=single-sentence, Encoder=BERT-base2020.10 | 66.7 | — | — | — | — | — | — | 63.9 | 88.7 | — | |
| TriMFEncoder=BERT_BASE, Representation Type=T-Concat2021.09 | 66.5 | — | — | — | — | — | — | 62.8 | — | — | |
| PURE-AEncoder=BERT_BASE, Representation Type=Levitated Marker2021.09 | 66.5 | — | — | — | — | — | — | — | — | — | |
| TriMFEvaluation protocol=boundary2021.01 | 66.49 | 65.87 | 67.12 | — | — | — | — | — | — | — | |
| TablERTEvaluation Criteria=ACE05♦2020.10 | 66.1 | 70.9 | 61.9 | — | — | — | — | — | — | — | |
| SPTree2019.02 | 65.3 | 70.1 | 61.2 | — | — | — | — | — | — | — | |
| Walk-based Modelwalk length (l)=42019.02 | 64.2 | 69.7 | 59.5 | — | — | — | — | — | — | — | |
| Walk-based Modelwalk length (l)=22019.02 | 63.6 | 69.9 | 58.4 | — | — | — | — | — | — | — | |
| DYGIE++Backbone=BERT-Base2019.09 | 63.4 | — | — | — | — | — | — | — | — | — | |
| Wadden et al.Averaging=micro-averaged2020.10 | 63.4 | — | — | — | — | — | — | — | — | — | |
| Wadden et al. (DYGIE++)Encoder=BERT-base, Cross-sentence information usage=true, Additional data usage=coreference2020.10 | 63.4 | — | — | — | — | — | — | — | 88.6 | — | |
| DYGIE++Evaluation protocol=boundary2021.01 | 63.4 | — | — | — | — | — | — | — | — | — | |
| Dygie++Space Complexity=O(n), Time Complexity=O(n²)2021.06 | 63.4 | — | — | — | — | — | — | — | — | — | |
| DyGIE++ (Wadden et al., 2019)Evaluation Criteria=ACE05♦2020.10 | 63.4 | — | — | — | — | — | — | — | — | — | |
| DYGIE++Encoder=BERT_BASE, Representation Type=T-Concat2021.09 | 63.4 | — | — | — | — | — | — | — | — | — | |
| DYGIE2019.04 | 63.2 | — | — | — | — | — | — | — | — | — | |
| DYGIEBackbone=ELMo2019.09 | 63.2 | — | — | — | — | — | — | — | — | — | |
| Luan et al.Averaging=micro-averaged2020.10 | 63.2 | — | — | — | — | — | — | — | — | — | |
| Luan et al. (2019)Encoder=LSTM + ELMo, Cross-sentence information usage=true, Additional data usage=coreference2020.10 | 63.2 | — | — | — | — | — | — | — | 88.4 | — | |
| DyGIEEvaluation protocol=boundary2021.01 | 63.2 | — | — | — | — | — | — | — | — | — | |
| DYGIEEncoder=ELMo, Representation Type=T-Concat2021.09 | 63.2 | — | — | — | — | — | — | — | — | — | |
| Dixit and AlAveraging=micro-averaged2020.10 | 62.8 | — | — | — | — | — | — | — | — | — | |
| SpanRESpace Complexity=O(n), Time Complexity=O(n²)2021.06 | 62.8 | — | — | — | — | — | — | — | — | — | |
| Dixit and Al-Onaizan (2019)Evaluation Criteria=ACE05♦2020.10 | 62.8 | 68 | 58.4 | — | — | — | — | — | — | — | |
| TriMFEvaluation protocol=strict2021.01 | 62.77 | 62.19 | 63.37 | — | — | — | — | — | — | — | |
| Sanh et al.2019.04 | 62.7 | — | — | — | — | — | — | — | — | — | |
| Walk-based Modelwalk length (l)=1, walks=No walks2019.02 | 62.7 | 71.9 | 55.6 | — | — | — | — | — | — | — | |
| Walk-based Modelwalk length (l)=82019.02 | 62.4 | 71.5 | 55.3 | — | — | — | — | — | — | — | |
| TablERTEvaluation Criteria=ACE05♠2020.10 | 62.4 | 67 | 58.5 | — | — | — | — | — | — | — | |
| MRC4ERE++Evaluation protocol=strict2021.01 | 62.1 | 62 | 62.2 | — | — | — | — | — | — | — | |
| Baselinemode=entity-only representations2019.02 | 61.4 | 72.5 | 53.3 | — | — | — | — | — | — | — | |
| Multi-turn QA2019.05 | 60.2 | 64.8 | 56.2 | — | — | — | — | — | — | — | |
| Multi-turn QAEvaluation protocol=strict2021.01 | 60.2 | 64.8 | 56.2 | — | — | — | — | — | — | — | |
| Multi-turn QA (Li et al., 2019)Evaluation Criteria=ACE05♠2020.10 | 60.2 | 64.8 | 56.2 | — | — | — | — | — | — | — | |
| Multi-turn QAEncoder=BERT_L2021.09 | 60.2 | — | — | — | — | — | — | — | — | — | |
| Sun et al.2019.05 | 59.6 | 64.9 | 55.1 | — | — | — | — | — | — | — | |
| MRT (Sun et al., 2018)Evaluation Criteria=ACE05♠2020.10 | 59.6 | 64.9 | 55.1 | — | — | — | — | — | — | — | |
| Zhang et al.2019.04 | 57.5 | — | — | — | — | — | — | — | — | — | |
| Zhang et al.2019.05 | 57.5 | — | — | — | — | — | — | — | — | — | |
| Zhang et al. (2017)Evaluation Criteria=ACE05♠2020.10 | 57.5 | — | — | — | — | — | — | — | — | — | |
| Katiyar and CardieAveraging=micro-averaged2020.10 | 55.9 | — | — | — | — | — | — | 53.6 | — | — | |
| Katiyar and CardieEncoder=LSTM2020.10 | 55.9 | — | — | — | — | — | — | 53.6 | 82.6 | — | |
| PointerNetSpace Complexity=O(n), Time Complexity=O(n²)2021.06 | 55.9 | — | — | — | — | — | — | — | — | — | |
| SPTree2016.01 | 55.6 | 57.2 | 54 | — | — | — | — | — | — | — | |
| Miwa and Bansal2019.04 | 55.6 | — | — | — | — | — | — | — | — | — | |
| Miwa and Bansal2019.05 | 55.6 | 57.2 | 54 | — | — | — | — | — | — | — | |
| SPTree (Miwa and Bansal, 2016)Evaluation Criteria=ACE05♠2020.10 | 55.6 | 57.2 | 54 | — | — | — | — | — | — | — | |
| SPtreeEncoder=LSTM, Representation Type=T-Concat2021.09 | 55.6 | — | — | — | — | — | — | — | — | — | |
| Katiyar and Cardie2019.05 | 53.6 | 55.5 | 51.8 | — | — | — | — | — | — | — | |
| Li and JiAveraging=micro-averaged2020.10 | 52.1 | — | — | — | — | — | — | 49.5 | — | — | |
| Li and JiEncoder=LSTM2020.10 | 52.1 | — | — | — | — | — | — | 49.5 | 80.8 | — | |
| Li and Ji (2014)Evaluation Criteria=ACE05♦2020.10 | 52.1 | 68.9 | 41.9 | — | — | — | — | — | — | — | |
| Li and JiEncoder=BERT_L2021.09 | 52.1 | — | — | — | — | — | — | 49.5 | — | — | |
| Li and Ji2016.01 | 49.5 | 65.4 | 39.8 | — | — | — | — | — | — | — | |
| Li and Ji2019.05 | 49.5 | 65.4 | 39.8 | — | — | — | — | — | — | — | |
| Li and Ji (2014)Evaluation Criteria=ACE05♠2020.10 | 49.5 | 65.4 | 39.8 | — | — | — | — | — | — | — | |
| BERT fine-tuning out-of-boxDomain Adaptation=No, Setting=Single-Relation per Pass2019.02 | — | — | — | 5.56 | 5.53 | 1.67 | 4.25 | — | — | — | |
| BERTspSetting=Multi-Relation per Pass2019.02 | — | — | — | 67.09 | 53.2 | 52.73 | 57.67 | — | — | — | |
| BERTspSetting=Single-Relation per Pass2019.02 | — | — | — | 66.95 | 55.43 | 54.39 | 58.92 | — | — | — | |
| BERTsp w/ entity-indicator on input-layerSetting=Multi-Relation per Pass2019.02 | — | — | — | 66.86 | 57.65 | 53.56 | 59.36 | — | — | — | |
| BERTsp w/ entity-indicator on input-layerSetting=Single-Relation per Pass2019.02 | — | — | — | 69.76 | 58.05 | 56.27 | 61.36 | — | — | — | |
| BERTsp w/ pos-emb on final att-layerSetting=Multi-Relation per Pass2019.02 | — | — | — | 69.13 | 58.68 | 55.04 | 60.95 | — | — | — | |
| Best published results w/o DA (from Fu et al.)Domain Adaptation=No, Setting=Single-Relation per Pass2019.02 | — | — | — | 64.44 | 54.58 | 57.02 | 58.68 | — | — | — | |
| Dixit and Al-OnaizanEncoder=LSTM + ELMo2020.10 | — | — | — | — | — | — | — | 62.8 | 86 | — | |
| Domain Adversarial NetworkDomain Adaptation=Yes, Setting=Single-Relation per Pass2019.02 | — | — | — | 65.16 | 55.55 | 57.19 | 59.3 | — | — | — | |
| Entity-Aware BERTspSetting=Multi-Relation per Pass2019.02 | — | — | — | 69.25 | 61.7 | 58.48 | 63.14 | — | — | — | |
| Entity-Aware BERTspSetting=Single-Relation per Pass2019.02 | — | — | — | 68.52 | 63.71 | 57.2 | 63.14 | — | — | — | |
| Genre Separation NetworkDomain Adaptation=Yes, Setting=Single-Relation per Pass2019.02 | — | — | — | 66.38 | 57.92 | 56.84 | 60.38 | — | — | — | |
| Hybrid FCMDomain Adaptation=No, Setting=Single-Relation per Pass2019.02 | — | — | — | 63.48 | 56.12 | 55.17 | 58.26 | — | — | — | |
| Li et al.Averaging=micro-averaged2020.10 | — | — | — | — | — | — | — | 60.2 | — | — | |
| Li et al.Encoder=BERT-large2020.10 | — | — | — | — | — | — | — | 60.2 | 84.8 | — | |
| Miwa and BansalAveraging=micro-averaged2020.10 | — | — | — | — | — | — | — | 55.6 | — | — | |
| Miwa and BansalEncoder=LSTM2020.10 | — | — | — | — | — | — | — | 55.6 | 83.4 | — | |
| Multi-turn QAPre-trained embedding=BERT, Match setting=exact match2021.08 | — | — | — | — | — | — | — | — | — | 60.2 | |
| PFNPre-trained embedding=BERT, Match setting=exact match2021.08 | — | — | — | — | — | — | — | — | — | 66.8 | |
| PUREPre-trained embedding=ALBERT, Match setting=exact match, Setting=single-sentence2021.08 | — | — | — | — | — | — | — | — | — | 65.6 | |
| SPTreeMatch setting=exact match2021.08 | — | — | — | — | — | — | — | — | — | 55.6 | |
| Structured PerceptronMatch setting=exact match2021.08 | — | — | — | — | — | — | — | — | — | 49.5 | |
| Sun et al.Averaging=micro-averaged2020.10 | — | — | — | — | — | — | — | 59.6 | — | — | |
| Table-SequencePre-trained embedding=ALBERT, Match setting=exact match2021.08 | — | — | — | — | — | — | — | — | — | 64.3 | |
| UniREEncoder=BERT_BASE, Representation Type=T-Concat2021.09 | — | — | — | — | — | — | — | 64.3 | — | — | |
| UniREEncoder=ALBERT_XXL, Representation Type=T-Concat2021.09 | — | — | — | — | — | — | — | 66 | — | — | |
| Zhang et al.Averaging=micro-averaged2020.10 | — | — | — | — | — | — | — | 57.5 | — | — | |
| Zhang et al.Encoder=LSTM2020.10 | — | — | — | — | — | — | — | 57.5 | 83.6 | — |