Joint Entity and Relation Extraction on NYT (test)
94.7PrecisionCGT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CGTInitialization=UniLM, Contrastive Objective=true2020.09 | 94.7 | 84.2 | 89.1 | |
| UniRelEntity Setting=Multi-token2022.11 | 93.7 | 93.2 | 93.4 | |
| PRGC (BERT)Entity Setting=Multi-token2022.11 | 93.5 | 91.9 | 92.7 | |
| GRTE (BERT)Entity Setting=Multi-token2022.11 | 93.4 | 93.5 | 93.4 | |
| SPNMatching Criteria=Partial Matching2020.11 | 93.3 | 91.7 | 92.5 | |
| OneRel (BERT)Entity Setting=Multi-token2022.11 | 93.2 | 92.6 | 92.9 | |
| SPNMatching Criteria=Exact Matching2020.11 | 92.5 | 92.2 | 92.3 | |
| TPLinker (BERT)Entity Setting=Multi-token2022.11 | 91.4 | 92.6 | 92 | |
| CGTInitialization=Random, Contrastive Objective=true2020.09 | 90.8 | 77.7 | 83.7 | |
| CasRelMatching Criteria=Exact Matching, Reproduced=true2020.11 | 90.1 | 88.5 | 89.3 | |
| CasRelMatching Criteria=Partial Matching2020.11 | 89.7 | 89.5 | 89.6 | |
| CasRelEncoder=BERT-BASE2020.08 | 89.7 | 89.5 | 89.6 | |
| BiTT-BERTEncoder=BERT-BASE2020.08 | 89.7 | 88 | 88.9 | |
| CasRelModel Type=Extractive2020.09 | 89.7 | 89.5 | 89.6 | |
| NovelTaggingEncoder=BERT-BASE2020.08 | 89 | 55.6 | 69.3 | |
| WDecMatching Criteria=Exact Matching2020.11 | 88.1 | 76.1 | 81.7 | |
| Attention as RelationMatching Criteria=Exact Matching2020.11 | 88.1 | 78.5 | 83 | |
| CGT (w/o contrastive)Initialization=UniLM, Contrastive Objective=false2020.09 | 87.3 | 81.5 | 84.3 | |
| RSANEntity Setting=Multi-token2022.11 | 85.7 | 83.6 | 84.6 | |
| ETL-spanEntity Setting=Multi-token2022.11 | 85.5 | 71.7 | 78 | |
| GraphRel-2pEncoder=BERT-BASE + GCN2020.08 | 82.5 | 57.9 | 68.1 | |
| PNDecMatching Criteria=Exact Matching2020.11 | 80.6 | 77.3 | 78.9 | |
| PNDecModel Type=Generative2020.09 | 80.6 | 77.3 | 78.9 | |
| CopyRRLMatching Criteria=Partial Matching2020.11 | 77.9 | 67.2 | 72.1 | |
| MrMepModel Type=Extractive2020.09 | 77.9 | 76.6 | 77.1 | |
| CopyMTL-MulDecoder strategy=MultiDecoder, Evaluation protocol=Strict2019.11 | 75.7 | 68.7 | 72 | |
| CopyMTL-MulMatching Criteria=Exact Matching2020.11 | 75.7 | 68.7 | 72 | |
| CopyMTL-MulEncoder=LSTM2020.08 | 75.7 | 68.7 | 72 | |
| CopyMTLModel Type=Generative2020.09 | 75.7 | 68.7 | 72 | |
| CopyMTL-OneDecoder strategy=OneDecoder, Evaluation protocol=Strict2019.11 | 72.7 | 69.2 | 70.9 | |
| CopyMTL-OneMatching Criteria=Exact Matching2020.11 | 72.7 | 69.2 | 70.9 | |
| HRLModel Type=Extractive2020.09 | 71.4 | 58.6 | 64.4 | |
| BiTT-LSTMEncoder=LSTM2020.08 | 66.5 | 76.3 | 71.1 | |
| NovelTaggingEvaluation protocol=Strict2019.11 | 64.2 | 31.7 | 42 | |
| GraphRel-2pNumber of phases=2p, Evaluation protocol=Strict2019.11 | 63.9 | 60 | 61.9 | |
| GraphRel-2pMatching Criteria=Partial Matching2020.11 | 63.9 | 60 | 61.9 | |
| GraphRel-2pEncoder=LSTM + GCN2020.08 | 63.9 | 60 | 61.9 | |
| GraphRel-1pNumber of phases=1p, Evaluation protocol=Strict2019.11 | 62.9 | 57.3 | 60 | |
| GraphRel-1pMatching Criteria=Partial Matching2020.11 | 62.9 | 57.3 | 60 | |
| NovelTaggingMatching Criteria=Partial Matching2020.11 | 62.4 | 31.7 | 42 | |
| NovelTaggingEncoder=LSTM2020.08 | 62.4 | 31.7 | 42 | |
| TaggingModel Type=Extractive2020.09 | 61.5 | 41.4 | 49.5 | |
| CopyRE-OneDecoder strategy=OneDecoder, Evaluation protocol=Less strict2019.11 | 61.2 | 53 | 57.1 | |
| CopyRE-MulDecoder strategy=MultiDecoder, Evaluation protocol=Less strict2019.11 | 61 | 56.6 | 58.7 | |
| CopyRE-MulMatching Criteria=Partial Matching2020.11 | 61 | 56.6 | 58.7 | |
| CopyRE-MulEncoder=LSTM2020.08 | 61 | 56.6 | 58.7 | |
| CopyREModel Type=Generative2020.09 | 61 | 56.6 | 58.7 | |
| MultiHeadEntity Setting=Multi-token2022.11 | 60.7 | 58.6 | 59.6 | |
| CopyRE-OneMatching Criteria=Partial Matching2020.11 | 59.4 | 53.1 | 56 | |
| PAEncoder=LSTM2020.08 | 49.4 | 59.1 | 53.8 | |
| CopyRE-MulEncoder=BERT-BASE2020.08 | 39.1 | 36.5 | 37.8 | |
| NovelTaggingEntity Setting=Multi-token2022.11 | 32.8 | 30.6 | 31.7 | |
| RINEvaluation Criteria=Partial Match2020.05 | 0.872 | 0.873 | 0.873 | |
| RINw/o interactionEvaluation Criteria=Partial Match2020.05 | 0.839 | 0.831 | 0.835 | |
| RINEvaluation Criteria=Exact Match2020.05 | 0.839 | 0.855 | 0.847 | |
| OrderRLEvaluation Criteria=Partial Match2020.05 | 0.779 | 0.672 | 0.721 | |
| RINw/o interactionEvaluation Criteria=Exact Match2020.05 | 0.774 | 0.764 | 0.769 | |
| CopyMLT-MulEvaluation Criteria=Exact Match2020.05 | 0.757 | 0.687 | 0.72 | |
| CopyMLT-OneEvaluation Criteria=Exact Match2020.05 | 0.727 | 0.692 | 0.709 | |
| GraphRel2pEvaluation Criteria=Exact Match2020.05 | 0.639 | 0.6 | 0.619 | |
| GraphRel1pEvaluation Criteria=Exact Match2020.05 | 0.629 | 0.573 | 0.6 | |
| NovelTaggingEvaluation Criteria=Exact Match2020.05 | 0.624 | 0.317 | 0.42 | |
| MultiDecoderEvaluation Criteria=Partial Match2020.05 | 0.61 | 0.566 | 0.587 | |
| OneDecoderEvaluation Criteria=Partial Match2020.05 | 0.594 | 0.531 | 0.56 |