Joint Entity and Relation Extraction on WebNLG (test)
94.6PrecisionCGT (w/o contrastive)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CGT (w/o contrastive)Initialization=UniLM, Contrastive Objective=false2020.09 | 94.6 | 70.5 | 80.8 | |
| CasRel2020.11 | 93.4 | 90.1 | 91.8 | |
| CasRelModel Type=Extractive2020.09 | 93.4 | 90.1 | 91.8 | |
| SPN2020.11 | 93.1 | 93.6 | 93.4 | |
| CGTInitialization=UniLM, Contrastive Objective=true2020.09 | 92.9 | 75.6 | 83.4 | |
| GRTE (BERT)Entity Setting=Multi-token2022.11 | 92.3 | 87.9 | 90 | |
| OneRel (BERT)Entity Setting=Multi-token2022.11 | 91.8 | 90.3 | 91 | |
| UniRelEntity Setting=Multi-token2022.11 | 91.8 | 90.5 | 91.1 | |
| PRGC (BERT)Entity Setting=Multi-token2022.11 | 89.9 | 87.2 | 88.5 | |
| BiTT-BERTEncoder=BERT-BASE2020.08 | 89.1 | 83 | 86.2 | |
| TPLinkerEncoder=BERT-BASE2020.08 | 88.9 | 84.5 | 86.7 | |
| TPLinker (BERT)Entity Setting=Multi-token2022.11 | 88.9 | 84.5 | 86.7 | |
| RINEvaluation Criteria=Partial Match2020.05 | 87.6 | 87 | 87.3 | |
| CGTInitialization=Random, Contrastive Objective=true2020.09 | 87.6 | 70.5 | 78.1 | |
| RINw/o interactionEvaluation Criteria=Partial Match2020.05 | 84.9 | 86.3 | 85.6 | |
| ETL-spanEntity Setting=Multi-token2022.11 | 84.3 | 82 | 83.1 | |
| BiTT-LSTMEncoder=LSTM2020.08 | 83.8 | 66 | 73.8 | |
| RSANEntity Setting=Multi-token2022.11 | 80.5 | 83.8 | 82.1 | |
| RINEvaluation Criteria=Exact Match2020.05 | 77.3 | 76.8 | 77 | |
| RINw/o interactionEvaluation Criteria=Exact Match2020.05 | 75 | 73.3 | 74.2 | |
| MrMepModel Type=Extractive2020.09 | 69.4 | 77 | 73 | |
| OrderRLEvaluation Criteria=Partial Match2020.05 | 63.3 | 59.9 | 61.6 | |
| CopyRRL2020.11 | 63.3 | 59.9 | 61.6 | |
| CopyMLT-MulEvaluation Criteria=Exact Match2020.05 | 58 | 54.9 | 56.4 | |
| CopyMTL-MulDecoder strategy=MultiDecoder, Evaluation protocol=Strict2019.11 | 58 | 54.9 | 56.4 | |
| CopyMTL-MulEncoder=LSTM2020.08 | 58 | 54.9 | 56.4 | |
| CopyMTLModel Type=Generative2020.09 | 58 | 54.9 | 56.4 | |
| CopyMLT-OneEvaluation Criteria=Exact Match2020.05 | 57.8 | 60.1 | 58.9 | |
| CopyMTL-OneDecoder strategy=OneDecoder, Evaluation protocol=Strict2019.11 | 57.8 | 60.1 | 58.9 | |
| MultiHeadEntity Setting=Multi-token2022.11 | 57.5 | 54.1 | 55.7 | |
| HRLModel Type=Extractive2020.09 | 53.8 | 53.8 | 53.8 | |
| NovelTaggingEvaluation Criteria=Exact Match2020.05 | 52.5 | 19.3 | 28.3 | |
| NovelTaggingEvaluation protocol=Strict2019.11 | 52.5 | 19.3 | 28.3 | |
| NovelTagging2020.11 | 52.5 | 19.3 | 28.3 | |
| NovelTaggingEncoder=LSTM2020.08 | 52.5 | 19.3 | 28.3 | |
| NovelTaggingEntity Setting=Multi-token2022.11 | 52.5 | 19.3 | 28.3 | |
| GraphRel2pEvaluation Criteria=Exact Match2020.05 | 44.7 | 41.1 | 42.9 | |
| GraphRel-2pNumber of phases=2p, Evaluation protocol=Strict2019.11 | 44.7 | 41.1 | 42.9 | |
| GraphRelConfig=2p2020.11 | 44.7 | 41.1 | 42.9 | |
| GraphRel-2pEncoder=LSTM + GCN2020.08 | 44.7 | 41.1 | 42.9 | |
| GraphRel1pEvaluation Criteria=Exact Match2020.05 | 42.3 | 39.2 | 40.7 | |
| GraphRel-1pNumber of phases=1p, Evaluation protocol=Strict2019.11 | 42.3 | 39.2 | 40.7 | |
| GraphRelConfig=1p2020.11 | 42.3 | 39.2 | 40.7 | |
| PNDecModel Type=Generative2020.09 | 38.1 | 36.9 | 37.5 | |
| MultiDecoderEvaluation Criteria=Partial Match2020.05 | 37.7 | 36.4 | 37.1 | |
| CopyREMode=Mul2020.11 | 37.7 | 36.4 | 37.1 | |
| CopyRE-MulEncoder=LSTM2020.08 | 37.7 | 36.4 | 37.1 | |
| CopyREModel Type=Generative2020.09 | 37.7 | 36.4 | 37.1 | |
| OneDecoderEvaluation Criteria=Partial Match2020.05 | 32.2 | 28.9 | 30.5 | |
| CopyREMode=One2020.11 | 32.2 | 28.9 | 30.5 | |
| CopyRE-MulDecoder strategy=MultiDecoder, Evaluation protocol=Less strict2019.11 | 31.9 | 27.3 | 29.4 | |
| CopyRE-OneDecoder strategy=OneDecoder, Evaluation protocol=Less strict2019.11 | 31.2 | 27.2 | 29.1 |