Hyper-relational extraction on HyperRED (dev)
91.9PrecisionText2NKG
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Text2NKGPLM=BERT-large (340M), Training Paradigm=Supervised2023.10 | 91.9 | 79.43 | 85.21 | |
| Text2NKGPLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 91.26 | 79.36 | 84.89 | |
| Text2NKG w/o alphaPLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 88.77 | 78.39 | 83.26 | |
| Text2NKG w/o DAPLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 76.02 | 72.28 | 74.1 | |
| Pipeline Baseline (Large)Parameters=680M2022.11 | 70.58 | 66.58 | 68.52 | |
| Pipeline BaselinePLM=BERT-large (340M), Training Paradigm=Supervised2023.10 | 70.58 | 66.58 | 68.52 | |
| Pipeline Baseline (Medium)Parameters=221M2022.11 | 69.7 | 62.33 | 65.8 | |
| Pipeline BaselineModel Scale=Base, Parameters=132M2022.11 | 69.23 | 58.21 | 63.24 | |
| Pipeline BaselineParameters=132M2022.11 | 69.23 | 58.21 | 63.24 | |
| Pipeline BaselinePLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 69.23 | 58.21 | 63.24 | |
| CubeREModel Scale=Large, Parameters=343M2022.11 | 68.75 | 68.88 | 68.81 | |
| CubeRE (Large)Parameters=343M2022.11 | 68.75 | 68.88 | 68.81 | |
| CubeREPLM=BERT-large (340M), Training Paradigm=Supervised2023.10 | 68.75 | 68.88 | 68.81 | |
| Generative BaselineModel Scale=Large, Parameters=400M2022.11 | 67.08 | 65.73 | 66.4 | |
| Generative Baseline (Large)Parameters=400M2022.11 | 67.08 | 65.73 | 66.4 | |
| Generative BaselinePLM=BERT-large (340M), Training Paradigm=Supervised2023.10 | 67.08 | 65.73 | 66.4 | |
| CubeREModel Scale=Base, Parameters=115M2022.11 | 66.14 | 64.39 | 65.24 | |
| CubeREParameters=115M2022.11 | 66.14 | 64.39 | 65.24 | |
| CubeREPLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 66.14 | 64.39 | 65.23 | |
| Generative BaselineModel Scale=Base, Parameters=140M2022.11 | 63.79 | 59.94 | 61.8 | |
| Generative BaselineParameters=140M2022.11 | 63.79 | 59.94 | 61.8 | |
| Generative BaselinePLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 63.79 | 59.94 | 61.8 | |
| Text2NKG w/o HMPLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 61.74 | 76.97 | 68.52 | |
| GPT-4PLM=gpt-4, Training Paradigm=Unsupervised2023.10 | 15.7324 | 15.2377 | 15.4811 | |
| ChatGPTPLM=gpt-3.5-turbo, Training Paradigm=Unsupervised2023.10 | 12.0583 | 11.2764 | 11.6542 |