Hyper-relational extraction on HyperRED (test)
91.06PrecisionText2NKG
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Text2NKGPLM=BERT-large (340M), Training Paradigm=Supervised2023.10 | 91.06 | 77.64 | 83.81 | |
| Text2NKGPLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 90.77 | 77.53 | 83.63 | |
| Text2NKGPLM=BERT-large (340M), Evaluation Protocol=Supervised2023.10 | 88.47 | 80.3 | 84.19 | |
| Text2NKG w/o alphaPLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 88.09 | 76.64 | 81.97 | |
| Text2NKGPLM=BERT-large (340M), Evaluation Protocol=Supervised2023.10 | 86.87 | 80.86 | 83.76 | |
| Text2NKGPLM=BERT-base (110M), Evaluation Protocol=Supervised2023.10 | 86.72 | 78.94 | 82.64 | |
| Text2NKGPLM=BERT-base (110M), Evaluation Protocol=Supervised2023.10 | 86.2 | 79.25 | 82.58 | |
| Text2NKGPLM=BERT-large (340M), Evaluation Protocol=Supervised2023.10 | 85.06 | 86.72 | 85.89 | |
| Text2NKGPLM=BERT-base (110M), Evaluation Protocol=Supervised2023.10 | 83.53 | 86.59 | 85.03 | |
| LasUIEPLM=T5-large (770M), Evaluation Protocol=Supervised2023.10 | 81.29 | 79.54 | 80.4 | |
| LasUIEPLM=T5-base (220M), Evaluation Protocol=Supervised2023.10 | 79.62 | 78.04 | 78.82 | |
| UIEPLM=T5-large (770M), Evaluation Protocol=Supervised2023.10 | 79.38 | 74.69 | 76.96 | |
| LasUIEPLM=T5-large (770M), Evaluation Protocol=Supervised2023.10 | 79.37 | 76.63 | 77.97 | |
| LasUIEPLM=T5-large (770M), Evaluation Protocol=Supervised2023.10 | 77.49 | 74.96 | 76.2 | |
| LasUIEPLM=T5-base (220M), Evaluation Protocol=Supervised2023.10 | 77.01 | 74.26 | 75.61 | |
| UIEPLM=T5-base (220M), Evaluation Protocol=Supervised2023.10 | 76.51 | 73.02 | 74.72 | |
| LasUIEPLM=T5-base (220M), Evaluation Protocol=Supervised2023.10 | 76.21 | 73.75 | 74.96 | |
| Text2EventPLM=T5-large (770M), Evaluation Protocol=Supervised2023.10 | 75.58 | 72.39 | 73.97 | |
| Text2EventPLM=T5-large (770M), Evaluation Protocol=Supervised2023.10 | 75.28 | 72.73 | 73.98 | |
| UIEPLM=T5-large (770M), Evaluation Protocol=Supervised2023.10 | 74.57 | 71.93 | 73.22 | |
| UIEPLM=T5-large (770M), Evaluation Protocol=Supervised2023.10 | 74.47 | 71.84 | 73.14 | |
| Text2EventPLM=T5-base (220M), Evaluation Protocol=Supervised2023.10 | 73.94 | 70.56 | 72.21 | |
| Text2EventPLM=T5-base (220M), Evaluation Protocol=Supervised2023.10 | 73.68 | 70.37 | 71.98 | |
| Text2NKG w/o DAPLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 73.55 | 70.63 | 72.06 | |
| Text2EventPLM=T5-large (770M), Evaluation Protocol=Supervised2023.10 | 73.21 | 70.85 | 72.01 | |
| Text2EventPLM=T5-base (220M), Evaluation Protocol=Supervised2023.10 | 72.73 | 68.45 | 70.52 | |
| UIEPLM=T5-base (220M), Evaluation Protocol=Supervised2023.10 | 72.17 | 69.84 | 70.98 | |
| UIEPLM=T5-base (220M), Evaluation Protocol=Supervised2023.10 | 72.03 | 68.74 | 70.34 | |
| Pipeline Baseline (Medium)Parameters=221M2022.11 | 69.38 | 61.96 | 65.46 | |
| Pipeline Baseline (Large)Parameters=680M2022.11 | 69.21 | 64.27 | 66.65 | |
| Pipeline BaselinePLM=BERT-large (340M), Training Paradigm=Supervised2023.10 | 69.21 | 64.27 | 66.65 | |
| Pipeline BaselineModel Scale=Base, Parameters=132M2022.11 | 69 | 57.55 | 62.75 | |
| Pipeline BaselineParameters=132M2022.11 | 69 | 57.55 | 62.75 | |
| Pipeline BaselinePLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 69 | 57.55 | 62.75 | |
| Generative BaselineModel Scale=Large, Parameters=400M2022.11 | 67.17 | 64.56 | 65.84 | |
| Generative Baseline (Large)Parameters=400M2022.11 | 67.17 | 64.56 | 65.84 | |
| Generative BaselinePLM=BERT-large (340M), Training Paradigm=Supervised2023.10 | 67.17 | 64.56 | 65.84 | |
| CubeREModel Scale=Large, Parameters=343M2022.11 | 66.39 | 67.12 | 66.75 | |
| CubeRE (Large)Parameters=343M2022.11 | 66.39 | 67.12 | 66.75 | |
| CubeREPLM=BERT-large (340M), Training Paradigm=Supervised2023.10 | 66.39 | 67.12 | 66.75 | |
| CubeREModel Scale=Base, Parameters=115M2022.11 | 65.82 | 64.28 | 65.04 | |
| CubeREParameters=115M2022.11 | 65.82 | 64.28 | 65.04 | |
| CubeREPLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 65.82 | 64.28 | 65.04 | |
| Generative BaselineModel Scale=Base, Parameters=140M2022.11 | 64.6 | 59.67 | 62.03 | |
| Generative BaselineParameters=140M2022.11 | 64.6 | 59.67 | 62.03 | |
| Generative BaselinePLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 64.6 | 59.67 | 62.03 | |
| Text2NKG w/o HMPLM=BERT-base (110M), Training Paradigm=Supervised2023.10 | 61.07 | 76.16 | 67.72 | |
| GPT-4PLM=gpt-4, Training Paradigm=Unsupervised2023.10 | 15.8187 | 15.4824 | 15.6487 | |
| GPT-4PLM=gpt-4, Evaluation Protocol=Unsupervised2023.10 | 13.6397 | 12.5355 | 13.0643 | |
| GPT-4PLM=gpt-4, Evaluation Protocol=Unsupervised2023.10 | 13.3681 | 14.6701 | 13.9888 | |
| GPT-4PLM=gpt-4, Evaluation Protocol=Unsupervised2023.10 | 13.0907 | 13.6701 | 13.3741 | |
| ChatGPTPLM=gpt-3.5-turbo, Evaluation Protocol=Unsupervised2023.10 | 11.4387 | 10.4203 | 10.9058 | |
| ChatGPTPLM=gpt-3.5-turbo, Training Paradigm=Unsupervised2023.10 | 11.4021 | 10.9134 | 11.1524 | |
| ChatGPTPLM=gpt-3.5-turbo, Evaluation Protocol=Unsupervised2023.10 | 11.2998 | 11.7852 | 11.5373 | |
| ChatGPTPLM=gpt-3.5-turbo, Evaluation Protocol=Unsupervised2023.10 | 10.4678 | 11.1628 | 10.8041 |