Ultra-fine entity typing on UFET (test)
59.3PrecisionPL
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| PLBackbone=BERT-base-cased, Model Scale=Large2022.12 | 59.3 | 42.6 | 49.6 | — | |
| PLBackbone=BERT-base-cased, Model Scale=Base2022.12 | 57.8 | 40.7 | 47.7 | — | |
| MCCE-B128 W/O C2CBackbone=RoBERTa-MNLI, Model Scale=Large, Recall Candidates=128, Paradigm=recall-expand-filter, C2C Attention=false, Distant Supervision=true2022.12 | 56.3 | 48.5 | 52.1 | — | |
| DFETBackbone=BERT-base-cased, Model Scale=Base, Distant Supervision=true2022.12 | 55.6 | 44.7 | 49.5 | — | |
| PL-NPCRFBackbone=BERT-base-cased, Model Scale=Large2022.12 | 55.3 | 46.7 | 50.6 | — | |
| LabelGCN2022.10 | 54.8 | 25.9 | 35.1 | 25.3 | |
| LRN2022.02 | 54.5 | 38.9 | 45.4 | — | |
| LRN2022.05 | 54.5 | 38.9 | 45.4 | — | |
| LRNBackbone=BERT-base-cased, Model Scale=Base2022.12 | 54.5 | 38.9 | 45.4 | — | |
| VANILLA CE128Backbone=RoBERTa, Model Scale=Large, Recall Candidates=128, Paradigm=recall-expand-filter2022.12 | 54.5 | 49.3 | 51.8 | — | |
| AttentiveNER2022.10 | 54.2 | 15.2 | 23.7 | 22.3 | |
| MCCE-S128 W/O C2CBackbone=BERT-base-cased, Model Scale=Large, Recall Candidates=128, Paradigm=recall-expand-filter, C2C Attention=false2022.12 | 54.1 | 47.1 | 50.4 | — | |
| MCCE-B128Backbone=BERT-base-cased, Model Scale=Large, Recall Candidates=128, Paradigm=recall-expand-filter2022.12 | 54 | 48.6 | 51.2 | — | |
| MLMET2022.02 | 53.6 | 45.3 | 49.1 | — | |
| MLMETAugmented training data=true2022.05 | 53.6 | 45.3 | 49.1 | — | |
| MLMETBackbone=BERT-base-cased, Model Scale=Base, Distant Supervision=true2022.12 | 53.6 | 45.3 | 49.1 | — | |
| MLMETreproduced=true2022.10 | 53.4 | 40.5 | 46.1 | 29.2 | |
| denoising enhanced ultra-fine entity typing modelthresholding=false2022.10 | 53.4 | 41.9 | 47 | 30.9 | |
| LITETraining Data Setting=NLI+L, Label Dependency=false2022.02 | 53.3 | 46.6 | 49.7 | — | |
| MCCE-S128Backbone=RoBERTa, Model Scale=Base, Recall Candidates=128, Paradigm=recall-expand-filter, Architecture=Sub-tokens2022.12 | 53.3 | 47.3 | 50.1 | — | |
| MCCE-S128Backbone=BERT-base-cased, Model Scale=Base, Recall Candidates=128, Paradigm=recall-expand-filter, Architecture=Sub-tokens2022.12 | 53.2 | 48.3 | 50.6 | — | |
| MCCE-S128 W/O C2CBackbone=RoBERTa, Model Scale=Base, Recall Candidates=128, Paradigm=recall-expand-filter, Architecture=Sub-tokens, C2C Attention=false2022.12 | 53.2 | 46.6 | 49.7 | — | |
| VectorEmb2022.10 | 53 | 36.3 | 43.1 | — | |
| Box4Types2022.02 | 52.8 | 38.8 | 44.8 | — | |
| Box4TypesAugmented training data=true, Model size=Large version2022.05 | 52.8 | 38.8 | 44.8 | — | |
| BoxEmb2022.10 | 52.8 | 38.8 | 44.8 | — | |
| BOX4TYPESBackbone=BERT-base-cased, Model Scale=Base2022.12 | 52.8 | 38.8 | 44.8 | — | |
| MCCE-B128 W/O C2CBackbone=BERT-base-cased, Model Scale=Large, Recall Candidates=128, Paradigm=recall-expand-filter, C2C Attention=false2022.12 | 52.8 | 48.3 | 50.4 | — | |
| MCCE-B128 W/O C2CBackbone=RoBERTa, Model Scale=Base, Recall Candidates=128, Paradigm=recall-expand-filter, Architecture=Fixed-size blocks, C2C Attention=false2022.12 | 52.7 | 46.4 | 49.3 | — | |
| MCCE-B128Backbone=RoBERTa, Model Scale=Base, Recall Candidates=128, Paradigm=recall-expand-filter, Architecture=Fixed-size blocks2022.12 | 52.5 | 47.9 | 50.1 | — | |
| MCCE-S128Backbone=BERT-base-cased, Model Scale=Large, Recall Candidates=128, Paradigm=recall-expand-filter2022.12 | 52.5 | 49.1 | 50.8 | — | |
| LITETraining Data Setting=NLI+L2022.02 | 52.4 | 48.9 | 50.6 | — | |
| LITE+NLI+LBackbone=RoBERTa-MNLI, Model Scale=Large, Distant Supervision=true, Note=Full LITE model2022.12 | 52.4 | 48.9 | 50.6 | — | |
| MCCE-S128 W/O C2CBackbone=BERT-base-cased, Model Scale=Base, Recall Candidates=128, Paradigm=recall-expand-filter, Architecture=Sub-tokens, C2C Attention=false2022.12 | 52.3 | 48.3 | 50.2 | — | |
| MCCE-B128Backbone=RoBERTa, Model Scale=Large, Recall Candidates=128, Paradigm=recall-expand-filter2022.12 | 51.9 | 50.8 | 51.4 | — | |
| BERT2022.10 | 51.6 | 33 | 40.2 | — | |
| MCCE-B128 W/O C2CBackbone=RoBERTa, Model Scale=Large, Recall Candidates=128, Paradigm=recall-expand-filter, C2C Attention=false2022.12 | 51.6 | 51.6 | 51.6 | — | |
| LDET2022.02 | 51.5 | 33 | 40.1 | — | |
| LDET2022.05 | 51.5 | 33 | 40.1 | — | |
| Filter+Relabel2022.10 | 51.5 | 33 | 40.2 | — | |
| LDETBackbone=BERT-base-cased, Model Scale=Base2022.12 | 51.5 | 33 | 40.1 | — | |
| MCCE-S128 W/O C2CBackbone=RoBERTa, Model Scale=Large, Recall Candidates=128, Paradigm=recall-expand-filter, C2C Attention=false2022.12 | 51.5 | 48.8 | 50.1 | — | |
| denoising enhanced ultra-fine entity typing modelthresholding=true2022.10 | 51.2 | 43.7 | 47.3 | 30.9 | |
| MLC-GCNBackbone=RoBERTa, Model Scale=Large2022.12 | 51.2 | 41 | 45.5 | — | |
| MCCE-S128Backbone=RoBERTa, Model Scale=Large, Recall Candidates=128, Paradigm=recall-expand-filter2022.12 | 50.8 | 49.8 | 50.3 | — | |
| LabelGCN2022.02 | 50.3 | 29.2 | 36.9 | — | |
| LabelGCN2022.05 | 50.3 | 29.2 | 36.9 | — | |
| VANILLA CE128Backbone=BERT-base-cased, Model Scale=Large, Recall Candidates=128, Paradigm=recall-expand-filter2022.12 | 50.3 | 49.6 | 49.9 | — | |
| UNISTModel size=Large version2022.05 | 50.2 | 49.6 | 49.9 | — | |
| MCCE-B128Backbone=BERT-base-cased, Model Scale=Base, Recall Candidates=128, Paradigm=recall-expand-filter, Architecture=Fixed-size blocks2022.12 | 49.9 | 50 | 49.9 | — | |
| MCCE-B128 W/O C2CBackbone=BERT-base-cased, Model Scale=Base, Recall Candidates=128, Paradigm=recall-expand-filter, Architecture=Fixed-size blocks, C2C Attention=false2022.12 | 49.9 | 48.2 | 49 | — | |
| VANILLA CE128Backbone=RoBERTa, Model Scale=Base, Recall Candidates=128, Paradigm=recall-expand-filter2022.12 | 49.6 | 49 | 49.3 | — | |
| UNISTModel size=Base version2022.05 | 49.2 | 49.4 | 49.3 | — | |
| LITETraining Data Setting=L2022.02 | 48.7 | 45.8 | 47.2 | — | |
| MLC-NPCRFBackbone=RoBERTa, Model Scale=Large2022.12 | 48.7 | 45.5 | 47 | — | |
| LITE+LBackbone=RoBERTa, Model Scale=Large, Distant Supervision=true, Note=LITE without NLI pretraining2022.12 | 48.7 | 45.8 | 47.2 | — | |
| UFET-biLSTM2022.02 | 48.1 | 23.2 | 31.3 | — | |
| UFET-biLSTM2022.05 | 48.1 | 23.3 | 31.3 | — | |
| MLCBackbone=RoBERTa, Model Scale=Large2022.12 | 47.8 | 40.4 | 43.8 | — | |
| VANILLA CE128Backbone=BERT-base-cased, Model Scale=Base, Recall Candidates=128, Paradigm=recall-expand-filter2022.12 | 47.2 | 48.5 | 47.8 | — | |
| Multi-task2022.10 | 47.1 | 24.2 | 32 | 23.4 | |
| MLCBackbone=BERT-base-cased, Model Scale=Base, Reimplemented=true2022.12 | 46.5 | 34.9 | 39.9 | — | |
| LITETraining Data Setting=NLI+D+L2022.02 | 45.4 | 49.9 | 47.4 | — | |
| MLCBackbone=RoBERTa, Model Scale=Base, Reimplemented=true2022.12 | 42.2 | 44.9 | 43.5 | — | |
| LITETraining Data Setting=D+L2022.02 | 27.5 | 56.4 | 37 | — | |
| LITETraining Data Setting=NLI2022.02 | 1.5 | 7.1 | 2.5 | — |