Ultra-Fine Entity Typing (dev)
60.7Total PrecisionLRN w/o IR
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LRN w/o IRAugmented Data=false, Inductive Reasoning=false2021.09 | 60.7 | 32.5 | 42.3 | 79.3 | 75.5 | 77.4 | 59.6 | 44.8 | 51.2 | 45.7 | 18.7 | 26.5 | |
| DenoiseET + ELMoBackbone=ELMo, Data Augmentation=None2019.05 | 55.6 | 28.1 | 37.3 | 69.3 | 77.3 | 73 | 47.9 | 35.4 | 40.7 | 48.9 | 12.6 | 20 | |
| DenoiseET + ELMo (Augmented)Backbone=ELMo, Data Augmentation=Choi et al. (2018)2019.05 | 55.2 | 26.4 | 35.7 | 69.4 | 72 | 70.7 | 46.6 | 38.5 | 42.2 | 48.7 | 10.3 | 17.1 | |
| BERT[in-house]Augmented Data=false2021.09 | 54.1 | 32.1 | 40.3 | 68.8 | 79.2 | 73.6 | 43.8 | 57.4 | 49.7 | 50.7 | 14.6 | 22.6 | |
| LRNAugmented Data=false, Inductive Reasoning=true2021.09 | 53.7 | 38.6 | 44.9 | 77.8 | 76.4 | 77.1 | 55.8 | 50.6 | 53 | 43.4 | 26 | 32.5 | |
| Vectorbackbone=BERT-Large2021.01 | 53.3 | 36.7 | 43.5 | 71.7 | 79.9 | 75.6 | 51.9 | 48.5 | 50.2 | 43.7 | 22.7 | 29.8 | |
| Boxbackbone=BERT-Large2021.01 | 52.9 | 39.1 | 45 | 71.2 | 82.5 | 76.4 | 50.9 | 55.2 | 53 | 45.4 | 24.5 | 31.9 | |
| BERT-Base, Uncased2019.05 | 51.6 | 32.8 | 40.1 | 67.4 | 80.6 | 73.4 | 41.6 | 54.7 | 47.3 | 46.3 | 15.6 | 23.4 | |
| BERTAugmented Data=false2021.09 | 51.6 | 32.8 | 40.1 | 67.4 | 80.6 | 73.4 | 41.6 | 54.7 | 47.3 | 46.3 | 15.6 | 23.4 | |
| DenoiseET + ELMo (Augmented + Filter/Relabel)Backbone=ELMo, Data Augmentation=Choi et al. (2018), Denoising=filter & relabel2019.05 | 50.7 | 33.1 | 40.1 | 66.9 | 80.7 | 73.2 | 41.7 | 46.2 | 43.8 | 45.6 | 17.4 | 25.2 | |
| ELMObackbone=ELMO, context=Onoe and Durrett, 20192021.01 | 50.7 | 33.1 | 40.1 | 66.9 | 80.7 | 73.2 | 41.7 | 46.2 | 43.8 | 45.6 | 17.4 | 25.2 | |
| ELMOAugmented Data=true2021.09 | 50.7 | 33.1 | 40.1 | 66.9 | 80.7 | 73.2 | 41.7 | 46.2 | 43.8 | 45.6 | 17.4 | 25.2 | |
| Label GCNcontext=Xiong et al., 20192021.01 | 49.3 | 28.1 | 35.8 | 66.2 | 68.8 | 67.5 | 43.9 | 40.7 | 42.2 | 42.4 | 14.2 | 21.3 | |
| LABELGCNAugmented Data=false2021.09 | 49.3 | 28.1 | 35.8 | 66.2 | 68.8 | 67.5 | 43.9 | 40.7 | 42.2 | 42.4 | 14.2 | 21.3 | |
| Choi et al. (2018) w augmentationData Augmentation=Yes2019.05 | 48.1 | 23.2 | 31.3 | 60.3 | 61.6 | 61 | 40.4 | 38.4 | 39.4 | 42.8 | 8.8 | 14.6 | |
| Choi et al. (2018)backbone=LSTM, embeddings=GloVe2021.01 | 48.1 | 23.2 | 31.3 | 60.3 | 61.6 | 61 | 40.4 | 38.4 | 39.4 | 42.8 | 8.8 | 14.6 | |
| Choi et al.Augmented Data=true2021.09 | 48.1 | 23.2 | 31.3 | 60.3 | 61.6 | 61 | 40.4 | 38.4 | 39.4 | 42.8 | 8.8 | 14.6 | |
| DenoiseET + GloVeBackbone=GloVe, Data Augmentation=None2019.05 | 46.4 | 23.3 | 31 | 57.7 | 65.5 | 61.4 | 41.3 | 31.3 | 35.6 | 42.4 | 9.2 | 15.1 | |
| HY XLargearchitecture=hyperbolic model, context=López and Strube, 20202021.01 | 43.4 | 34.2 | 38.2 | 61.4 | 73.9 | 67.1 | 35.7 | 46.6 | 40.4 | 36.5 | 19.9 | 25.7 | |
| HY LargeAugmented Data=false2021.09 | 43.4 | 34.2 | 38.2 | 61.4 | 73.9 | 67.1 | 35.7 | 46.6 | 40.4 | 36.5 | 19.9 | 25.7 |