Named Entity Recognition on Conll 2003
96.77F1 ScoreYAYI-UIE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| YAYI-UIEEvaluation Protocol=Supervised2024.11 | 96.77 | — | — | — | — | |
| YAYI-UIEevaluation=supervised2024.11 | 96.77 | — | — | — | — | |
| MUCOShot=5-shot2021.06 | 95.78 | — | — | 96.23 | 95.35 | |
| MAMLShot=5-shot2021.06 | 95.37 | — | — | 94.76 | 96.04 | |
| KnowCoder-XEvaluation Protocol=Supervised2024.11 | 94.69 | — | — | — | — | |
| KnowCoder-Xevaluation=supervised2024.11 | 94.69 | — | — | — | — | |
| WPNShot=5-shot2021.06 | 94.68 | — | — | 94.39 | 95 | |
| KnowCoderevaluation=supervised2024.11 | 94.1 | — | — | — | — | |
| GNERevaluation=supervised2024.11 | 93.6 | — | — | — | — | |
| UniversalNERevaluation=supervised2024.11 | 93.3 | — | — | — | — | |
| RoBERTa-BiLSTM + CRFBackbone=RoBERTa, Architecture Component=BiLSTM + CRF2023.05 | 92.91 | — | — | — | — | |
| GOLLIEevaluation=supervised2024.11 | 92.9 | — | — | — | — | |
| RoBERTa-BiLSTM + self-attentionBackbone=RoBERTa, Architecture Component=BiLSTM + self-attention2023.05 | 92.73 | — | — | — | — | |
| RoBERTa-BiLSTM + Global Context MechanismBackbone=RoBERTa, Architecture Component=BiLSTM + Global Context Mechanism2023.05 | 92.71 | — | — | — | — | |
| RoBERTa + CRFBackbone=RoBERTa, Architecture Component=CRF2023.05 | 92.62 | — | — | — | — | |
| RoBERTa + Global Context MechanismBackbone=RoBERTa, Architecture Component=Global Context Mechanism2023.05 | 92.61 | — | — | — | — | |
| GLINERevaluation=supervised2024.11 | 92.6 | — | — | — | — | |
| RoBERTa + self-attentionBackbone=RoBERTa, Architecture Component=self-attention2023.05 | 92.58 | — | — | — | — | |
| B2NEREvaluation Protocol=Supervised2024.11 | 92.56 | — | — | — | — | |
| B2NERevaluation=supervised2024.11 | 92.56 | — | — | — | — | |
| RoBERTa-BiLSTMBackbone=RoBERTa, Architecture Component=BiLSTM2023.05 | 92.52 | — | — | — | — | |
| BiLSTM(2) (RoBERTa features)Backbone=RoBERTa, Architecture Component=BiLSTM(2)2023.05 | 92.52 | — | — | — | — | |
| IEPILEEvaluation Protocol=Supervised2024.11 | 92.49 | — | — | — | — | |
| IEPILEevaluation=supervised2024.11 | 92.49 | — | — | — | — | |
| RoBERTaBackbone=RoBERTa2023.05 | 92.46 | — | — | — | — | |
| RoBERTa + linearBackbone=RoBERTa, Architecture Component=linear2023.05 | 92.41 | — | — | — | — | |
| BERTEvaluation Protocol=Supervised2024.11 | 92.4 | — | — | — | — | |
| BERTevaluation=supervised2024.11 | 92.4 | — | — | — | — | |
| GRNExternal knowledge=true2019.07 | 92.34 | — | 92.45 | 92.04 | 92.65 | |
| Peters et al. 2018External knowledge=true2019.07 | 92.22 | — | — | — | — | |
| BERT-BiLSTM + CRFBackbone=BERT, Architecture Component=BiLSTM + CRF2023.05 | 91.98 | — | — | — | — | |
| Peters et al. 2017External knowledge=true2019.07 | 91.93 | — | — | — | — | |
| BERT + CRFBackbone=BERT, Architecture Component=CRF2023.05 | 91.85 | — | — | — | — | |
| BERT + linearBackbone=BERT, Architecture Component=linear2023.05 | 91.84 | — | — | — | — | |
| BERT-BiLSTM + Global Context MechanismBackbone=BERT, Architecture Component=BiLSTM + Global Context Mechanism2023.05 | 91.84 | — | — | — | — | |
| BERT-BiLSTMBackbone=BERT, Architecture Component=BiLSTM2023.05 | 91.76 | — | — | — | — | |
| BiLSTM(2) (BERT features)Backbone=BERT, Architecture Component=BiLSTM(2)2023.05 | 91.74 | — | — | — | — | |
| BERT + Global Context MechanismBackbone=BERT, Architecture Component=Global Context Mechanism2023.05 | 91.72 | — | — | — | — | |
| BERT + self-attentionBackbone=BERT, Architecture Component=self-attention2023.05 | 91.66 | — | — | — | — | |
| Tran et al. 2010External knowledge=true2019.07 | 91.66 | — | — | — | — | |
| Chiu and Nichols 2016External knowledge=true2019.07 | 91.62 | — | — | 91.39 | 91.85 | |
| BERT-BiLSTM + self-attentionBackbone=BERT, Architecture Component=BiLSTM + self-attention2023.05 | 91.58 | — | — | — | — | |
| InstructUIEevaluation=supervised2024.11 | 91.53 | — | — | — | — | |
| BERTBackbone=BERT2023.05 | 91.49 | — | — | — | — | |
| GRNExternal knowledge=false2019.07 | 91.44 | — | 91.67 | 91.31 | 91.57 | |
| Ye and Ling 2018External knowledge=false2019.07 | 91.38 | — | 91.53 | — | — | |
| ELMoModel Category=Language Models, Source=Publicly available2019.04 | 91.34 | — | — | — | — | |
| Transfer Learning ModelTransfer setting=w/ transfer2017.03 | 91.26 | — | — | — | — | |
| Yang et al. 2017External knowledge=true2019.07 | 91.26 | — | — | — | — | |
| Liu et al. 2018External knowledge=false2019.07 | 91.24 | — | 91.35 | — | — | |
| Ma & Hovy (2016)2017.03 | 91.21 | — | — | — | — | |
| Ma and Hovy 2016External knowledge=false2019.07 | 91.21 | — | — | 91.35 | 91.06 | |
| Luo et al. (2015)2017.03 | 91.2 | — | — | — | — | |
| Transfer Learning ModelTransfer setting=w/o transfer2017.03 | 91.2 | — | — | — | — | |
| Luo et al. 2015External knowledge=true2019.07 | 91.2 | — | — | — | — | |
| CNN-BiLSTM-CRFExternal knowledge=false2019.07 | 91.17 | — | 91.45 | — | — | |
| Lample et al. (2016)2017.03 | 90.94 | — | — | — | — | |
| Lample et al. 2016External knowledge=false2019.07 | 90.94 | — | — | — | — | |
| Chiu and Nichols 2016External knowledge=false2019.07 | 90.91 | — | — | 90.75 | 91.08 | |
| Passos et al. (2014)2017.03 | 90.9 | — | — | — | — | |
| Peters et al. 2017External knowledge=false2019.07 | 90.87 | — | — | — | — | |
| ID-CNNExternal knowledge=false2019.07 | 90.54 | — | — | — | — | |
| GloVeModel Category=Word vectors, Source=Publicly available2019.04 | 90.25 | — | — | — | — | |
| CNN-BiLSTM-Att-CRFExternal knowledge=false2019.07 | 90.24 | — | 90.48 | — | — | |
| Huang et al. (2015)2017.03 | 90.1 | — | — | — | — | |
| Luo et al. 2015External knowledge=false2019.07 | 89.9 | — | — | — | — | |
| Zukov-Gregoric et al. 2017External knowledge=false2019.07 | 89.83 | — | — | — | — | |
| OursModel Category=Language Models, Source=Small similar source2019.04 | 89.78 | — | — | — | — | |
| Collobert et al. (2011)2017.03 | 89.59 | — | — | — | — | |
| Collobert et al. 2011External knowledge=true2019.07 | 89.59 | — | — | — | — | |
| Liu, Baldwin, and Cohn 2017External knowledge=false2019.07 | 89.5 | — | — | — | — | |
| Collobert et al. 2011External knowledge=false2019.07 | 88.67 | — | — | — | — | |
| Zhuo et al. 2016External knowledge=false2019.07 | 88.12 | — | — | — | — | |
| Gillick et al. (2015)2017.03 | 86.5 | — | — | — | — | |
| OursModel Category=Word vectors, Source=Small similar source2019.04 | 86.36 | — | — | — | — | |
| Rei 2017External knowledge=false2019.07 | 86.26 | — | — | — | — | |
| Rei, Crichton, and Pyysalo 2016External knowledge=false2019.07 | 84.09 | — | — | — | — | |
| LTCShot=5-shot2021.06 | 83.97 | — | — | 85.89 | 82.41 | |
| MUCOshot=1-shot2021.06 | 82.69 | — | — | 81.7 | 83.98 | |
| WPNshot=1-shot2021.06 | 81.4 | — | — | 77.87 | 86.58 | |
| MAMLshot=1-shot2021.06 | 79.8 | — | — | 75.95 | 85.69 | |
| LTCshot=1-shot2021.06 | 73.31 | — | — | 78.19 | 70.36 | |
| PNShot=5-shot2021.06 | 72.7 | — | — | 74.36 | 71.46 | |
| BERTShot=5-shot2021.06 | 70.54 | — | — | 73.94 | 68.28 | |
| BERTshot=1-shot2021.06 | 54.28 | — | — | 61 | 50.46 | |
| PNshot=1-shot2021.06 | 52.1 | — | — | 55.78 | 50.72 | |
| BigBirdMaximum Sequence Length=40962024.10 | — | 55.53 | — | — | — | |
| ChuLoModel Variant=All2024.10 | — | 93.34 | — | — | — | |
| Gemini 1.5 Pro2024.10 | — | 30.36 | — | — | — | |
| GPT-4o2024.10 | — | 22.9 | — | — | — | |
| LongformerMaximum Sequence Length=40962024.10 | — | 55.6 | — | — | — |