Named Entity Recognition on Wnut 2017
68.72F1 ScoreKnowCoder-X
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| KnowCoder-Xevaluation=supervised2024.11 | 68.72 | — | — | — | |
| KnowCoderevaluation=supervised2024.11 | 66.4 | — | — | — | |
| RoBERTa-BiLSTM-CRF + WOMEnhancement=Window-oriented Optimization Module2026.04 | 61.39 | — | — | — | |
| Context + WOMEnhancement=Window-oriented Optimization Module2026.04 | 60.91 | — | — | — | |
| CL-KLEvaluation Setting=W/ CONTEXT2021.05 | 60.45 | — | — | — | |
| SoTAModel=RoBERTa2024.09 | 60.45 | — | — | — | |
| CL-KL2026.04 | 60.45 | — | — | — | |
| SubRegWeighModel=RoBERTa, Variant=K-means2024.09 | 60.29 | — | — | — | |
| CL-L2Evaluation Setting=W/ CONTEXT2021.05 | 60.26 | — | — | — | |
| W/ CONTEXT (Baseline)Evaluation Setting=W/ CONTEXT2021.05 | 60.2 | — | — | — | |
| CrossWeighModel=RoBERTa2024.09 | 60.19 | — | — | — | |
| SubRegWeighModel=RoBERTa, Variant=Random2024.09 | 60.15 | — | — | — | |
| SubRegWeighModel=RoBERTa, Variant=Cos-Sim2024.09 | 60.05 | — | — | — | |
| VanillaModel=RoBERTa2024.09 | 60.04 | — | — | — | |
| RoBERTa-BiLSTM + Global Context MechanismBackbone=RoBERTa, Architecture Component=BiLSTM + Global Context Mechanism2023.05 | 59.61 | — | — | — | |
| RoBERTa-BiLSTMBackbone=RoBERTa, Architecture Component=BiLSTM2023.05 | 59.58 | — | — | — | |
| RoBERTa-BiLSTM-CRFArchitecture=BiLSTM-CRF, Backbone=RoBERTa2026.04 | 59.54 | — | — | — | |
| CL-KLEvaluation Setting=W/O CONTEXT2021.05 | 59.33 | — | — | — | |
| RoBERTa + CRFBackbone=RoBERTa, Architecture Component=CRF2023.05 | 59.31 | — | — | — | |
| RoBERTa + Global Context MechanismBackbone=RoBERTa, Architecture Component=Global Context Mechanism2023.05 | 59.27 | — | — | — | |
| Context(RoBERTa)Backbone=RoBERTa2026.04 | 59.03 | — | — | — | |
| RoBERTa - large + RegLERSUBWORD=true, CLASS=true, TEMP=true2021.04 | 58.9 | — | — | 58.3 | |
| RoBERTa-BiLSTM + self-attentionBackbone=RoBERTa, Architecture Component=BiLSTM + self-attention2023.05 | 58.89 | — | — | — | |
| CL-L2Evaluation Setting=W/O CONTEXT2021.05 | 58.68 | — | — | — | |
| RoBERTaBackbone=RoBERTa2023.05 | 58.29 | — | — | — | |
| BiLSTM(2) (RoBERTa features)Backbone=RoBERTa, Architecture Component=BiLSTM(2)2023.05 | 58.16 | — | — | — | |
| RoBERTa + linearBackbone=RoBERTa, Architecture Component=linear2023.05 | 57.97 | — | — | — | |
| W/O CONTEXT (Baseline)Evaluation Setting=W/O CONTEXT2021.05 | 57.86 | — | — | — | |
| RoBERTa + self-attentionBackbone=RoBERTa, Architecture Component=self-attention2023.05 | 57.79 | — | — | — | |
| RoBERTa - large + BPSUBWORD=false, CLASS=false, TEMP=false2021.04 | 57.5 | — | — | 56.1 | |
| RoBERTa-BiLSTM + CRFBackbone=RoBERTa, Architecture Component=BiLSTM + CRF2023.05 | 57.46 | — | — | — | |
| RoBERTa^ASABackbone architecture=RoBERTa, ASA=true2022.06 | 57.3 | — | — | — | |
| RoBERTa - largeSUBWORD=false, CLASS=false, TEMP=false2021.04 | 57.1 | — | — | 55.9 | |
| Nguyen et al.2021.05 | 56.5 | — | — | — | |
| BERT-BiLSTM + Global Context MechanismBackbone=BERT, Architecture Component=BiLSTM + Global Context Mechanism2023.05 | 56.26 | — | — | — | |
| MINER + WOMEnhancement=Window-oriented Optimization Module2026.04 | 55.88 | — | — | — | |
| BERT-BiLSTM + CRFBackbone=BERT, Architecture Component=BiLSTM + CRF2023.05 | 55.82 | — | — | — | |
| SpanNER + WOMEnhancement=Window-oriented Optimization Module2026.04 | 55.77 | — | — | — | |
| BERT-BiLSTMBackbone=BERT, Architecture Component=BiLSTM2023.05 | 55.76 | — | — | — | |
| LUKEEvaluation Setting=W/O CONTEXT2021.05 | 55.22 | — | — | — | |
| BERT + CRFBackbone=BERT, Architecture Component=CRF2023.05 | 55.16 | — | — | — | |
| BERT + self-attentionBackbone=BERT, Architecture Component=self-attention2023.05 | 55.08 | — | — | — | |
| RoBERTaBackbone architecture=RoBERTa2022.06 | 54.8 | — | — | — | |
| BERT + Global Context MechanismBackbone=BERT, Architecture Component=Global Context Mechanism2023.05 | 54.78 | — | — | — | |
| BERT-BiLSTM + self-attentionBackbone=BERT, Architecture Component=BiLSTM + self-attention2023.05 | 54.56 | — | — | — | |
| MINER(BERT)Backbone=BERT2026.04 | 54.53 | — | — | — | |
| BiLSTM(2) (BERT features)Backbone=BERT, Architecture Component=BiLSTM(2)2023.05 | 54.17 | — | — | — | |
| BERTBackbone=BERT2023.05 | 53.77 | — | — | — | |
| YAYI-UIEevaluation=supervised2024.11 | 53.7 | — | — | — | |
| BERT + linearBackbone=BERT, Architecture Component=linear2023.05 | 53.41 | — | — | — | |
| SPANNERvariant=both (sp2)2021.06 | 52.97 | — | — | — | |
| RoBERTa - base + RegLERSUBWORD=true, CLASS=true, TEMP=true2021.04 | 52.8 | — | — | 51.2 | |
| RegLER(BERT)Backbone=BERT2026.04 | 52.74 | — | — | — | |
| RoBERTa - base + BPSUBWORD=false, CLASS=false, TEMP=false2021.04 | 52.5 | — | — | 50.8 | |
| RoBERTa - baseSUBWORD=false, CLASS=false, TEMP=false2021.04 | 52.4 | — | — | 50.1 | |
| SPANNERvariant=generic (sp1)2021.06 | 52.05 | — | — | — | |
| GoLLIE-7BParameters=7B2026.04 | 52 | — | — | — | |
| SpanNER(BERT)Backbone=BERT2026.04 | 51.2 | — | — | — | |
| BERT - large + RegLERSUBWORD=true, CLASS=true, TEMP=true2021.04 | 50.7 | — | — | 49.3 | |
| AESINER2020.10 | 50.68 | — | — | — | |
| Nie et al.2021.05 | 50.36 | — | — | — | |
| BERT†^ASABackbone architecture=BERT, Pre-training strategy=longer-trained with MLM, ASA=true2022.06 | 50.3 | — | — | — | |
| FlairCrossWeigh=true2019.09 | 50.03 | — | — | — | |
| BERT - large + BPSUBWORD=false, CLASS=false, TEMP=false2021.04 | 49.9 | — | — | 48.1 | |
| BERT^ASABackbone architecture=BERT, ASA=true2022.06 | 49.8 | — | — | — | |
| BERT - largeSUBWORD=false, CLASS=false, TEMP=false2021.04 | 49.6 | — | — | 47.7 | |
| Akbik et al. (2019)2020.10 | 49.59 | — | — | — | |
| Devlin et al. (2019)text_encoder=BERT2020.10 | 49.52 | — | — | — | |
| BERT†Backbone architecture=BERT, Pre-training strategy=longer-trained with MLM2022.06 | 49.2 | — | — | — | |
| Yan et al. (2019)2020.10 | 48.98 | — | — | — | |
| FlairCrossWeigh=false2019.09 | 48.96 | — | — | — | |
| sq4Char/Sub.=ELMo, Word=rand, Sent.=LSTM2021.06 | 48.91 | — | — | — | |
| Luo et al. (2018)result_source=Run by authors2020.10 | 48.63 | — | — | — | |
| BERT2026.04 | 48.14 | — | — | — | |
| Dang et al. (2018)2020.10 | 48.12 | — | — | — | |
| BERTBackbone architecture=BERT2022.06 | 47.8 | — | — | — | |
| BERT - base + RegLERSUBWORD=true, CLASS=true, TEMP=true2021.04 | 47.8 | — | — | 46.6 | |
| sq5Char/Sub.=ELMo, Word=none, Sent.=LSTM2021.06 | 47.35 | — | — | — | |
| sq3Char/Sub.=BERT, Word=none, Sent.=LSTM2021.06 | 46.47 | — | — | — | |
| sq2Char/Sub.=BERT, Word=GloVe, Sent.=LSTM2021.06 | 45.23 | — | — | — | |
| BERT - baseSUBWORD=false, CLASS=false, TEMP=false2021.04 | 45.1 | — | — | 43.1 | |
| sq0Char/Sub.=Flair, Word=GloVe, Sent.=LSTM2021.06 | 44.75 | — | — | — | |
| BERT - base + BPSUBWORD=false, CLASS=false, TEMP=false2021.04 | 44.3 | — | — | 42.3 | |
| sq1Char/Sub.=Flair, Word=none, Sent.=LSTM2021.06 | 43.57 | — | — | — | |
| Zhou et al. (2019)2020.10 | 42.83 | — | — | — | |
| Zhou et al.2021.05 | 42.83 | — | — | — | |
| sq6Char/Sub.=cnn, Word=glove, Sent.=LSTM2021.06 | 36.41 | — | — | — | |
| BERTevaluation=supervised, implementation=this paper2024.11 | 36.16 | — | — | — | |
| sq7Char/Sub.=cnn, Word=none, Sent.=LSTM2021.06 | 33.72 | — | — | — | |
| sq8Char/Sub.=none, Word=glove, Sent.=LSTM2021.06 | 18.77 | — | — | — | |
| sq9Char/Sub.=none, Word=rand, Sent.=LSTM2021.06 | 18.39 | — | — | — | |
| CL-KL+SUBWORD=false, CLASS=false, TEMP=false2021.04 | — | — | — | 60.5 | |
| CONTAINERShot=1-shot, Evaluation Protocol=original paper2022.11 | — | 24.2 | — | — | |
| CONTAINERShot=1-shot, Evaluation Protocol=reproduced2022.11 | — | 26.67 | — | — | |
| CONTAINERShot=5-shot, Evaluation Protocol=original paper2022.11 | — | 27.7 | — | — | |
| CONTAINERShot=5-shot, Evaluation Protocol=reproduced2022.11 | — | 35.83 | — | — | |
| DemonstrateNERShot=1-shot, Evaluation Protocol=reproduced2022.11 | — | 20.38 | — | — | |
| DemonstrateNERShot=5-shot, Evaluation Protocol=reproduced2022.11 | — | 43.85 | — | — | |
| DualEncoderShot=1-shot, Evaluation Protocol=reproduced2022.11 | — | 36.03 | — | — | |
| DualEncoderShot=5-shot, Evaluation Protocol=reproduced2022.11 | — | 36.82 | — | — |