Named Entity Recognition on WNUT 2017 (test)
60.98F1 ScoreEnd-to-end BLSTM-CRF
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| End-to-end BLSTM-CRFClass=location2019.06 | 60.98 | 78.12 | 50 | — | — | — | — | — | — | — | — | |
| Stacked BLSTM-CRFClass=person2019.06 | 60.25 | 73.49 | 51.05 | — | — | — | — | — | — | — | — | |
| End-to-end BLSTM-CRFClass=person2019.06 | 59.92 | 71.15 | 51.75 | — | — | — | — | — | — | — | — | |
| UH-RITUALClass=person2019.06 | 58.66 | 70.72 | 50.12 | — | — | — | — | — | — | — | — | |
| CA-MTLBackbone=RoBERTa-Large, Architecture=24-task CA-MTL2020.09 | 58 | — | — | — | — | — | — | — | — | — | — | |
| XLM-R_LARGEBackbone=XLM-R-Large2020.09 | 57.1 | — | — | — | — | — | — | — | — | — | — | |
| Stacked BLSTM-CRFClass=location2019.06 | 57.04 | 62.2 | 52.67 | — | — | — | — | — | — | — | — | |
| RoBERTa_LARGEBackbone=RoBERTa-Large2020.09 | 56.9 | — | — | — | — | — | — | — | — | — | — | |
| MINERBackbone=Roberta large2022.04 | 54.86 | — | — | — | — | — | — | — | — | — | — | |
| MINERBackbone=Bert large2022.04 | 54.52 | — | — | — | — | — | — | — | — | — | — | |
| UH-RITUALClass=location2019.06 | 52.86 | 56.92 | 49.33 | — | — | — | — | — | — | — | — | |
| DataAug2022.04 | 52.29 | — | — | — | — | — | — | — | — | — | — | |
| MINERBackbone=Albert large2022.04 | 51.94 | — | — | — | — | — | — | — | — | — | — | |
| SpanNERBackbone=Bert large2022.04 | 51.83 | — | — | — | — | — | — | — | — | — | — | |
| SpanNERBackbone=Roberta large2022.04 | 51.65 | — | — | — | — | — | — | — | — | — | — | |
| VaniIB2022.04 | 51.6 | — | — | — | — | — | — | — | — | — | — | |
| InferNER2022.04 | 50.52 | — | — | — | — | — | — | — | — | — | — | |
| SANERaugmentation module=true, gate module=true2020.10 | 50.36 | — | — | — | — | — | — | — | — | — | — | |
| SA-NER2022.04 | 50.36 | — | — | — | — | — | — | — | — | — | — | |
| MIN2022.04 | 49.93 | — | — | — | — | — | — | — | — | — | — | |
| Akbik et al.2020.10 | 49.59 | — | — | — | — | — | — | — | — | — | — | |
| Devlin et al.2020.10 | 49.52 | — | — | — | — | — | — | — | — | — | — | |
| SpanNERBackbone=AlBert large2022.04 | 49.13 | — | — | — | — | — | — | — | — | — | — | |
| Yan et al.2020.10 | 48.98 | — | — | — | — | — | — | — | — | — | — | |
| REAMOinput_modality=Text+Image, scenario=zero-shot2024.06 | 47.4 | — | — | — | — | — | — | — | — | — | — | |
| BiLSTM-CRF+BERT uncased + truecaserTrain Case=Uncased, Embeddings=BERT uncased, Truecaser=included2019.12 | 46.9 | — | — | — | — | — | — | — | — | — | — | |
| BiLSTM-CRF+BERT uncasedTrain Case=Uncased, Embeddings=BERT uncased2019.12 | 46.1 | — | — | — | — | — | — | — | — | — | — | |
| Akbik, Blythe, and Vollgraf 2018Train Case=Cased2019.12 | 46 | — | — | — | — | — | — | — | — | — | — | |
| Aguilar et al. 2018Train Case=Uncased2019.12 | 45.6 | — | — | — | — | — | — | — | — | — | — | |
| Stacked BLSTM-CRFClass=Overall2019.06 | 45.55 | 61.06 | 36.33 | — | — | — | — | — | — | — | — | |
| MiniGPT-v2input_modality=Text+Image, scenario=zero-shot2024.06 | 45.4 | — | — | — | — | — | — | — | — | — | — | |
| End-to-end BLSTM-CRFClass=Overall2019.06 | 44.14 | 66.67 | 32.99 | — | — | — | — | — | — | — | — | |
| BiLSTM-CRF+GloVe uncased + truecaserTrain Case=Cased, Embeddings=GloVe uncased, Truecaser=included2019.12 | 43.7 | — | — | — | — | — | — | — | — | — | — | |
| BiLSTM-CRF+GloVe uncased + truecaserTrain Case=Uncased, Embeddings=GloVe uncased, Truecaser=included2019.12 | 43.2 | — | — | — | — | — | — | — | — | — | — | |
| Cross-BiLSTM-CNN2019.08 | 42.85 | 58.28 | 33.92 | — | — | — | — | — | — | — | — | |
| Zhou et al.2020.10 | 42.83 | — | — | — | — | — | — | — | — | — | — | |
| GloVe + Gold case vectorsTrain Case=Uncased, Embeddings=GloVe + Gold case vectors2019.12 | 42.8 | — | — | — | — | — | — | — | — | — | — | |
| Att-BiLSTM-CNN2019.08 | 42.26 | 55.82 | 34.08 | — | — | — | — | — | — | — | — | |
| UH-RITUALClass=Overall2019.06 | 41.86 | 57.54 | 32.9 | — | — | — | — | — | — | — | — | |
| BiLSTM-CRF+GloVe uncasedTrain Case=Uncased, Embeddings=GloVe uncased2019.12 | 41.1 | — | — | — | — | — | — | — | — | — | — | |
| BiLSTM-CRF+GloVe uncasedTrain Case=Cased, Embeddings=GloVe uncased2019.12 | 41 | — | — | — | — | — | — | — | — | — | — | |
| Baseline-BiLSTM-CNN2019.08 | 40.68 | 53.24 | 32.93 | — | — | — | — | — | — | — | — | |
| CoFEE2022.04 | 39.1 | — | — | — | — | — | — | — | — | — | — | |
| Data augmentation (Mayhew et al. 2019)Train Case=Cased+Uncased2019.12 | 38.6 | — | — | — | — | — | — | — | — | — | — | |
| CRF-BiLSTMcharacter-based word vectors=true2019.08 | 38.24 | — | — | — | — | — | — | — | — | — | — | |
| Data augmentation (Mayhew et al. 2019)Train Case=Cased+Uncased2019.12 | 34.8 | — | — | — | — | — | — | — | — | — | — | |
| BiLSTM-CRF+GloVe uncased + truecaserTrain Case=Cased, Embeddings=GloVe uncased, Truecaser=included2019.12 | 30.2 | — | — | — | — | 12.5 | — | — | — | — | — | |
| Stacked BLSTM-CRFClass=product2019.06 | 28.57 | 40.58 | 22.05 | — | — | — | — | — | — | — | — | |
| Stacked BLSTM-CRFClass=group2019.06 | 27.59 | 47.76 | 19.39 | — | — | — | — | — | — | — | — | |
| InstructBLIP+SEEMinput_modality=Text+Image, scenario=zero-shot2024.06 | 26.9 | — | — | — | — | — | — | — | — | — | — | |
| UH-RITUALClass=corporation2019.06 | 26.55 | 31.91 | 22.73 | — | — | — | — | — | — | — | — | |
| Stacked BLSTM-CRFClass=corporation2019.06 | 24.76 | 33.33 | 19.7 | — | — | — | — | — | — | — | — | |
| MAML2022.04 | 24.19 | — | — | — | — | — | — | — | — | — | — | |
| UH-RITUALClass=group2019.06 | 24.14 | 41.79 | 16.97 | — | — | — | — | — | — | — | — | |
| End-to-end BLSTM-CRFClass=group2019.06 | 23.65 | 63.16 | 14.55 | — | — | — | — | — | — | — | — | |
| LLaVA+SEEMinput_modality=Text+Image, scenario=zero-shot2024.06 | 23 | — | — | — | — | — | — | — | — | — | — | |
| Stacked BLSTM-CRFClass=creative-work2019.06 | 22.83 | 50 | 14.79 | — | — | — | — | — | — | — | — | |
| End-to-end BLSTM-CRFClass=creative-work2019.06 | 17.75 | 55.56 | 10.56 | — | — | — | — | — | — | — | — | |
| End-to-end BLSTM-CRFClass=corporation2019.06 | 17.39 | 30.77 | 12.12 | — | — | — | — | — | — | — | — | |
| BiLSTM-CRF+GloVe uncasedTrain Case=Cased, Embeddings=GloVe uncased2019.12 | 15.7 | — | — | — | — | — | — | — | — | — | — | |
| End-to-end BLSTM-CRFClass=product2019.06 | 14.81 | 34.29 | 9.45 | — | — | — | — | — | — | — | — | |
| UH-RITUALClass=product2019.06 | 14.46 | 30.77 | 9.45 | — | — | — | — | — | — | — | — | |
| UH-RITUALClass=creative-work2019.06 | 12.79 | 36.67 | 7.75 | — | — | — | — | — | — | — | — | |
| Aguilar et al.Lexical Normalization=hard2020.05 | — | — | — | 41.9 | 40.2 | — | — | — | — | — | — | |
| Arcada2019.06 | — | — | — | 39.98 | 37.77 | — | — | — | — | — | — | |
| Bert-basebackbone=BERT-base, shot-size=52022.03 | — | — | — | — | — | — | — | 23.2 | — | — | — | |
| BERTweetLexical Normalization=soft2020.05 | — | — | — | 56.5 | 55.1 | — | — | — | — | — | — | |
| BERTweetLexical Normalization=hard2020.05 | — | — | — | 55.6 | 54.1 | — | — | — | — | — | — | |
| BiLSTM-CRF+BERT uncased + truecaserTrain Case=Uncased, Embeddings=BERT uncased, Truecaser=included2019.12 | — | — | — | — | — | 0.3 | — | — | — | — | — | |
| BiLSTM-CRF+GloVe uncased + truecaserTrain Case=Uncased, Embeddings=GloVe uncased, Truecaser=included2019.12 | — | — | — | — | — | 1.4 | — | — | — | — | — | |
| CONTAINER#Param=345M, shot=1-shot2023.05 | — | — | — | — | — | — | — | 15.64 | — | — | — | |
| CONTAINER#Param=345M, shot=5-shot2023.05 | — | — | — | — | — | — | — | 20.37 | — | — | — | |
| Cross Weigh2022.09 | — | — | — | — | — | — | — | — | 50.3 | — | — | |
| DATNetLexical Normalization=soft2020.05 | — | — | — | 42.3 | — | — | — | — | — | — | — | |
| Devlin et al.re-implemented=true2020.10 | — | — | — | — | — | — | 46.73 | — | — | — | — | |
| Drexel-CCI2019.06 | — | — | — | 26.3 | 25.26 | — | — | — | — | — | — | |
| FLYTXT2019.06 | — | — | — | 38.35 | 36.31 | — | — | — | — | — | — | |
| GPT-J-6B#Param=6B, shot=1-shot2023.05 | — | — | — | — | — | — | — | 31.41 | — | — | — | |
| GPT-J-6B#Param=6B, shot=5-shot2023.05 | — | — | — | — | — | — | — | 30.93 | — | — | — | |
| GPT-Neox-20B#Param=20B, shot=1-shot2023.05 | — | — | — | — | — | — | — | 36.29 | — | — | — | |
| GPT-Neox-20B#Param=20B, shot=5-shot2023.05 | — | — | — | — | — | — | — | 35.68 | — | — | — | |
| GPT2-xl#Param=1.5B, shot=1-shot2023.05 | — | — | — | — | — | — | — | 22.63 | — | — | — | |
| GPT2-xl#Param=1.5B, shot=5-shot2023.05 | — | — | — | — | — | — | — | 24.86 | — | — | — | |
| MetaNER#Param=770M, shot=1-shot2023.05 | — | — | — | — | — | — | — | 31.59 | — | — | — | |
| MetaNER#Param=770M, shot=5-shot2023.05 | — | — | — | — | — | — | — | 36.52 | — | — | — | |
| MetaNER-base#Param=220M, shot=1-shot2023.05 | — | — | — | — | — | — | — | 25.55 | — | — | — | |
| MetaNER-base#Param=220M, shot=5-shot2023.05 | — | — | — | — | — | — | — | 30.41 | — | — | — | |
| MIC-CIS2019.06 | — | — | — | 37.06 | 34.25 | — | — | — | — | — | — | |
| NNShot#Param=345M, shot=1-shot2023.05 | — | — | — | — | — | — | — | 15.26 | — | — | — | |
| NNShot#Param=345M, shot=5-shot2023.05 | — | — | — | — | — | — | — | 21.78 | — | — | — | |
| OPT-13B#Param=13B, shot=1-shot2023.05 | — | — | — | — | — | — | — | 27.74 | — | — | — | |
| OPT-13B#Param=13B, shot=5-shot2023.05 | — | — | — | — | — | — | — | 28.36 | — | — | — | |
| OPT-30B#Param=30B, shot=1-shot2023.05 | — | — | — | — | — | — | — | 25.85 | — | — | — | |
| OPT-30B#Param=30B, shot=5-shot2023.05 | — | — | — | — | — | — | — | 27.44 | — | — | — | |
| OPT-66B#Param=66B, shot=1-shot2023.05 | — | — | — | — | — | — | — | 30.77 | — | — | — | |
| OPT-66B#Param=66B, shot=5-shot2023.05 | — | — | — | — | — | — | — | 32 | — | — | — | |
| Protobackbone=RoBERTa, shot-size=52022.03 | — | — | — | — | — | — | — | 29.5 | — | — | — | |
| Proto-DSbackbone=RoBERTa, shot-size=5, distant-supervision=true2022.03 | — | — | — | — | — | — | — | 35.9 | — | — | — | |
| ProtoNet#Param=345M, shot=1-shot2023.05 | — | — | — | — | — | — | — | 9.74 | — | — | — | |
| ProtoNet#Param=345M, shot=5-shot2023.05 | — | — | — | — | — | — | — | 23.03 | — | — | — |