Named Entity Recognition on CoNLL German 2003 (test)
88.34F1 ScoreTransformer-Linear (+DEV)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Transformer-Linear (+DEV)Approach=Fine-tuning (Ablations), Document features=yes (+enforce), Training=Includes dev data2020.11 | 88.34 | — | — | |
| Flair2019.08 | 88.32 | — | — | |
| ACEFine-tuning=true, Trained on both train and development set=true2020.10 | 88.3 | — | — | |
| Akbik, Blythe, and Vollgrafmodel_year=20182019.11 | 88.2 | — | — | |
| XLM-RFine-tuning=true, Trained on both train and development set=true2020.10 | 87.7 | — | — | |
| Transformer-LinearApproach=Fine-tuning (Ablations), Document features=yes (+enforce)2020.11 | 87.35 | — | — | |
| Transformer-LinearApproach=Fine-tuning, Document features=yes2020.11 | 86.99 | — | — | |
| Transformer-LinearApproach=Fine-tuning, Document features=no2020.11 | 86.6 | — | — | |
| Yu et al. (2020)Document features=yes2020.11 | 86.4 | — | — | |
| Yu et al.2020.10 | 86.4 | — | — | |
| seq2seqembeddings=BERT + Flair2019.08 | 85.1 | — | — | |
| Straková et al. (2019)Document features=yes2020.11 | 85.1 | — | — | |
| Straková et al.2020.10 | 85.1 | — | — | |
| LSTM-CRF (all layer mean)Approach=Feature-based, Document features=yes2020.11 | 84.86 | — | — | |
| Trankit2021.01 | 84.6 | — | — | |
| LSTM-CRFembeddings=BERT2019.08 | 84.53 | — | — | |
| LSTM-CRFembeddings=BERT + Flair2019.08 | 84.44 | — | — | |
| seq2seqembeddings=BERT2019.08 | 84.19 | — | — | |
| BERT-MLtraining=multi-task learning2019.11 | 83.3 | — | — | |
| LSTM-CRF (all layer mean)Approach=Feature-based, Document features=no2020.11 | 82.88 | — | — | |
| BERT-SLtraining=single language2019.11 | 82.7 | — | — | |
| seq2seqembeddings=Flair2019.08 | 82.68 | — | — | |
| FLAIRNumber of entity types=4, Training status=Retrained2020.03 | 82.5 | — | — | |
| FlairVersion=v0.72021.01 | 82.5 | — | — | |
| LSTM-CRFembeddings=Flair2019.08 | 82.35 | — | — | |
| StanzaNumber of entity types=4, Training status=Retrained2020.03 | 81.9 | — | — | |
| StanzaVersion=v1.1.12021.01 | 81.9 | — | — | |
| LSTM-CRFembeddings=None2019.08 | 79.89 | — | — | |
| LSTM-CRF+char-IntNet-9Character Embedding Model=IntNet-92018.10 | 79.43 | — | — | |
| seq2seqembeddings=None2019.08 | 79.09 | — | — | |
| Lample et al.model_year=20162019.11 | 78.8 | — | — | |
| LSTM-CRFcharacter modeling=true2016.03 | 78.76 | — | — | |
| LSTM-CRF+char-LSTMCharacter Embedding Model=LSTM, Reference=Lample et al., 20162018.10 | 78.76 | — | — | |
| Lample et al.2019.08 | 78.76 | — | — | |
| LSTM-CRF+char-IntNet-5Character Embedding Model=IntNet-52018.10 | 78.58 | — | — | |
| LSTM-CRF+char-LSTMCharacter Embedding Model=LSTM, Type=Re-implementation2018.10 | 78.31 | — | — | |
| LSTM-CRF+char-CNNCharacter Embedding Model=CNN, Type=Re-implementation2018.10 | 78.15 | — | — | |
| XLM-REvaluation Protocol=fine-tune2021.01 | 76.9 | 75.5 | 78.4 | |
| ixa-pipe-nercFeatures=de-cluster + dict2017.01 | 76.42 | 83.72 | 70.3 | |
| Faruqui and Padó2017.01 | 76.4 | 86.4 | 68.5 | |
| Gillick et al.external labeled data=true2016.03 | 76.22 | — | — | |
| Gillick et al.2019.08 | 76.22 | — | — | |
| Qi et al.external labeled data=false2016.03 | 75.72 | — | — | |
| S-LSTMcharacter modeling=true2016.03 | 75.66 | — | — | |
| ixa-pipe-nercFeatures=CWac500 + W2VWac100 (de-cluster)2017.01 | 75.51 | 83.43 | 68.96 | |
| Ando and Zhangexternal labeled data=false2016.03 | 75.27 | — | — | |
| LSTM-CRFcharacter modeling=false2016.03 | 75.06 | — | — | |
| Flair-MLtraining=multilingual2019.11 | 74.9 | — | — | |
| Wu et al. (2020) [3]2021.01 | 74.82 | — | — | |
| mBERTEvaluation Protocol=fine-tune2021.01 | 74.6 | 73.1 | 76.2 | |
| ixa-pipe-nercFeatures=L + Clark deWac 500 (CWac500)2017.01 | 74.25 | 82.76 | 67.33 | |
| ixa-pipe-nercFeatures=L + Brown deWac (BWac)2017.01 | 73.57 | 82.26 | 66.53 | |
| LSTM-CRFType=Baseline2018.10 | 73.42 | — | — | |
| ixa-pipe-nercFeatures=L + Word2vec deWac 100 (W2VWac100)2017.01 | 73.41 | 81.96 | 66.46 | |
| XLM-REvaluation Protocol=zero-transfer2021.01 | 73.4 | 67.9 | 79.8 | |
| OursSource Language=English, Backbone=BERT_BASE, Layer Freezing Policy=Bottom 3 layers2020.04 | 73.22 | — | — | |
| Wu et al. (2020) [2]2021.01 | 73.22 | — | — | |
| Wu et al.Source Language=English2020.04 | 73.16 | — | — | |
| Wu et al. (2020) [1]2021.01 | 73.16 | — | — | |
| Florian et al.external labeled data=true2016.03 | 72.41 | — | — | |
| Florian et al.2017.01 | 72.41 | 80.19 | 63.71 | |
| CoNLL'02 and '03 baseline2019.11 | 72.4 | — | — | |
| mBERTEvaluation Protocol=zero-transfer2021.01 | 72.1 | 67.6 | 77.4 | |
| Gillick et al.external labeled data=false2016.03 | 72.08 | — | — | |
| Moon et al.Source Language=English, Backbone=BERT_BASE, Layer Freezing Policy=Bottom 3 layers2020.04 | 71.42 | — | — | |
| Moon et al. (2019)2021.01 | 71.42 | — | — | |
| Wu and DredzeSource Language=English, Backbone=BERT_BASE, Layer Freezing Policy=Bottom 3 layers2020.04 | 71.1 | — | — | |
| Wu and Dredze (2019)2021.01 | 71.1 | — | — | |
| ixa-pipe-nercFeatures=Local (L)2017.01 | 69.14 | 81.7 | 59.92 | |
| S-LSTMcharacter modeling=false2016.03 | 65.87 | — | — | |
| Bari et al. (2019)2021.01 | 65.24 | — | — | |
| spaCyNumber of entity types=4, Training status=Retrained2020.03 | 63.9 | — | — | |
| spaCyVersion=v2.32021.01 | 63.9 | — | — | |
| Mayhew et al.Source Language=English2020.04 | 59.11 | — | — | |
| Ni et al.Source Language=English2020.04 | 58.5 | — | — | |
| Xie et al.Source Language=English2020.04 | 57.76 | — | — | |
| Tsai et al.Source Language=English2020.04 | 48.12 | — | — | |
| Täckström et al.Source Language=English2020.04 | 40.4 | — | — |