Named Entity Recognition on CoNLL Dutch 2002 (test)
95.7F1 ScoreACE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ACEFine-tuning=true, Trained on both train and development set=true2020.10 | 95.7 | — | — | — | |
| XLM-RFine-tuning=true, Trained on both train and development set=true2020.10 | 95.3 | — | — | — | |
| Yu et al.2020.10 | 93.7 | — | — | — | |
| Straková et al.2020.10 | 92.7 | — | — | — | |
| LSTM-CRFembeddings=BERT + Flair2019.08 | 92.69 | — | — | — | |
| LSTM-CRFembeddings=BERT2019.08 | 92.48 | — | — | — | |
| seq2seqembeddings=BERT2019.08 | 92.46 | — | — | — | |
| seq2seqembeddings=BERT + Flair2019.08 | 92.34 | — | — | — | |
| Trankit2021.01 | 91.8 | — | — | — | |
| FlairVersion=v0.7, retrained=true2021.01 | 91.3 | — | — | — | |
| BERT-MLtraining=multi-task learning2019.11 | 91.1 | — | — | — | |
| BERT-SLtraining=single language2019.11 | 90.6 | — | — | — | |
| Akbik, Blythe, and Vollgrafmodel_year=20182019.11 | 90.4 | — | — | — | |
| StanzaVersion=v1.1.12021.01 | 89.2 | — | — | — | |
| Flair-MLtraining=multilingual2019.11 | 88.9 | — | — | — | |
| seq2seqembeddings=Flair2019.08 | 88.67 | — | — | — | |
| LSTM-CRFembeddings=Flair2019.08 | 88.31 | — | — | — | |
| LSTM-CRF+char-IntNet-5Character Embedding Model=IntNet-5, Lexicon Usage=false2018.10 | 87.81 | — | — | — | |
| seq2seqembeddings=None2019.08 | 87.59 | — | — | — | |
| LSTM-CRFembeddings=None2019.08 | 87.42 | — | — | — | |
| LSTM-CRF+char-IntNet-9Character Embedding Model=IntNet-9, Lexicon Usage=false2018.10 | 87.38 | — | — | — | |
| LSTM-CRF+char-LSTMCharacter Embedding Model=LSTM, Lexicon Usage=false, Type=Re-implementation2018.10 | 87.13 | — | — | — | |
| LSTM-CRF+char-CNNCharacter Embedding Model=CNN, Lexicon Usage=false, Type=Re-implementation2018.10 | 86.69 | — | — | — | |
| ixa-pipe-nercFeatures=charngram 1:6 + nl-cluster-dict2017.01 | 85.04 | 86.57 | 83.56 | — | |
| GRM-CRF+char-GRUCharacter Embedding Model=GRU, Lexicon Usage=false, Reference=Yang et al., 20172018.10 | 85 | — | — | — | |
| ixa-pipe-nercFeatures=nl-cluster + dict2017.01 | 84.43 | 85.92 | 83 | — | |
| ixa-pipe-nercFeatures=charngram 1:6 + nl-cluster2017.01 | 83.94 | 85.52 | 82.42 | — | |
| ixa-pipe-nercFeatures=BW+CW400+W2VW100 (nl-cluster)2017.01 | 83.46 | 84.44 | 82.49 | — | |
| UniTransMode=Zero-resource cross-lingual, Source language=English, BERT layers freezing=bottom 3 layers2020.07 | 82.9 | — | — | — | |
| Wu et al. (2020) [3]2021.01 | 82.9 | — | — | — | |
| Gillick et al.External labeled data=true2016.03 | 82.84 | — | — | — | |
| Gillick et al.2019.08 | 82.84 | — | — | — | |
| LSTM-CRFCharacter-level modeling=true2016.03 | 81.74 | — | — | — | |
| LSTM-CRF+char-LSTMCharacter Embedding Model=LSTM, Lexicon Usage=false, Reference=Lample et al., 20162018.10 | 81.74 | — | — | — | |
| Lample et al.2019.08 | 81.74 | — | — | — | |
| Lample et al.model_year=20162019.11 | 81.7 | — | — | — | |
| ixa-pipe-nercFeatures=L + Clark wiki 400 (CW400)2017.01 | 81.67 | 83.18 | 80.21 | — | |
| ixa-pipe-nercFeatures=L + Word2vec wiki 100 (W2VW100)2017.01 | 81.62 | 83.11 | 80.18 | — | |
| ixa-pipe-nercFeatures=L + Brown wiki (BW)2017.01 | 81.32 | 83.12 | 79.56 | — | |
| XLM-REvaluation Protocol=zero-transfer2021.01 | 81.2 | 82.2 | 80.3 | — | |
| Data TransferMode=Zero-resource cross-lingual, Source language=English, BERT layers freezing=bottom 3 layers, Status=reimplemented2020.07 | 80.98 | — | — | — | |
| OursSource Language=English, Backbone=BERT_BASE, Layer Freezing Policy=Bottom 3 layers2020.04 | 80.89 | — | — | — | |
| Wu et al. (2020) [2]2021.01 | 80.89 | — | — | — | |
| Model TransferMode=Zero-resource cross-lingual, Source language=English, BERT layers freezing=bottom 3 layers, Status=reimplemented2020.07 | 80.61 | — | — | — | |
| Wu et al.Source Language=English2020.04 | 80.44 | — | — | — | |
| Wu et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 80.44 | — | — | — | |
| Wu et al. (2020) [1]2021.01 | 80.44 | — | — | — | |
| Moon et al.Source Language=English, Backbone=BERT_BASE, Layer Freezing Policy=Bottom 3 layers2020.04 | 80.38 | — | — | — | |
| Moon et al. (2019)2021.01 | 80.38 | — | — | — | |
| S-LSTMCharacter-level modeling=true2016.03 | 79.88 | — | — | — | |
| LSTM-CRFCharacter Embedding Model=none, Lexicon Usage=false, Type=Baseline2018.10 | 79.87 | — | — | — | |
| XLM-REvaluation Protocol=fine-tune2021.01 | 79.7 | 81.2 | 78.4 | — | |
| Curran and ClarkFeatures=N/A2017.01 | 79.63 | 79.91 | 79.35 | — | |
| Wu and DredzeSource Language=English, Backbone=BERT_BASE, Layer Freezing Policy=Bottom 3 layers2020.04 | 79.5 | — | — | — | |
| Wu and DredzeMode=Zero-resource cross-lingual, Source language=English, BERT layers freezing=bottom 3 layers2020.07 | 79.5 | — | — | — | |
| Wu and Dredze (2019)2021.01 | 79.5 | — | — | — | |
| Nothman et al.External labeled data=false2016.03 | 78.6 | — | — | — | |
| mBERTEvaluation Protocol=zero-transfer2021.01 | 78.6 | 77.8 | 79.3 | — | |
| mBERTEvaluation Protocol=fine-tune2021.01 | 78.6 | 80.5 | 76.7 | — | |
| Gillick et al.External labeled data=false2016.03 | 78.08 | — | — | — | |
| CoNLL'02 and '03 baseline2019.11 | 77.1 | — | — | — | |
| Carreras et al.External labeled data=false2016.03 | 77.05 | — | — | — | |
| Carreras et al.Features=N/A2017.01 | 77.05 | 77.83 | 76.29 | — | |
| ixa-pipe-nercFeatures=Local (L)2017.01 | 76.83 | 78.97 | 74.8 | — | |
| Bari et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 74.61 | — | — | — | |
| Bari et al. (2019)2021.01 | 74.6 | — | — | — | |
| spaCyVersion=v2.32021.01 | 73.8 | — | — | — | |
| LSTM-CRFCharacter-level modeling=false2016.03 | 73.14 | — | — | — | |
| Xie et al.Source Language=English2020.04 | 71.25 | — | — | — | |
| Xie et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 71.25 | — | — | — | |
| S-LSTMCharacter-level modeling=false2016.03 | 69.9 | — | — | — | |
| Jain et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 69.9 | — | — | — | |
| Mayhew et al.Source Language=English2020.04 | 66.5 | — | — | — | |
| Ni et al.Source Language=English2020.04 | 65.4 | — | — | — | |
| Ni et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 65.4 | — | — | — | |
| nl-clusters-dictFeatures=clusters + dict2017.01 | 64.16 | — | — | — | |
| Mayhew et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 63.37 | — | — | — | |
| nl-clustersFeatures=clusters2017.01 | 62.9 | — | — | — | |
| W2VW100Features=W2VW1002017.01 | 61.67 | — | — | — | |
| CW400Features=CW4002017.01 | 61.59 | — | — | — | |
| Tsai et al.Source Language=English2020.04 | 61.56 | — | — | — | |
| Tsai et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 61.56 | — | — | — | |
| BWFeatures=BW2017.01 | 61.53 | — | — | — | |
| Täckström et al.Source Language=English2020.04 | 58.4 | — | — | — | |
| Täckström et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 58.4 | — | — | — | |
| LocalFeatures=Local2017.01 | 57.53 | — | — | — | |
| sonar-nerdFeatures=sonar-nerd2017.01 | 56.67 | — | — | — | |
| FLAIRNumber of entity types=4, Training status=Retrained2020.03 | — | — | — | 90.3 | |
| spaCyNumber of entity types=4, Training status=Retrained2020.03 | — | — | — | 73.8 | |
| StanzaNumber of entity types=4, Training status=Retrained2020.03 | — | — | — | 89.2 |