Named Entity Recognition on CoNLL Spanish NER 2002 (test)
95.9F1 ScoreACE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ACEFine-tuning=true, Trained on both train and development set=true2020.10 | 95.9 | — | — | — | — | |
| Yu et al. (2020)Document features=yes2020.11 | 90.3 | — | — | — | — | |
| Yu et al.2020.10 | 90.3 | — | — | — | — | |
| Transformer-LinearApproach=Fine-tuning, Document features=yes2020.11 | 90.14 | — | — | — | — | |
| Transformer-LinearApproach=Fine-tuning, Document features=no2020.11 | 89.94 | — | — | — | — | |
| XLM-RFine-tuning=true, Trained on both train and development set=true2020.10 | 89.3 | — | — | — | — | |
| LSTM-CRF (all layer mean)Approach=Feature-based, Document features=yes2020.11 | 88.98 | — | — | — | — | |
| Trankit2021.01 | 88.9 | — | — | — | — | |
| seq2seqembeddings=BERT2019.08 | 88.81 | — | — | — | — | |
| Straková et al. (2019)Document features=yes2020.11 | 88.81 | — | — | — | — | |
| Straková et al.2020.10 | 88.8 | — | — | — | — | |
| LSTM-CRF (all layer mean)Approach=Feature-based, Document features=no2020.11 | 88.78 | — | — | — | — | |
| LSTM-CRFembeddings=BERT2019.08 | 88.77 | — | — | — | — | |
| StanzaVersion=v1.1.12021.01 | 88.1 | — | — | — | — | |
| BERT-MLtraining=multi-task learning2019.11 | 87.9 | — | — | — | — | |
| BERT-SLtraining=single language2019.11 | 87.5 | — | — | — | — | |
| FlairVersion=v0.72021.01 | 87.3 | — | — | — | — | |
| SPANNER2021.06 | 87.26 | — | — | — | 2.6 | |
| VOF12021.06 | 87.24 | — | — | — | 2.8 | |
| Hire-NERvariant=ALL (+ sentence-level + document-level)2019.11 | 87.08 | — | — | — | — | |
| VCF12021.06 | 87 | — | — | — | 2.8 | |
| Hire-NERvariant=+ sentence-level2019.11 | 86.95 | — | — | — | — | |
| Hire-NERvariant=+ document-level2019.11 | 86.76 | — | — | — | — | |
| LSTM-CRF+char-IntNet-5Character Embedding Model=IntNet-5, Lexicon Usage=false2018.10 | 86.68 | — | — | — | — | |
| Xin et al. 20182019.11 | 86.68 | — | — | — | — | |
| Hire-NERvariant=base model2019.11 | 86.65 | — | — | — | — | |
| Flair-MLtraining=multilingual2019.11 | 86.6 | — | — | — | — | |
| LSTM-CRFembeddings=None2019.08 | 86.34 | — | — | — | — | |
| VM2021.06 | 86.23 | — | — | — | 4.3 | |
| LSTM-CRF+char-LSTMCharacter Embedding Model=LSTM, Lexicon Usage=false, Type=Re-implementation2018.10 | 86.12 | — | — | — | — | |
| seq2seqembeddings=None2019.08 | 86.04 | — | — | — | — | |
| LSTM-CRF+char-CNNCharacter Embedding Model=CNN, Lexicon Usage=false, Type=Re-implementation2018.10 | 85.91 | — | — | — | — | |
| Lample et al.model_year=20162019.11 | 85.8 | — | — | — | — | |
| Yang, Salakhutdinov, and Cohen 20172019.11 | 85.77 | — | — | — | — | |
| LSTM-CRFcharacter-level modeling=true2016.03 | 85.75 | — | — | — | — | |
| LSTM-CRF+char-LSTMCharacter Embedding Model=LSTM, Lexicon Usage=false, Reference=Lample et al., 20162018.10 | 85.75 | — | — | — | — | |
| Lample et al.2019.08 | 85.75 | — | — | — | — | |
| Lample et al. 20162019.11 | 85.75 | — | — | — | — | |
| LSTM-CRF+char-IntNet-9Character Embedding Model=IntNet-9, Lexicon Usage=false2018.10 | 85.71 | — | — | — | — | |
| GRM-CRF+char-GRUCharacter Embedding Model=GRU, Lexicon Usage=false, Reference=Yang et al., 20172018.10 | 84.69 | — | — | — | — | |
| es-clusterFeatures=BP+C(W400+G400)+W2VG4002017.01 | 84.16 | 84.18 | 84.15 | — | — | |
| charngram 1:6 + es-clusterFeatures=charngram 1:6 + es-cluster2017.01 | 84.14 | 84.27 | 84.01 | — | — | |
| S-LSTMcharacter-level modeling=true2016.03 | 83.93 | — | — | — | — | |
| LSTM-CRFcharacter-level modeling=false2016.03 | 83.44 | — | — | — | — | |
| Gillick et al. (2015)external labeled data=true2016.03 | 82.95 | — | — | — | — | |
| Gillick et al.2019.08 | 82.95 | — | — | — | — | |
| Gillick et al. 20152019.11 | 82.95 | — | — | — | — | |
| L + Brown periodico (BP)Features=L + Brown periodico (BP)2017.01 | 82.45 | 82.44 | 82.46 | — | — | |
| Santos and Guimarães (2015)external labeled data=false2016.03 | 82.21 | — | — | — | — | |
| Gillick et al. (2015)external labeled data=false2016.03 | 81.83 | — | — | — | — | |
| L + Clark giga 400 (CG400)Features=L + Clark giga 400 (CG400)2017.01 | 81.78 | 81.75 | 81.82 | — | — | |
| L + Word2vec giga 400 (W2VG400)Features=L + Word2vec giga 400 (W2VG400)2017.01 | 81.67 | 81.56 | 81.76 | — | — | |
| CoNLL'02 and '03 baseline2019.11 | 81.4 | — | — | — | — | |
| Carreras et al. (2002)external labeled data=true2016.03 | 81.39 | — | — | — | — | |
| Carreras et al. (2002)Features=Default as per paper2017.01 | 81.39 | 81.38 | 81.4 | — | — | |
| L + Clark wiki 400 (CW400)Features=L + Clark wiki 400 (CW400)2017.01 | 81.03 | 81.07 | 81 | — | — | |
| XLM-REvaluation Protocol=zero-transfer2021.01 | 80.8 | 79.8 | 81.9 | — | — | |
| LSTM-CRFCharacter Embedding Model=none, Lexicon Usage=false, Type=Baseline2018.10 | 80.33 | — | — | — | — | |
| Local (L)Features=Local (L)2017.01 | 79.52 | 79.49 | 79.54 | — | — | |
| S-LSTMcharacter-level modeling=false2016.03 | 79.46 | — | — | — | — | |
| UniTransMode=Zero-resource cross-lingual, Source language=English, BERT layers freezing=bottom 3 layers2020.07 | 79.31 | — | — | — | — | |
| Wu et al. (2020) [3]2021.01 | 79.31 | — | — | — | — | |
| XLM-REvaluation Protocol=fine-tune2021.01 | 78.9 | 76.9 | 81 | — | — | |
| Data TransferMode=Zero-resource cross-lingual, Source language=English, BERT layers freezing=bottom 3 layers, Status=reimplemented2020.07 | 78.14 | — | — | — | — | |
| mBERTEvaluation Protocol=fine-tune2021.01 | 77.6 | 77.7 | 77.6 | — | — | |
| spaCyVersion=v2.32021.01 | 77.5 | — | — | — | — | |
| OursSource Language=English, Backbone=BERT_BASE, Layer Freezing Policy=Bottom 3 layers2020.04 | 76.94 | — | — | — | — | |
| Wu et al. (2020) [2]2021.01 | 76.94 | — | — | — | — | |
| Wu et al.Source Language=English2020.04 | 76.75 | — | — | — | — | |
| Wu et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 76.75 | — | — | — | — | |
| Wu et al. (2020) [1]2021.01 | 76.75 | — | — | — | — | |
| Model TransferMode=Zero-resource cross-lingual, Source language=English, BERT layers freezing=bottom 3 layers, Status=reimplemented2020.07 | 76.34 | — | — | — | — | |
| Bari et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 75.93 | — | — | — | — | |
| Bari et al. (2019)2021.01 | 75.9 | — | — | — | — | |
| Moon et al.Source Language=English, Backbone=BERT_BASE, Layer Freezing Policy=Bottom 3 layers2020.04 | 75.67 | — | — | — | — | |
| Moon et al. (2019)2021.01 | 75.67 | — | — | — | — | |
| mBERTEvaluation Protocol=zero-transfer2021.01 | 75.2 | 72.4 | 78.2 | — | — | |
| Wu and DredzeSource Language=English, Backbone=BERT_BASE, Layer Freezing Policy=Bottom 3 layers2020.04 | 74.5 | — | — | — | — | |
| Wu and DredzeMode=Zero-resource cross-lingual, Source language=English, BERT layers freezing=bottom 3 layers2020.07 | 74.5 | — | — | — | — | |
| Wu and Dredze (2019)2021.01 | 74.5 | — | — | — | — | |
| Jain et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 73.5 | — | — | — | — | |
| es-clustersFeatures=clusters2017.01 | 72.51 | — | — | — | — | |
| Xie et al.Source Language=English2020.04 | 72.37 | — | — | — | — | |
| Xie et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 72.37 | — | — | — | — | |
| W2VG400Features=W2VG4002017.01 | 72.05 | — | — | — | — | |
| es-clusters-dictFeatures=clusters + dict2017.01 | 71.88 | — | — | — | — | |
| BPFeatures=BP2017.01 | 70.96 | — | — | — | — | |
| CG400 + CW400Features=CG400 + CW4002017.01 | 70.64 | — | — | — | — | |
| LocalFeatures=Local2017.01 | 68.48 | — | — | — | — | |
| Mayhew et al.Source Language=English2020.04 | 65.95 | — | — | — | — | |
| Ni et al.Source Language=English2020.04 | 65.1 | — | — | — | — | |
| Ni et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 65.1 | — | — | — | — | |
| Mayhew et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 64.1 | — | — | — | — | |
| Tsai et al.Source Language=English2020.04 | 60.55 | — | — | — | — | |
| Tsai et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 60.55 | — | — | — | — | |
| Täckström et al.Source Language=English2020.04 | 59.3 | — | — | — | — | |
| Täckström et al.Mode=Zero-resource cross-lingual, Source language=English2020.07 | 59.3 | — | — | — | — | |
| Stanford-ancoraFeatures=Stanford-ancora2017.01 | 49.92 | — | — | — | — | |
| FLAIRNumber of entity types=4, Training status=Retrained2020.03 | — | — | — | 87.3 | — | |
| spaCyNumber of entity types=4, Training status=Retrained2020.03 | — | — | — | 77.5 | — |