Named Entity Recognition on ONTONOTES (test)
89.71F1 ScoreBiLSTM-CRF Flair
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| BiLSTM-CRF FlairLM embeddings or extra data=true, Backbone=Flair2019.06 | 89.71 | — | — | |
| Merge and LabelLM embeddings or extra data=true, Backbone=BERT2019.06 | 89.2 | — | — | |
| BiLSTM-CRF with CVTLM embeddings or extra data=true2019.06 | 88.81 | — | — | |
| This workLexical features=true, Gazetteers=false, Capitalization features=true, Word embeddings=true, Character-level embeddings=true, Language Model embeddings=false, Lexical Similarity features=true2018.06 | 87.95 | — | — | |
| BiLSTM-CRF lexLM embeddings or extra data=true2019.06 | 87.95 | — | — | |
| Merge and LabelContextual Embeddings=false2019.06 | 87.59 | — | — | |
| Strubell et al.Lexical features=false, Gazetteers=true, Capitalization features=true, Word embeddings=true, Character-level embeddings=true, Language Model embeddings=false, Lexical Similarity features=false2018.06 | 86.99 | — | — | |
| BiLSTM-CRFContextual Embeddings=false, variant=Strubell et al.2019.06 | 86.99 | — | — | |
| ID-CNNContextual Embeddings=false2019.06 | 86.84 | — | — | |
| Shen et al.Lexical features=false, Gazetteers=false, Capitalization features=true, Word embeddings=true, Character-level embeddings=true, Language Model embeddings=false, Lexical Similarity features=false2018.06 | 86.52 | — | — | |
| Chiu and NicholsLexical features=false, Gazetteers=true, Capitalization features=true, Word embeddings=true, Character-level embeddings=true, Language Model embeddings=false, Lexical Similarity features=false2018.06 | 86.28 | — | — | |
| BiLSTM-CRFContextual Embeddings=false2019.06 | 86.28 | — | — | |
| Ratinov and RothLexical features=true, Gazetteers=true, Capitalization features=true, Word embeddings=true, Character-level embeddings=false, Language Model embeddings=false, Lexical Similarity features=false2018.06 | 84.88 | — | — | |
| Durrett and KleinLexical features=true, Gazetteers=true, Capitalization features=true, Word embeddings=false, Character-level embeddings=false, Language Model embeddings=false, Lexical Similarity features=false2018.06 | 84.04 | — | — | |
| Finkel and ManningLexical features=true, Gazetteers=true, Capitalization features=true, Word embeddings=false, Character-level embeddings=false, Language Model embeddings=false, Lexical Similarity features=false2018.06 | 82.42 | — | — | |
| Passos et al.Lexical features=true, Gazetteers=true, Capitalization features=true, Word embeddings=true, Character-level embeddings=true, Language Model embeddings=false, Lexical Similarity features=false2018.06 | 82.24 | — | — | |
| CPLLType=Ours2023.05 | 81.47 | — | — | |
| Clean dataMethod Category=Upper Bound2023.05 | 81.47 | — | — | |
| LW lossMethod Category=PLL2023.05 | 80.45 | — | — | |
| Token-level VotingMethod Category=Voting2023.05 | 80.12 | — | — | |
| Entity-level VotingMethod Category=Voting2023.05 | 79.3 | — | — | |
| Yang et al.Input=Gold segmentation, Reference Year=2016, semi-supervised=true, discrete features=true2018.05 | 76.4 | 72.98 | 80.15 | |
| Word-based LSTM-CRF (+char+bichar)Input=Gold segmentation2018.05 | 75.77 | 78.62 | 73.13 | |
| PRODEN-mlpMethod Category=PLL2023.05 | 75.36 | — | — | |
| Che et al.Input=Gold segmentation, Reference Year=2013, semi-supervised=true2018.05 | 75.02 | 77.71 | 72.51 | |
| Wang et al.Input=Gold segmentation, Reference Year=2013, semi-supervised=true2018.05 | 74.32 | 76.43 | 72.32 | |
| Lattice LSTMInput=No segmentation2018.05 | 73.88 | 76.35 | 71.56 | |
| Char-based LSTM-CRF (+bichar+softword)Input=No segmentation2018.05 | 71.81 | 74.36 | 69.43 | |
| Word-based LSTM-CRF (+char+bichar)Input=Automatic segmentation2018.05 | 71.7 | 73.36 | 70.12 | |
| Word baselineInput=Gold segmentation2018.05 | 69.52 | 76.66 | 63.6 | |
| Yang et al.Input=Gold segmentation, Reference Year=20162018.05 | 68.57 | 65.59 | 71.84 | |
| Word baselineInput=Automatic segmentation2018.05 | 65.63 | 72.84 | 59.72 | |
| SeqcrowdMethod Category=Annotator2023.05 | 65.34 | — | — | |
| Char baselineInput=No segmentation2018.05 | 64.3 | 68.79 | 60.35 |