Named Entity Recognition on CoNLL 2003 (dev)
97.21F1 ScoreLUKE-CR
Evaluation Results
| Method | Links | |
|---|---|---|
| LUKE-CRBackbone=LUKE, Noise handling=CR2021.04 | 97.21 | |
| LUKE-CrossWeighBackbone=LUKE, Noise handling=CrossWeigh2021.04 | 97.09 | |
| LUKEBackbone=LUKE, Noise handling=None2021.04 | 97.03 | |
| BERTLARGEEvaluation Approach=Fine-tuning2018.10 | 96.6 | |
| BERTLARGE-CRBackbone=BERT-large, Noise handling=CR2021.04 | 96.59 | |
| BERTBASEEvaluation Approach=Fine-tuning2018.10 | 96.4 | |
| BERTLARGE-CrossWeighBackbone=BERT-large, Noise handling=CrossWeigh2021.04 | 96.32 | |
| BERTLARGEBackbone=BERT-large, Noise handling=None2021.04 | 96.16 | |
| BERTBASEEvaluation Approach=Feature-based, Layer Selection=Concat Last Four Hidden2018.10 | 96.1 | |
| BERTBASEEvaluation Approach=Feature-based, Layer Selection=Weighted Sum Last Four Hidden2018.10 | 95.9 | |
| BERTBASE-CRBackbone=BERT-base, Noise handling=CR2021.04 | 95.87 | |
| ELMo2018.10 | 95.7 | |
| BERTBASE-CrossWeighBackbone=BERT-base, Noise handling=CrossWeigh2021.04 | 95.65 | |
| BERTBASEEvaluation Approach=Feature-based, Layer Selection=Second-to-Last Hidden2018.10 | 95.6 | |
| BERTBASEBackbone=BERT-base, Noise handling=None2021.04 | 95.58 | |
| BERTBASEEvaluation Approach=Feature-based, Layer Selection=Weighted Sum All 12 Layers2018.10 | 95.5 | |
| Baselinew-bits=32, e-bits=32, Size (MB)=410.9, Size-w/o-e (MB)=324.52019.09 | 95 | |
| BERTBASEEvaluation Approach=Feature-based, Layer Selection=Last Hidden2018.10 | 94.9 | |
| Q-BERTw-bits=4, e-bits=8, Size (MB)=62.2, Size-w/o-e (MB)=40.62019.09 | 94.9 | |
| Q-BERTw-bits=8, e-bits=8, Size (MB)=102.8, Size-w/o-e (MB)=81.22019.09 | 94.79 | |
| Q-BERTw-bits=3, e-bits=8, Size (MB)=52.1, Size-w/o-e (MB)=30.52019.09 | 94.78 | |
| Wang et al. (2021a)2021.05 | 94.6 | |
| Q-BERTMPw-bits=2/4 MP, e-bits=8, Size (MB)=52.1, Size-w/o-e (MB)=30.52019.09 | 94.55 | |
| Q-BERTMPw-bits=2/3 MP, e-bits=8, Size (MB)=45, Size-w/o-e (MB)=23.42019.09 | 94.37 | |
| Yamada et al. (2020)2021.05 | 94.3 | |
| W/ DOC CONTEXTcontext_type=document-level contexts2021.05 | 94.12 | |
| CL-KLcontext_type=retrieved contexts, method=Cooperative Learning KL2021.05 | 93.85 | |
| DeBERTa_largeModel Size=Large2020.06 | 93.8 | |
| Luoma and Pyysalo (2020)trained on training and development sets=true2021.05 | 93.74 | |
| CL-L2context_type=retrieved contexts, method=Cooperative Learning L22021.05 | 93.68 | |
| W/ CONTEXTcontext_type=retrieved contexts2021.05 | 93.55 | |
| Yu et al. (2020)trained on training and development sets=true2021.05 | 93.5 | |
| RoBERTa_largeModel Size=Large2020.06 | 93.4 | |
| W/O CONTEXTcontext_type=none2021.05 | 93.3 | |
| BERT_largeModel Size=Large2020.06 | 92.8 | |
| Q-BERTw-bits=2, e-bits=8, Size (MB)=42, Size-w/o-e (MB)=20.42019.09 | 91.06 | |
| BERTBASEEvaluation Approach=Feature-based, Layer Selection=Embeddings2018.10 | 91 | |
| DirectQw-bits=4, e-bits=8, Size (MB)=62.2, Size-w/o-e (MB)=40.62019.09 | 89.86 | |
| DirectQw-bits=3, e-bits=8, Size (MB)=52.1, Size-w/o-e (MB)=30.52019.09 | 84.92 | |
| DirectQw-bits=2, e-bits=8, Size (MB)=42, Size-w/o-e (MB)=20.42019.09 | 54.5 |