List your company
ResearchFormerly WizWand

Named Entity Recognition on CoNLL 2003 (dev)

97.21F1 ScoreLUKE-CR
52.791664.323375.85587.3867Oct 11, 2018Mar 16, 2019Aug 20, 2019Jan 24, 2020Jun 28, 2020Dec 2, 2020May 8, 2021
Updated 8mo ago

Evaluation results

MethodLinks
LUKE-CRBackbone=LUKE, Noise handling=CR2021.0497.21
LUKE-CrossWeighBackbone=LUKE, Noise handling=CrossWeigh2021.0497.09
LUKEBackbone=LUKE, Noise handling=None2021.0497.03
BERTLARGEEvaluation Approach=Fine-tuning2018.1096.6
BERTLARGE-CRBackbone=BERT-large, Noise handling=CR2021.0496.59
BERTBASEEvaluation Approach=Fine-tuning2018.1096.4
BERTLARGE-CrossWeighBackbone=BERT-large, Noise handling=CrossWeigh2021.0496.32
BERTLARGEBackbone=BERT-large, Noise handling=None2021.0496.16
BERTBASEEvaluation Approach=Feature-based, Layer Selection=Concat Last Four Hidden2018.1096.1
BERTBASEEvaluation Approach=Feature-based, Layer Selection=Weighted Sum Last Four Hidden2018.1095.9
BERTBASE-CRBackbone=BERT-base, Noise handling=CR2021.0495.87
ELMo2018.1095.7
BERTBASE-CrossWeighBackbone=BERT-base, Noise handling=CrossWeigh2021.0495.65
BERTBASEEvaluation Approach=Feature-based, Layer Selection=Second-to-Last Hidden2018.1095.6
BERTBASEBackbone=BERT-base, Noise handling=None2021.0495.58
BERTBASEEvaluation Approach=Feature-based, Layer Selection=Weighted Sum All 12 Layers2018.1095.5
Baselinew-bits=32, e-bits=32, Size (MB)=410.9, Size-w/o-e (MB)=324.52019.0995
BERTBASEEvaluation Approach=Feature-based, Layer Selection=Last Hidden2018.1094.9
Q-BERTw-bits=4, e-bits=8, Size (MB)=62.2, Size-w/o-e (MB)=40.62019.0994.9
Q-BERTw-bits=8, e-bits=8, Size (MB)=102.8, Size-w/o-e (MB)=81.22019.0994.79
Q-BERTw-bits=3, e-bits=8, Size (MB)=52.1, Size-w/o-e (MB)=30.52019.0994.78
Wang et al. (2021a)2021.0594.6
Q-BERTMPw-bits=2/4 MP, e-bits=8, Size (MB)=52.1, Size-w/o-e (MB)=30.52019.0994.55
Q-BERTMPw-bits=2/3 MP, e-bits=8, Size (MB)=45, Size-w/o-e (MB)=23.42019.0994.37
Yamada et al. (2020)2021.0594.3
W/ DOC CONTEXTcontext_type=document-level contexts2021.0594.12
CL-KLcontext_type=retrieved contexts, method=Cooperative Learning KL2021.0593.85
DeBERTa_largeModel Size=Large2020.0693.8
Luoma and Pyysalo (2020)trained on training and development sets=true2021.0593.74
CL-L2context_type=retrieved contexts, method=Cooperative Learning L22021.0593.68
W/ CONTEXTcontext_type=retrieved contexts2021.0593.55
Yu et al. (2020)trained on training and development sets=true2021.0593.5
RoBERTa_largeModel Size=Large2020.0693.4
W/O CONTEXTcontext_type=none2021.0593.3
BERT_largeModel Size=Large2020.0692.8
Q-BERTw-bits=2, e-bits=8, Size (MB)=42, Size-w/o-e (MB)=20.42019.0991.06
BERTBASEEvaluation Approach=Feature-based, Layer Selection=Embeddings2018.1091
DirectQw-bits=4, e-bits=8, Size (MB)=62.2, Size-w/o-e (MB)=40.62019.0989.86
DirectQw-bits=3, e-bits=8, Size (MB)=52.1, Size-w/o-e (MB)=30.52019.0984.92
DirectQw-bits=2, e-bits=8, Size (MB)=42, Size-w/o-e (MB)=20.42019.0954.5