Named Entity Recognition on NCBI-disease (test)
90.08PrecisionSOTA (Tian et al., 2020)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SOTA (Tian et al., 2020)2023.05 | 90.08 | — | — | — | — | — | |
| SpanPred x SEQcombiner=MajVote2023.05 | 89.6 | — | — | 91.9 | — | 87.4 | |
| SpanPred x SEQ x SeqCRFcombiner=MajVote2023.05 | 89.5 | — | — | 88.8 | — | 90.1 | |
| Meta(SpanPred U SEQ)combiner=Meta2023.05 | 89.1 | — | — | 86.3 | — | 92.2 | |
| SpanPred2023.05 | 89 | — | — | 88.1 | — | 89.9 | |
| SciFiveModel Size=Large, Pre-training=PMC2021.05 | 88.82 | — | — | 90.14 | 89.23 | — | |
| SEQ2023.05 | 88.7 | — | — | 87.8 | — | 89.5 | |
| SciFiveModel Size=Base, Pre-training=PubMed2021.05 | 88.65 | — | — | 90.14 | 87.96 | — | |
| BioBERTModel Size=Large2021.05 | 88.52 | — | — | 89.82 | 88.46 | — | |
| SciFiveModel Size=Large, Pre-training=PubMed+PMC2021.05 | 88.32 | — | — | 90.14 | 88.46 | — | |
| BlueBERTModel Size=Base2021.05 | 88.28 | — | — | 89.3 | 89.71 | — | |
| BioBERT2021.04 | 88.22 | — | — | 91.25 | 89.71 | — | |
| T5Model Size=Base2021.05 | 88.22 | — | — | 91.25 | 88.54 | — | |
| SpanPred U SEQcombiner=Union2023.05 | 88.2 | — | — | 84.6 | — | 92.2 | |
| BlueBERTModel Size=Large2021.05 | 88.1 | — | — | 90.14 | 89.17 | — | |
| SeqCRF2023.05 | 87.9 | — | — | 86.2 | — | 89.6 | |
| SciFiveModel Size=Base, Pre-training=PubMed+PMC2021.05 | 87.7 | — | — | 89.9 | 88.78 | — | |
| T5Model Size=Large2021.05 | 87.7 | — | — | 89.9 | 89.11 | — | |
| SciFiveModel Size=Large, Pre-training=PubMed2021.05 | 87.64 | — | — | 89.3 | 89.47 | — | |
| SciFiveModel Size=Base, Pre-training=PMC2021.05 | 87.48 | — | — | 90.14 | 89.39 | — | |
| BioBERT-v1.1Number of Parameters=> 60M2022.09 | 87.23 | — | — | 90.07 | 88.62 | — | |
| BERTModel Size=Base, Variant=base2021.05 | 87.18 | — | — | 89.93 | 85.63 | — | |
| SpanPred U SEQ U SeqCRFcombiner=Union2023.05 | 87.1 | — | — | 81.4 | — | 93.8 | |
| CompactBioBERTNumber of Parameters=> 60M2022.09 | 86.91 | — | — | 90.5 | 88.67 | — | |
| DistilBioBERTNumber of Parameters=> 60M2022.09 | 86.74 | — | — | 89.14 | 87.93 | — | |
| BioBERTModel Size=Base2021.05 | 86.28 | — | — | 89.71 | 88.79 | — | |
| ELECTRAMedAveraging=5 runs with different seeds2021.04 | 85.87 | — | — | 89.29 | 87.54 | — | |
| DistilBERTNumber of Parameters=> 60M2022.09 | 85.02 | — | — | 87.78 | 86.38 | — | |
| SOTAModel Size=Base2021.05 | 84.12 | — | — | 87.19 | 88.6 | — | |
| TinyBioBERTNumber of Parameters=15M2022.09 | 82.11 | — | — | 88.57 | 85.22 | — | |
| HighGEN + RoSTERDictionary=Pseudo2022.10 | 77.4 | — | — | 69.4 | 73.2 | — | |
| RoSTERDictionary=Full2022.10 | 75.9 | — | — | 72.7 | 74.3 | — | |
| GeNER + RoSTERDictionary=Pseudo2022.10 | 74.1 | — | — | 68.1 | 71 | — | |
| HighGEN + BONDDictionary=Pseudo2022.10 | 72.9 | — | — | 67.6 | 70.2 | — | |
| GeNER + BONDDictionary=Pseudo2022.10 | 70.8 | — | — | 63.5 | 67 | — | |
| HighGEN + RoSTERDictionary=Pseudo, Ablation=w/o L2022.10 | 69.7 | — | — | 73.2 | 71.4 | — | |
| StandardDictionary=Full2022.10 | 67.5 | — | — | 65.7 | 66.6 | — | |
| HighGEN + StandardDictionary=Pseudo2022.10 | 66.4 | — | — | 44.6 | 53.3 | — | |
| BONDDictionary=Full2022.10 | 63.7 | — | — | 70.6 | 67 | — | |
| GeNER + StandardDictionary=Pseudo2022.10 | 59 | — | — | 37.6 | 45.9 | — | |
| BERT-BaseEvaluation Protocol=Frozen2019.03 | — | — | 84.06 | — | — | — | |
| BERT-BaseEvaluation Protocol=Finetune2019.03 | — | — | 86.88 | — | — | — | |
| BioBERTtraining_mode=finetune2019.03 | — | 89.36 | — | — | — | — | |
| BioFLAIR2021.04 | — | — | — | — | 88.85 | — | |
| CONTAINER#Param=345M, shot=1-shot2023.05 | — | — | — | — | — | 23.24 | |
| CONTAINER#Param=345M, shot=5-shot2023.05 | — | — | — | — | — | 27.02 | |
| GPT-J-6B#Param=6B, shot=1-shot2023.05 | — | — | — | — | — | 35.82 | |
| GPT-J-6B#Param=6B, shot=5-shot2023.05 | — | — | — | — | — | 40.98 | |
| GPT-Neox-20B#Param=20B, shot=1-shot2023.05 | — | — | — | — | — | 35.42 | |
| GPT-Neox-20B#Param=20B, shot=5-shot2023.05 | — | — | — | — | — | 42.85 | |
| GPT2-xl#Param=1.5B, shot=1-shot2023.05 | — | — | — | — | — | 25.54 | |
| GPT2-xl#Param=1.5B, shot=5-shot2023.05 | — | — | — | — | — | 33.25 | |
| MetaNER#Param=770M, shot=1-shot2023.05 | — | — | — | — | — | 40.01 | |
| MetaNER#Param=770M, shot=5-shot2023.05 | — | — | — | — | — | 44.92 | |
| MetaNER-base#Param=220M, shot=1-shot2023.05 | — | — | — | — | — | 35 | |
| MetaNER-base#Param=220M, shot=5-shot2023.05 | — | — | — | — | — | 37.24 | |
| NNShot#Param=345M, shot=1-shot2023.05 | — | — | — | — | — | 31.59 | |
| NNShot#Param=345M, shot=5-shot2023.05 | — | — | — | — | — | 33.14 | |
| OPT-13B#Param=13B, shot=1-shot2023.05 | — | — | — | — | — | 23.73 | |
| OPT-13B#Param=13B, shot=5-shot2023.05 | — | — | — | — | — | 34 | |
| OPT-30B#Param=30B, shot=1-shot2023.05 | — | — | — | — | — | 22.31 | |
| OPT-30B#Param=30B, shot=5-shot2023.05 | — | — | — | — | — | 32.76 | |
| OPT-66B#Param=66B, shot=1-shot2023.05 | — | — | — | — | — | 25.87 | |
| OPT-66B#Param=66B, shot=5-shot2023.05 | — | — | — | — | — | 34.58 | |
| ProtoNet#Param=345M, shot=1-shot2023.05 | — | — | — | — | — | 24.73 | |
| ProtoNet#Param=345M, shot=5-shot2023.05 | — | — | — | — | — | 42.32 | |
| SciBERTtraining_mode=finetune2019.03 | — | 88.57 | — | — | — | — | |
| SCIBERTEvaluation Protocol=Frozen2019.03 | — | — | 86.39 | — | — | — | |
| SCIBERTEvaluation Protocol=Finetune2019.03 | — | — | 88.57 | — | — | — | |
| SOTA (Dogan et al., 2014)2019.03 | — | — | 89.36 | — | — | — | |
| Spark NLP2021.04 | — | — | — | — | 89.13 | — | |
| StructShot#Param=345M, shot=1-shot2023.05 | — | — | — | — | — | 19.87 | |
| StructShot#Param=345M, shot=5-shot2023.05 | — | — | — | — | — | 31.48 | |
| T5-xl#Param=3B, shot=1-shot2023.05 | — | — | — | — | — | 23.1 | |
| T5-xl#Param=3B, shot=5-shot2023.05 | — | — | — | — | — | 36.78 | |
| T5-xxl#Param=11B, shot=1-shot2023.05 | — | — | — | — | — | 12.19 | |
| T5-xxl#Param=11B, shot=5-shot2023.05 | — | — | — | — | — | 26.34 | |
| T5v1.1-large#Param=770M, shot=1-shot2023.05 | — | — | — | — | — | 26.02 | |
| T5v1.1-large#Param=770M, shot=5-shot2023.05 | — | — | — | — | — | 37.63 |