Named Entity Recognition on CoNLL 03
78.1F1 ScoreJPT-4B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| JPT-4Bzero-shot=true2026.04 | 78.1 | — | — | — | — | — | — | |
| UniNER-7Bzero-shot=true2026.04 | 72.2 | — | — | — | — | — | — | |
| GLiNER-Lzero-shot=true2026.04 | 64.6 | — | — | — | — | — | — | |
| CL-KLEvaluation Setting=W/ CONTEXT2021.05 | 0.9356 | — | — | — | — | — | — | |
| Cloze-CNNTraining Setting=standard2021.01 | 0.935 | — | — | — | — | — | — | |
| Baevski et al.2021.05 | 0.935 | — | — | — | — | — | — | |
| Yu et al., 2020b2022.05 | 0.935 | — | — | — | — | — | — | |
| CL-L2Evaluation Setting=W/ CONTEXT2021.05 | 0.9347 | — | — | — | — | — | — | |
| Wang et al.2021.05 | 0.9343 | — | — | — | — | — | — | |
| Li et al.2021.05 | 0.9333 | — | — | — | — | — | — | |
| BERT-MRC+DSCTraining Setting=standard2021.01 | 0.933 | — | — | — | — | — | — | |
| W/ CONTEXT (Baseline)Evaluation Setting=W/ CONTEXT2021.05 | 0.9327 | — | — | — | — | — | — | |
| Wang et al. (2021a)Type=Comparable SOTA2022.03 | 0.9321 | — | — | — | — | — | — | |
| CL-KLEvaluation Setting=W/O CONTEXT2021.05 | 0.9321 | — | — | — | — | — | — | |
| LS-unLLaMA-2-7BModel Variant=LS-unLLaMA, Model Scale=7B2023.10 | 0.9319 | — | — | — | — | — | — | |
| DEEPSTRUCTmode=multi-task2022.05 | 0.931 | — | — | — | — | — | — | |
| CL-L2Evaluation Setting=W/O CONTEXT2021.05 | 0.9308 | — | — | — | — | — | — | |
| W/O CONTEXT (Baseline)Evaluation Setting=W/O CONTEXT2021.05 | 0.9303 | — | — | — | — | — | — | |
| BERT-MRCTraining Setting=standard2021.01 | 0.93 | — | — | — | — | — | — | |
| DEEPSTRUCTmode=w/ finetune2022.05 | 0.93 | — | — | — | — | — | — | |
| UIEBackbone=T5-v1.1-large, Pre-training=Large-scale structure-text data2022.03 | 0.9299 | — | — | — | — | — | — | |
| InstructUIESetting=Instruction-tuning2023.10 | 0.9294 | — | — | — | — | — | — | |
| BERT-LargeSetting=Discriminant baselines2023.10 | 0.928 | — | — | — | — | — | — | |
| RANSetting=Discriminant baselines2023.10 | 0.9268 | — | — | — | — | — | — | |
| RoBERTa-LargeSetting=Discriminant baselines2023.10 | 0.9259 | — | — | — | — | — | — | |
| LUKEEvaluation Setting=W/O CONTEXT2021.05 | 0.9242 | — | — | — | — | — | — | |
| BERT-BaseSetting=Discriminant baselines2023.10 | 0.924 | — | — | — | — | — | — | |
| SELBackbone=T5-v1.1-large, Pre-training=None2022.03 | 0.9217 | — | — | — | — | — | — | |
| RoBERTa-BaseSetting=Discriminant baselines2023.10 | 0.9213 | — | — | — | — | — | — | |
| TANLTraining Setting=multi-dataset2021.01 | 0.92 | — | — | — | — | — | — | |
| TANLTraining Setting=standard2021.01 | 0.917 | — | — | — | — | — | — | |
| TANLTraining Setting=multi-task2021.01 | 0.917 | — | — | — | — | — | — | |
| TANL2022.05 | 0.917 | — | — | — | — | — | — | |
| LS-unLLaMA-2-13BModel Variant=LS-unLLaMA, Model Scale=13B2023.10 | 0.9146 | — | — | — | — | — | — | |
| Regularized BNNorm Position=Post-Norm2022.10 | 0.914 | — | — | — | — | — | — | |
| LayerNormNorm Position=Post-Norm2022.10 | 0.913 | — | — | — | — | — | — | |
| LayerNormNorm Position=Pre-Norm2022.10 | 0.91 | — | — | — | — | — | — | |
| BatchNormNorm Position=Post-Norm2022.10 | 0.909 | — | — | — | — | — | — | |
| GSLTraining Setting=standard2021.01 | 0.907 | — | — | — | — | — | — | |
| Regularized BNNorm Position=Pre-Norm2022.10 | 0.906 | — | — | — | — | — | — | |
| BatchNormNorm Position=Pre-Norm2022.10 | 0.899 | — | — | — | — | — | — | |
| UIE-base w/o SSIdata_regime=10%2022.03 | 0.8984 | — | — | — | — | — | — | |
| UIE-basedata_regime=10%2022.03 | 0.8963 | — | — | — | — | — | — | |
| Fine-tuned T5-basedata_regime=10%2022.03 | 0.8891 | — | — | — | — | — | — | |
| UIE-base w/o SSIdata_regime=5%2022.03 | 0.8841 | — | — | — | — | — | — | |
| UIE-basedata_regime=5%2022.03 | 0.8834 | — | — | — | — | — | — | |
| T5-v1.1-basedata_regime=10%2022.03 | 0.877 | — | — | — | — | — | — | |
| Fine-tuned T5-basedata_regime=5%2022.03 | 0.8767 | — | — | — | — | — | — | |
| UIE-basedata_regime=AVE-R2022.03 | 0.8694 | — | — | — | — | — | — | |
| UIE-base w/o SSIdata_regime=AVE-R2022.03 | 0.8672 | — | — | — | — | — | — | |
| T5-v1.1-basedata_regime=5%2022.03 | 0.8571 | — | — | — | — | — | — | |
| Fine-tuned T5-basedata_regime=AVE-R2022.03 | 0.8503 | — | — | — | — | — | — | |
| T5-v1.1-basedata_regime=AVE-R2022.03 | 0.8305 | — | — | — | — | — | — | |
| UIE-basedata_regime=1%2022.03 | 0.8284 | — | — | — | — | — | — | |
| UIE-base w/o SSIdata_regime=1%2022.03 | 0.8191 | — | — | — | — | — | — | |
| Rainbow CuckooShots=4, Evaluation Protocol=In-domain Transfer2025.02 | 0.7994 | — | — | — | — | — | — | |
| Fine-tuned T5-basedata_regime=1%2022.03 | 0.7851 | — | — | — | — | — | — | |
| T5-v1.1-basedata_regime=1%2022.03 | 0.7574 | — | — | — | — | — | — | |
| LS-LLaMA-2-7BModel Variant=LS-LLaMA, Model Scale=7B2023.10 | 0.7476 | — | — | — | — | — | — | |
| NuNER+Shots=4, Evaluation Protocol=Few-shot2025.02 | 0.7415 | — | — | — | — | — | — | |
| LS-LLaMA-2-13BModel Variant=LS-LLaMA, Model Scale=13B2023.10 | 0.7412 | — | — | — | — | — | — | |
| UIE-basedata_regime=10-Shot2022.03 | 0.739 | — | — | — | — | — | — | |
| CuckooShots=4, Evaluation Protocol=Few-shot2025.02 | 0.736 | — | — | — | — | — | — | |
| BCLModel=Qwen-2.5-7b, Token Cost=5012026.06 | 0.7283 | — | — | — | — | — | — | |
| Cuckoo (Only Post-train)Shots=4, Evaluation Protocol=Few-shot2025.02 | 0.728 | — | — | — | — | — | — | |
| UIE-base w/o SSIdata_regime=10-Shot2022.03 | 0.7247 | — | — | — | — | — | — | |
| Cuckoo (Only Pre-train)Shots=4, Evaluation Protocol=Few-shot2025.02 | 0.7246 | — | — | — | — | — | — | |
| MRQAShots=4, Evaluation Protocol=Few-shot2025.02 | 0.7245 | — | — | — | — | — | — | |
| MetaIE+Shots=4, Evaluation Protocol=Few-shot2025.02 | 0.7133 | — | — | — | — | — | — | |
| BCLModel=Llama-3.1-8b, Token Cost=5012026.06 | 0.6914 | — | — | — | — | — | — | |
| ChatGPTSetting=Zero- and few-shot2023.10 | 0.672 | — | — | — | — | — | — | |
| UIE-basedata_regime=5-Shot2022.03 | 0.6709 | — | — | — | — | — | — | |
| MultiNERD+Shots=4, Evaluation Protocol=Few-shot2025.02 | 0.6678 | — | — | — | — | — | — | |
| One-shotModel=Llama-3.1-8b, Token Cost=3852026.06 | 0.6578 | — | — | — | — | — | — | |
| BCLModel=Pixtral-12B, Token Cost=5012026.06 | 0.6554 | — | — | — | — | — | — | |
| Fine-tuned T5-basedata_regime=10-Shot2022.03 | 0.6531 | — | — | — | — | — | — | |
| BCLModel=Qwen-2.5-3b, Token Cost=5012026.06 | 0.6512 | — | — | — | — | — | — | |
| GuideNERModel=Qwen-2.5-7b, Token Cost=5062026.06 | 0.651 | — | — | — | — | — | — | |
| UIE-base w/o SSIdata_regime=5-Shot2022.03 | 0.6476 | — | — | — | — | — | — | |
| GuideNERModel=Pixtral-12B, Token Cost=5062026.06 | 0.6476 | — | — | — | — | — | — | |
| GuideNERModel=Qwen-2.5-3b, Token Cost=5062026.06 | 0.6332 | — | — | — | — | — | — | |
| One-shotModel=Qwen-2.5-7b, Token Cost=3852026.06 | 0.6282 | — | — | — | — | — | — | |
| UIE-basedata_regime=AVE-S2022.03 | 0.6247 | — | — | — | — | — | — | |
| GuideNERModel=Llama-3.1-8b, Token Cost=5062026.06 | 0.6138 | — | — | — | — | — | — | |
| ChatIEModel=Pixtral-12B, Token Cost=9422026.06 | 0.6085 | — | — | — | — | — | — | |
| One-shotModel=Qwen-2.5-3b, Token Cost=3852026.06 | 0.6055 | — | — | — | — | — | — | |
| UIE-base w/o SSIdata_regime=AVE-S2022.03 | 0.6025 | — | — | — | — | — | — | |
| One-shotModel=Pixtral-12B, Token Cost=3852026.06 | 0.6025 | — | — | — | — | — | — | |
| CodeIEModel=Qwen-2.5-7b, Token Cost=11722026.06 | 0.6 | — | — | — | — | — | — | |
| T5-v1.1-basedata_regime=10-Shot2022.03 | 0.5889 | — | — | — | — | — | — | |
| ChatIEModel=Llama-3.1-8b, Token Cost=9422026.06 | 0.5512 | — | — | — | — | — | — | |
| Fine-tuned T5-basedata_regime=5-Shot2022.03 | 0.5485 | — | — | — | — | — | — | |
| CodeIEModel=Pixtral-12B, Token Cost=11722026.06 | 0.5339 | — | — | — | — | — | — | |
| VanillaBackbone=Qwen2.5-7B-Instruct2026.05 | 0.51 | — | — | — | — | — | — | |
| EVA (text)Backbone=Qwen2.5-7B-Instruct2026.05 | 0.51 | — | — | — | — | — | — | |
| SafeDecodingBackbone=Qwen2.5-7B-Instruct2026.05 | 0.509 | — | — | — | — | — | — | |
| LoRABackbone=Qwen2.5-7B-Instruct2026.05 | 0.507 | — | — | — | — | — | — | |
| Circuit BreakersBackbone=Qwen2.5-7B-Instruct2026.05 | 0.504 | — | — | — | — | — | — | |
| OPT-C4-TuluV3Shots=4, Evaluation Protocol=Few-shot2025.02 | 0.5024 | — | — | — | — | — | — | |
| LEDBackbone=Qwen2.5-7B-Instruct2026.05 | 0.499 | — | — | — | — | — | — |