Named Entity Recognition on CrossNER AI
64.5F1 ScoreToMMeR
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ToMMeRDecoding Strategy=Greedy (flat) decoding, Zero-shot=true, Nested=false, #samples=879, #entities=3,7762025.10 | 64.5 | — | 87.2 | 51.2 | |
| gliner_large-v2.5Evaluation Protocol=Zero-shot2026.02 | 54.9 | — | — | — | |
| Qwen3-32Bcheckpoint=open-source base, fine-tuning=none, parameters=32B2026.06 | 54.89 | — | — | — | |
| gliner_medium-v2.5Evaluation Protocol=Zero-shot2026.02 | 54.7 | — | — | — | |
| gliner_small-v2.5Evaluation Protocol=Zero-shot2026.02 | 54.2 | — | — | — | |
| Agents-K1 (4B)training=GRPO on IEPile, parameters=4B2026.06 | 54 | — | — | — | |
| ToMMeRDecoding Strategy=Threshold decoding, Zero-shot=true, Nested=false, #samples=879, #entities=3,7762025.10 | 51.5 | — | 97 | 35 | |
| Qwen3-8Bcheckpoint=open-source base, fine-tuning=none, parameters=8B2026.06 | 49.33 | — | — | — | |
| Qwen3-4Bcheckpoint=open-source base, fine-tuning=none, parameters=4B2026.06 | 48.62 | — | — | — | |
| DADGMode=Zero-shot, Num. of Samples=3000, Cost (USD)=13.61, Time (Mins)=40†2022.12 | 47.22 | — | — | — | |
| DADGMode=Zero-shot, Num. of Samples=1500, Cost (USD)=6.77, Time (Mins)=20†2022.12 | 46.9 | — | — | — | |
| PGINum. of Samples=431, Cost (USD)=63.23, Time (Mins)=20†2022.12 | 46.65 | — | — | — | |
| PGDGMode=Zero-shot, Num. of Samples=1500, Cost (USD)=7.78, Time (Mins)=17†2022.12 | 42.63 | — | — | — | |
| Human LabeledNum. of Samples=100, Cost (USD)=17 - 42.85, Time (Mins)=652022.12 | 42 | — | — | — | |
| PGDGMode=Zero-shot, Num. of Samples=3000, Cost (USD)=13.56, Time (Mins)=33†2022.12 | 41.35 | — | — | — | |
| PGDAMode=10-shot, Num. of Samples=100, Cost (USD)=15.39, Time (Mins)=212022.12 | 23.08 | — | — | — | |
| chatgptSetting=zero-shot2023.04 | — | 54.4 | — | — | |
| davinciSetting=zero-shot2023.04 | — | 2.97 | — | — | |
| InstructUIESetting=Zero-shot2023.04 | — | 49 | — | — | |
| USMSetting=Zero-shot2023.04 | — | 28.18 | — | — |