Named Entity Recognition on PAN-X
58.4Macro Avg ScoreGemma3-27B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Gemma3-27B2026.01 | 58.4 | — | — | — | |
| Qwen3-32B2026.01 | 55.4 | — | — | — | |
| OTTERArchitecture=Cross-Encoder, Backbone=mmBERT, Training Steps=100k, Thresholding Strategy=Best per language2026.01 | 54.9 | — | — | — | |
| OTTERArchitecture=Cross-Encoder, Backbone=mmBERT, Training Steps=100k2026.01 | 53.5 | — | — | — | |
| GLiNER-multi-v2.12026.01 | 53.2 | — | — | — | |
| OTTERArchitecture=Cross-Encoder, Backbone=mmBERT2026.01 | 51.6 | — | — | — | |
| GLiNER-x-base2026.01 | 50.9 | — | — | — | |
| OTTERArchitecture=Bi-Encoder, Backbone=mmBERT2026.01 | 50.9 | — | — | — | |
| OTTERArchitecture=Bi-Encoder, Backbone=RemBERT2026.01 | 50.8 | — | — | — | |
| OTTERArchitecture=Bi-Encoder, Backbone=mmBERT, Training Steps=100k2026.01 | 50.5 | — | — | — | |
| GPT-52026.01 | 47.7 | — | — | — | |
| OTTERArchitecture=Cross-Encoder, Backbone=RemBERT2026.01 | 43.6 | — | — | — | |
| WikiNeural2026.01 | 40.3 | — | — | — | |
| SFTFine-tuning strategy=Sequential Fine-Tuning, Stage=After inception2022.09 | 0.825 | — | — | — | |
| LAFTFine-tuning strategy=Language-Agnostic Fine-Tuning, Stage=After inception2022.09 | 0.8165 | — | — | — | |
| FFTFine-tuning strategy=Full Fine-Tuning, Stage=After inception2022.09 | 0.81 | — | — | — | |
| FFTStrategy=Full Fine-Tuning2022.09 | — | 1.23 | 50 | — | |
| LAFTStrategy=Language-Aware Fine-Tuning2022.09 | — | 4.62 | 255 | — | |
| LAFT-URIELStrategy=Language-Aware Fine-Tuning with URIEL features2022.09 | — | 6.3 | 410 | — | |
| MEXMAevaluation_protocol=zero-shot, encoder_state=frozen, training_data=English2026.03 | — | — | — | 59.3 | |
| OmniSONARevaluation_protocol=zero-shot, encoder_state=frozen, training_data=English2026.03 | — | — | — | 62.3 | |
| OmniSONAR-Tokenevaluation_protocol=zero-shot, encoder_state=frozen, training_data=English2026.03 | — | — | — | 63 | |
| SFTStrategy=Sparse Fine-Tuning2022.09 | — | 1.26 | 76 | — | |
| SONARevaluation_protocol=zero-shot, encoder_state=frozen, training_data=English2026.03 | — | — | — | 58.2 | |
| XLM-Alignevaluation_protocol=zero-shot, encoder_state=frozen, training_data=English2026.03 | — | — | — | 56 | |
| XLM-Robertaevaluation_protocol=zero-shot, encoder_state=frozen, training_data=English2026.03 | — | — | — | 61.4 |