Named Entity Recognition on MasakhaNER (test)
71.7F1 ScoreLTSFT
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LTSFTmode=upper-bound2023.06 | 71.7 | — | 83.5 | 76.7 | 67.4 | 67.9 | 54.7 | 74.6 | 79.4 | 66.3 | 74.8 | |
| BISTIL-TFLRF=2, Type=Teacher-Free2023.06 | 71.3 | -0.4 | 81 | 74.8 | 67.5 | 67.3 | 55 | 72.9 | 78.4 | 69 | 75.7 | |
| BISTIL-STLRF=2, Type=Student-Teacher2023.06 | 69.8 | -1.9 | 81.3 | 74.1 | 65.9 | 66.7 | 53.5 | 72.1 | 77.1 | 64.6 | 72.8 | |
| BISTIL-TFLRF=3, Type=Teacher-Free2023.06 | 69.6 | -2.1 | 79.6 | 74.8 | 64.6 | 64.5 | 56.7 | 70.6 | 77.2 | 66.1 | 72.8 | |
| BISTIL-STLRF=3, Type=Student-Teacher2023.06 | 69.1 | -2.6 | 80.3 | 74 | 63.1 | 64.6 | 54.7 | 69.6 | 76.9 | 68 | 70.5 | |
| DISTILMBERTmodel_type=distilled2023.06 | 68.4 | -3.3 | 81.1 | 73.2 | 65.3 | 63.4 | 50 | 69.2 | 77.7 | 64.4 | 71.2 | |
| BothSetting=Few-Text, Lexicon=PanLex+Induced2022.03 | 61.1 | 1.9 | — | — | — | — | — | — | — | — | — | |
| Pseudo Trans-trainSetting=Few-Text, Lexicon=PanLex2022.03 | 59.4 | 5.8 | — | — | — | — | — | — | — | — | — | |
| Both + Label DistillationSetting=Few-Text, Lexicon=PanLex+Induced2022.03 | 59.4 | 1 | — | — | — | — | — | — | — | — | — | |
| BothSetting=Few-Text, Lexicon=PanLex2022.03 | 59.2 | 5.6 | — | — | — | — | — | — | — | — | — | |
| Pseudo MLMSetting=Few-Text, Lexicon=PanLex+Induced2022.03 | 58.5 | 0.9 | — | — | — | — | — | — | — | — | — | |
| Both + Label DistillationSetting=Few-Text, Lexicon=PanLex2022.03 | 58.4 | -0.8 | — | — | — | — | — | — | — | — | — | |
| Pseudo MLMSetting=Few-Text, Lexicon=PanLex2022.03 | 57.4 | 3.8 | — | — | — | — | — | — | — | — | — | |
| BothSetting=No-Text, Lexicon=PanLex2022.03 | 54.6 | 8.5 | — | — | — | — | — | — | — | — | — | |
| Pseudo Trans-trainSetting=No-Text, Lexicon=PanLex2022.03 | 54.4 | 8.3 | — | — | — | — | — | — | — | — | — | |
| Gold MLMSetting=Few-Text, Lexicon=None2022.03 | 53.6 | — | — | — | — | — | — | — | — | — | — | |
| Both + Label DistillationSetting=No-Text, Lexicon=PanLex2022.03 | 53.5 | -1.1 | — | — | — | — | — | — | — | — | — | |
| Pseudo MLMSetting=No-Text, Lexicon=PanLex2022.03 | 52.6 | 6.5 | — | — | — | — | — | — | — | — | — | |
| mBERTSetting=Baseline, Lexicon=None2022.03 | 46.1 | — | — | — | — | — | — | — | — | — | — |