Knowledge Probing on mLAMA
43.7Accuracy (DE)mLUKE-E_base
Evaluation results
| Method | Links | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| mLUKE-E_basecandidate_prompt=[X] & [Y]2021.10 | 43.7 | 43.2 | — | 34.2 | 35.3 | — | — | — | 39.3 | 37.3 | 32.3 | 34.4 | 36.4 | 47.6 | 37.7 | 41.6 | 37.7 | 44.8 | 31.4 | 50.1 | 41.6 | |
| mLUKE-E_basecandidate_prompt=[Y]2021.10 | 32.9 | 34.9 | — | 27.7 | 24.7 | — | — | — | 28.9 | 25.6 | 21.6 | 25.2 | 28.5 | 35.3 | 27.2 | 26.3 | 25.7 | 34.7 | 23.8 | 39.1 | 29.5 | |
| XLM-R_basetraining=extra training2021.10 | 29.3 | 31.6 | — | 25 | 19.2 | — | — | — | 24.7 | 20.7 | 14 | 18.2 | 26.4 | 38.2 | 19.1 | 21.4 | 20.5 | 29.6 | 20.6 | 33.8 | 28.1 | |
| mBERT2021.10 | 28.6 | 34.8 | — | 27.1 | 19.2 | — | — | — | 24.3 | 15.1 | 12.7 | 19.4 | 30.2 | 37.4 | 14.2 | 17.8 | 21.9 | 32 | 17.4 | 36.5 | 24.2 | |
| mLUKE-W_base2021.10 | 25.7 | 27.1 | — | 23 | 15.9 | — | — | — | 22.9 | 21.3 | 12.9 | 17.5 | 23.3 | 36.6 | 18 | 17.9 | 20.2 | 29.4 | 19.6 | 31 | 26.9 | |
| Prix-LM2021.10 | 20.7 | 23.8 | 21.8 | 17.8 | 16.1 | 7.4 | 23.9 | 13.1 | 18.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| XLM-R_base2021.10 | 18.4 | 24.2 | — | 16.1 | 14.5 | — | — | — | 16.5 | 14.9 | 7.5 | 12.7 | 18.5 | 24.6 | 11.4 | 10.9 | 16.6 | 22.2 | 12.6 | 23 | 15.5 | |
| XLM-R2021.10 | 13.9 | 21 | 19.3 | 7.6 | 5.6 | 6.1 | 20.5 | 6.1 | 12.5 | — | — | — | — | — | — | — | — | — | — | — | — |