Cross-lingual Language Understanding on XTREME Leaderboard (test)
87.5Sentence-pair Classification AccFILTER
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FILTER# PT params=559M, # FT params=559M, Langs=100, Add. task data=false, Translation data=true, Fine-tuning data source=translations or additional task data2020.10 | 87.5 | 71.9 | 68.5 | 68.5 | 76 | — | |
| FILTERBackbone=XLM-R, Layers=24, Hidden states=1024, Parameters=Same as XLM-R2020.09 | 87.5 | 71.9 | — | 68.5 | 77 | 84.5 | |
| VECO# PT params=662M, # FT params=662M, Langs=50, Add. task data=false, Translation data=true, Fine-tuning data source=translations or additional task data2020.10 | 87 | 70.4 | 68 | 68 | 75.1 | — | |
| VECOStatus=Unpublished work2020.09 | 84.7 | 70.4 | — | 67.2 | 74.8 | 80.5 | |
| RemBERT# PT params=995M, # FT params=575M, Langs=110, Add. task data=false, Translation data=false, Fine-tuning data source=only English task data2020.10 | 84.2 | 73.3 | 68.6 | 68.6 | 75.4 | — | |
| STiLTs# PT params=559M, # FT params=559M, Langs=100, Add. task data=true, Translation data=false, Fine-tuning data source=translations or additional task data2020.10 | 83.9 | 69.4 | 67.2 | 67.2 | 73.5 | — | |
| X-STILTS2020.09 | 83.9 | 69.4 | — | 67.2 | 73.5 | 76.5 | |
| XLM-R# PT params=559M, # FT params=559M, Langs=100, Add. task data=false, Translation data=false, Fine-tuning data source=only English task data2020.10 | 82.8 | 69 | 62.3 | 62.3 | 71.4 | — | |
| XLM-R2020.09 | 82.8 | 69 | — | 62.3 | 68.2 | 61.6 | |
| XLM2020.09 | 75 | 65.6 | — | 43.9 | 55.8 | 44.7 | |
| MMTE2020.09 | 74.3 | 65.3 | — | 52.3 | 59.3 | 48.9 | |
| mBERT2020.09 | 73.7 | 66.3 | — | 53.8 | 59.6 | 47.7 |