Natural Language Understanding on Russian SuperGLUE (test)
62.6LiDiRus (Corr)Human Performance
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Human Performancetype=human performance2021.05 | 62.6 | 68 | 70.2 | 98.2 | 80.6 | 42 | 92 | 80.5 | 84 | 91.5 | 93 | 89 | 81.1 | |
| SOTAtype=state-of-the-art2021.05 | 23.1 | 45.2 | 54.6 | 90.8 | 94.1 | 81.9 | 87.1 | 72.9 | 66.9 | 91.7 | 92 | 92.4 | 67.9 | |
| H r.(b)approach=heuristics-based, combined_with=random balanced choice baseline2021.05 | 18.2 | 40.1 | 43.8 | 47 | 66.9 | 20.2 | 54.8 | 54.3 | 60.4 | 62.9 | 26 | 25.7 | 45.4 | |
| H rand.approach=heuristics-based, combined_with=random choice baseline2021.05 | 14.9 | 40.1 | 43.6 | 50.8 | 66.9 | 19.5 | 54.7 | 49.7 | 56.5 | 62.9 | 26 | 25.7 | 44.5 | |
| H maj.approach=heuristics-based, combined_with=majority class baseline2021.05 | 14.7 | 40 | 43.8 | 47.8 | 67.1 | 23.7 | 54.9 | 59.5 | 66.9 | 64.2 | 26 | 25.7 | 46.8 | |
| Random Choicebaseline_type=random choice2021.05 | 2.4 | 33.2 | 34.7 | 47.4 | 47.7 | 7.8 | 50.3 | 50.1 | 48.7 | 49.4 | 25 | 24.7 | 37.2 | |
| Majority Classbaseline_type=majority class2021.05 | 0 | 21.7 | 48.4 | 49.8 | 0 | 0 | 51.3 | 58.7 | 66.9 | 50.3 | 25 | 24.7 | 37.4 | |
| Random Balanced Choicebaseline_type=random balanced choice2021.05 | 0 | 31.9 | 37.4 | 48 | 45 | 7.1 | 48.3 | 52.8 | 59.7 | 52 | 25 | 24.7 | 38.5 |