MultiNLI
Benchmarks
TaskDatasetSOTA resultTrendResults
MultiNLI (test)
88.05Average Worst-Group Accuracy
81
MultiNLI matched (test)
85.38Accuracy
65
MultiNLI Mismatched
79.1Accuracy
60
MultiNLI mismatched (test)
81.4Accuracy
56
MultiNLI Matched
80.2Accuracy
49
MultiNLI mismatched (cross-domain) RepEval 2017 (test)
75.8Accuracy
25
MultiNLI
82.4Accuracy
23
MultiNLI matched (dev)
88.4Accuracy
23
MultiNLI (test)
81.3WGA
18
MultiNLI matched (in-domain) RepEval 2017 (test)
76.8Accuracy
18
MultiNLI Mismatch (test)
0.007ECE
16
MultiNLI (Match)
1.02ECE
16
MultiNLI mismatched (dev)
88.4Accuracy
11
MultiNLI matched/mismatched
92.6Accuracy
10
MultiNLI
23R@1
9
MultiNLI
3.43Wasserstein Distance
9
MultiNLI matched (in-domain)
74.6Accuracy
8
MultiNLI matched (val)
91.7Accuracy
8
MultiNLI Noise (test)
87.57Accuracy
7
MultiNLI Emoji (test)
89.8Accuracy
7
MultiNLI Slang (test)
90.06Accuracy
7
MultiNLI
81.1Average Accuracy
7
MultiNLI WILDS (test)
82.1IID Accuracy
6
MultiNLI reconstructed with controlled shortcut injection (test)
0.797MSTPS
5
MultiNLI controlled shortcut injection
32.3Accuracy
5