STS-B
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
STS-B
92.9Spearman's Rho (x100)
158
STS-B (test)
80.22Semantic Consistency
18
STS-B-DIR Few-shot (test)
0.781MSE
14
STS-B-DIR Medium-shot (test)
0.899MSE
14
STS-B-DIR Many-shot (test)
0.795MSE
14
STS-B-DIR All (test)
0.892MSE
14
STS-B-DIR Few-shot
0.781MSE
14
STS-B-DIR Medium-shot
0.899MSE
14
STS-B-DIR Many-shot
0.795MSE
14
STS-B DIR (All)
0.892MSE
14
STS-B (test)
95.94PICP
12
STS-B
0.595Accuracy
10
STS-B
61.08CCC
9
Multilingual STS-B (val)
77.48Spearman Correlation
8
STS-B (test)
76.5Kendall's Tau (STS-B)
6
STS-B (dev)
0.918Pearson Correlation
6
STS-B DIR (test)
0.877MSE (All)
6
STS-B DIR Few
2.152NLL
5
STS-B-DIR Medium
2.754NLL
5
STS-B-DIR Many
1.81NLL
5
STS-B DIR (All)
1.996NLL
5
STS-B (test)
88.94Spearman Corr (%)
5
STS-B
22.63Latency (ms)
4
STS-B (test)
89.52Accuracy
3
STS-B
54.53StereoSet Score
3