Semantic Textual Similarity on STS-12
0.7154Spearman Correlation (rho)S-ROBERTa (base)
Evaluation results
| Method | Links | |
|---|---|---|
| S-ROBERTa (base)supervision=supervised2021.04 | 0.7154 | |
| S-BERT (base)supervision=supervised2021.04 | 0.7097 | |
| DINO (STS-A-x2)supervision=unsupervised, unlabeled_data=STS2021.04 | 0.7027 | |
| MIPICBackbone=BERT, Dimension=7682026.04 | 0.6764 | |
| MIPICBackbone=BERT, Dimension=5122026.04 | 0.6755 | |
| MICBackbone=BERT, Dimension=7682026.05 | 0.6754 | |
| MIPICBackbone=BERT, Dimension=2562026.04 | 0.6708 | |
| MIPICBackbone=BERT, Dimension=1282026.04 | 0.6632 | |
| Unsup SimCSEBackbone=BERT, Dimension=7682026.04 | 0.6631 | |
| Unsup SimCSEBackbone=BERT, Dimension=5122026.04 | 0.6616 | |
| MIPICBackbone=BERT, Dimension=642026.04 | 0.6588 | |
| Unsup SimCSEBackbone=BERT, Dimension=2562026.04 | 0.6583 | |
| MICBackbone=TinyBERT 6L, Dimension=7682026.05 | 0.6573 | |
| Unsup SimCSEBackbone=BERT, Dimension=1282026.04 | 0.6545 | |
| BERTlarge-flow (target)Backbone=BERT-large, Pooling Strategy=Last 2 layers average, Flow Training Source=Target dataset2020.11 | 0.652 | |
| MRLBackbone=BERT, Dimension=5122026.04 | 0.6509 | |
| DINO (STS-A-x1x2)supervision=unsupervised2021.04 | 0.6487 | |
| MIPICBackbone=BERT, Dimension=322026.04 | 0.6487 | |
| MRLBackbone=BERT, Dimension=7682026.04 | 0.6484 | |
| MIPICBackbone=TinyBERT 6L, Dimension=7682026.04 | 0.6457 | |
| USEsupervision=supervised2021.04 | 0.6449 | |
| ESEBackbone=TinyBERT 6L, Dimension=7682026.04 | 0.6416 | |
| MIPICBackbone=TinyBERT 6L, Dimension=5122026.04 | 0.6414 | |
| MRLBackbone=BERT, Dimension=2562026.04 | 0.6413 | |
| MIPICBackbone=TinyBERT 6L, Dimension=2562026.04 | 0.6382 | |
| MICBackbone=BERT, Dimension=322026.05 | 0.6376 | |
| MIPICBackbone=TinyBERT 6L, Dimension=1282026.04 | 0.6374 | |
| BERTbase-flow (target)Backbone=BERT-base, Pooling Strategy=Last 2 layers average, Flow Training Source=Target dataset2020.11 | 0.6348 | |
| Li et al. (2020)supervision=unsupervised, unlabeled_data=STS2021.04 | 0.6348 | |
| MIPICBackbone=TinyBERT 6L, Dimension=642026.04 | 0.627 | |
| MRLBackbone=BERT, Dimension=1282026.04 | 0.6234 | |
| Unsup SimCSEBackbone=TinyBERT 6L, Dimension=7682026.04 | 0.6219 | |
| MIPICBackbone=BERT, Dimension=162026.04 | 0.6208 | |
| MRLBackbone=TinyBERT 6L, Dimension=7682026.04 | 0.6178 | |
| Unsup SimCSEBackbone=TinyBERT 6L, Dimension=5122026.04 | 0.6177 | |
| BERTlarge-flow (NLI*)Backbone=BERT-large, Pooling Strategy=Last 2 layers average, Flow Training Source=NLI corpus2020.11 | 0.6172 | |
| MICBackbone=TinyBERT 6L, Dimension=322026.05 | 0.6164 | |
| MIPICBackbone=TinyBERT 6L, Dimension=322026.04 | 0.616 | |
| MRLBackbone=TinyBERT 6L, Dimension=5122026.04 | 0.6152 | |
| MRLBackbone=BERT, Dimension=642026.04 | 0.6124 | |
| ESEBackbone=TinyBERT 6L, Dimension=2562026.04 | 0.6123 | |
| Unsup SimCSEBackbone=BERT, Dimension=642026.04 | 0.6122 | |
| Unsup SimCSEBackbone=TinyBERT 6L, Dimension=2562026.04 | 0.6114 | |
| ESEBackbone=BERT, Dimension=5122026.04 | 0.6106 | |
| MICBackbone=BERT, Dimension=162026.05 | 0.6086 | |
| ESEBackbone=TinyBERT 6L, Dimension=5122026.04 | 0.6082 | |
| ESEBackbone=BERT, Dimension=7682026.04 | 0.6043 | |
| ESEBackbone=TinyBERT 6L, Dimension=1282026.04 | 0.6038 | |
| ESEBackbone=BERT, Dimension=2562026.04 | 0.6033 | |
| MRLBackbone=TinyBERT 6L, Dimension=2562026.04 | 0.6017 | |
| ESEBackbone=BERT, Dimension=1282026.04 | 0.5997 | |
| MRLBackbone=TinyBERT 6L, Dimension=1282026.04 | 0.598 | |
| MRLBackbone=BERT, Dimension=322026.04 | 0.5978 | |
| BERTbase-flow (NLI*)Backbone=BERT-base, Pooling Strategy=Last 2 layers average, Flow Training Source=NLI corpus2020.11 | 0.5954 | |
| Li et al. (2020)supervision=unsupervised, unlabeled_data=NLI2021.04 | 0.5954 | |
| Unsup SimCSEBackbone=TinyBERT 6L, Dimension=1282026.04 | 0.595 | |
| MIPICBackbone=TinyBERT 6L, Dimension=162026.04 | 0.5934 | |
| ESEBackbone=TinyBERT 6L, Dimension=642026.04 | 0.5832 | |
| BERTbase-last2avgBackbone=BERT-base, Pooling Strategy=Last 2 layers average2020.11 | 0.5784 | |
| BERTlarge-last2avgBackbone=BERT-large, Pooling Strategy=Last 2 layers average2020.11 | 0.5768 | |
| MICBackbone=TinyBERT 6L, Dimension=162026.05 | 0.57 | |
| Zhang et al. (2020)supervision=unsupervised, unlabeled_data=NLI2021.04 | 0.5677 | |
| ESEBackbone=TinyBERT 6L, Dimension=322026.04 | 0.5602 | |
| MRLBackbone=TinyBERT 6L, Dimension=642026.04 | 0.5567 | |
| MRLBackbone=TinyBERT 6L, Dimension=322026.04 | 0.5525 | |
| ESEBackbone=BERT, Dimension=642026.04 | 0.5523 | |
| Avg. GloVe embeddingsBackbone=GloVe, Pooling Strategy=Average2020.11 | 0.5514 | |
| Avg. GloVesupervision=unsupervised2021.04 | 0.5514 | |
| MRLBackbone=BERT, Dimension=162026.04 | 0.5513 | |
| MRLBackbone=BERT, Dimension=162026.05 | 0.5513 | |
| Unsup SimCSEBackbone=BERT, Dimension=322026.04 | 0.5384 | |
| ESEBackbone=BERT, Dimension=322026.04 | 0.5378 | |
| Unsup SimCSEBackbone=TinyBERT 6L, Dimension=642026.04 | 0.5348 | |
| InferSent, Glovesupervision=supervised2021.04 | 0.5286 | |
| MRLBackbone=TinyBERT 6L, Dimension=162026.04 | 0.5148 | |
| MRLBackbone=TinyBERT 6L, Dimension=162026.05 | 0.5148 | |
| ESEBackbone=BERT, Dimension=162026.04 | 0.5134 | |
| ESEBackbone=BERT, Dimension=162026.05 | 0.5134 | |
| ESEBackbone=TinyBERT 6L, Dimension=162026.04 | 0.509 | |
| ESEBackbone=TinyBERT 6L, Dimension=162026.05 | 0.509 | |
| BERTbaseBackbone=BERT-base, Pooling Strategy=Last layer average2020.11 | 0.4907 | |
| Unsup SimCSEBackbone=BERT, Dimension=162026.04 | 0.4788 | |
| Unsup SimCSEBackbone=BERT, Dimension=162026.05 | 0.4788 | |
| Unsup SimCSEBackbone=TinyBERT 6L, Dimension=322026.04 | 0.4709 | |
| BERTlargeBackbone=BERT-large, Pooling Strategy=Last layer average2020.11 | 0.4689 | |
| Unsup SimCSEBackbone=TinyBERT 6L, Dimension=162026.04 | 0.4568 | |
| Unsup SimCSEBackbone=TinyBERT 6L, Dimension=162026.05 | 0.4568 | |
| Avg. BERT embeddingsBackbone=BERT, Pooling Strategy=Average2020.11 | 0.3878 | |
| Avg. BERTsupervision=unsupervised2021.04 | 0.3878 | |
| BERT CLS-vectorBackbone=BERT, Pooling Strategy=[CLS]2020.11 | 0.2016 | |
| BERT CLSsupervision=unsupervised2021.04 | 0.2016 |