Semantic Textual Similarity on STS 2014
66.93Spearman CorrelationMIC
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MICBackbone=BERT, Dimension=7682026.05 | 66.93 | — | — | — | — | |
| MICBackbone=TinyBERT 6L, Dimension=7682026.05 | 63.84 | — | — | — | — | |
| MICBackbone=BERT, Dimension=162026.05 | 56.08 | — | — | — | — | |
| MICBackbone=TinyBERT 6L, Dimension=162026.05 | 55.61 | — | — | — | — | |
| BERT-NLI-STSb-largeTraining=Trained on NLI data + STS benchmark data, Size=large2019.08 | 0.8877 | — | — | — | — | |
| BERT-NLI-STSb-baseTraining=Trained on NLI data + STS benchmark data, Size=base2019.08 | 0.8833 | — | — | — | — | |
| SROBERTa-NLI-STSb-largeTraining=Trained on NLI data + STS benchmark data, Size=large2019.08 | 0.8615 | — | — | — | — | |
| SBERT-NLI-STSb-largeTraining=Trained on NLI data + STS benchmark data, Size=large2019.08 | 0.861 | — | — | — | — | |
| BERT-STSb-largeTraining=Trained on STS benchmark dataset, Size=large2019.08 | 0.8564 | — | — | — | — | |
| SBERT-NLI-STSb-baseTraining=Trained on NLI data + STS benchmark data, Size=base2019.08 | 0.8535 | — | — | — | — | |
| SROBERTa-STSb-largeTraining=Trained on STS benchmark dataset, Size=large2019.08 | 0.8502 | — | — | — | — | |
| SROBERTa-STSb-baseTraining=Trained on STS benchmark dataset, Size=base2019.08 | 0.8492 | — | — | — | — | |
| SROBERTa-NLI-STSb-baseTraining=Trained on NLI data + STS benchmark data, Size=base2019.08 | 0.8479 | — | — | — | — | |
| SBERT-STSb-baseTraining=Trained on STS benchmark dataset, Size=base2019.08 | 0.8467 | — | — | — | — | |
| SBERT-STSb-largeTraining=Trained on STS benchmark dataset, Size=large2019.08 | 0.8445 | — | — | — | — | |
| BERT-STSb-baseTraining=Trained on STS benchmark dataset, Size=base2019.08 | 0.843 | — | — | — | — | |
| SBERT-NLI-largeTraining=Pretrained on NLI datasets, Size=large2019.08 | 0.7923 | — | — | — | — | |
| SBERT-NLI-baseTraining=Pretrained on NLI datasets, Size=base2019.08 | 0.7703 | — | — | — | — | |
| Universal Sentence EncoderTraining=Not trained for STS2019.08 | 0.7492 | — | — | — | — | |
| BERTlarge-flow (target)Backbone=BERT-large, Pooling Strategy=Last 2 layers average, Flow Training Source=Target dataset2020.11 | 0.6942 | — | — | — | — | |
| BERTbase-flow (target)Backbone=BERT-base, Pooling Strategy=Last 2 layers average, Flow Training Source=Target dataset2020.11 | 0.6842 | — | — | — | — | |
| InferSent - GloVeTraining=Not trained for STS2019.08 | 0.6803 | — | — | — | — | |
| HBMPEmbedding Dimension=1200D, Evaluation Protocol=SentEval2018.08 | 0.68 | 0.71 | — | — | — | |
| BiLSTM-Max (on AllNLI)Training Methodology=Supervised, Source Dataset=AllNLI2017.05 | 0.67 | 0.7 | — | — | — | |
| InferSentEvaluation Protocol=SentEval2018.08 | 0.67 | 0.7 | — | — | — | |
| BERTlarge-flow (NLI*)Backbone=BERT-large, Pooling Strategy=Last 2 layers average, Flow Training Source=NLI corpus2020.11 | 0.6634 | — | — | — | — | |
| HBMPEmbedding Dimension=600D, Evaluation Protocol=SentEval2018.08 | 0.66 | 0.7 | — | — | — | |
| BERTbase-flow (NLI*)Backbone=BERT-base, Pooling Strategy=Last 2 layers average, Flow Training Source=NLI corpus2020.11 | 0.6466 | — | — | — | — | |
| BERTbase-last2avgBackbone=BERT-base, Pooling Strategy=Last 2 layers average2020.11 | 0.6248 | — | — | — | — | |
| BERTlarge-last2avgBackbone=BERT-large, Pooling Strategy=Last 2 layers average2020.11 | 0.6102 | — | — | — | — | |
| Avg. GloVe embeddingsBackbone=GloVe, Pooling Strategy=Average2020.11 | 0.5973 | — | — | — | — | |
| Avg. GloVe embeddingsTraining=Not trained for STS2019.08 | 0.5802 | — | — | — | — | |
| Avg. BERT embeddingsBackbone=BERT, Pooling Strategy=Average2020.11 | 0.5798 | — | — | — | — | |
| Unigram-TFIDFTraining Methodology=Unsupervised2017.05 | 0.57 | 0.58 | — | — | — | |
| BERTbaseBackbone=BERT-base, Pooling Strategy=Last layer average2020.11 | 0.5475 | — | — | — | — | |
| BERTlargeBackbone=BERT-large, Pooling Strategy=Last layer average2020.11 | 0.4927 | — | — | — | — | |
| Avg. BERT embeddingsTraining=Not trained for STS2019.08 | 0.4635 | — | — | — | — | |
| SkipThoughtEvaluation Protocol=SentEval2018.08 | 0.45 | 0.44 | — | — | — | |
| BERT CLS-vectorBackbone=BERT, Pooling Strategy=[CLS]2020.11 | 0.2009 | — | — | — | — | |
| BERTBackbone=BERT-base, Evaluation Protocol=Fine-tuned2021.06 | — | 0.836 | — | — | — | |
| BERT + AlignerBackbone=BERT-base, Evaluation Protocol=Fine-tuned, Alignment Feature=Conditional alignment probability2021.06 | — | 0.837 | — | — | — | |
| LapMechepsilon (ϵ)=52026.02 | — | — | 1.03 | 20.42 | 39.76 | |
| LapMechepsilon (ϵ)=102026.02 | — | — | 4.04 | 21.86 | 70.28 | |
| LapMechepsilon (ϵ)=202026.02 | — | — | 8.64 | 28.05 | 79.16 | |
| LapMechepsilon (ϵ)=302026.02 | — | — | 11.22 | 30.38 | 79.47 | |
| LapMechepsilon (ϵ)=402026.02 | — | — | 13.67 | 32.09 | 79.43 | |
| Non-protectedepsilon (ϵ)=infinity2026.02 | — | — | 21.97 | 35.99 | 79.25 | |
| PurMechepsilon (ϵ)=52026.02 | — | — | 1.55 | 20.88 | 39.71 | |
| PurMechepsilon (ϵ)=102026.02 | — | — | 4.13 | 21.84 | 70.25 | |
| PurMechepsilon (ϵ)=202026.02 | — | — | 8.77 | 27.73 | 79.16 | |
| PurMechepsilon (ϵ)=302026.02 | — | — | 11.26 | 30.39 | 79.47 | |
| PurMechepsilon (ϵ)=402026.02 | — | — | 13.5 | 32.05 | 79.43 | |
| SIF (GloVe + WR)Training Methodology=Unsupervised2017.05 | — | 0.69 | — | — | — | |
| SPARSEepsilon (ϵ)=52026.02 | — | — | 0.3 | 18.05 | 48.47 | |
| SPARSEepsilon (ϵ)=102026.02 | — | — | 2.41 | 21.1 | 74.44 | |
| SPARSEepsilon (ϵ)=202026.02 | — | — | 9.46 | 28.56 | 79.31 | |
| SPARSEepsilon (ϵ)=302026.02 | — | — | 14.7 | 32.95 | 79.37 | |
| SPARSEepsilon (ϵ)=402026.02 | — | — | 16.12 | 34.81 | 79.32 |