Semantic Textual Similarity on STS Tasks (STS12-16, STS-B, SICK-R) (test)
82.4STS12 ScoreSimCSE-BERT
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| SimCSE-BERTzero-shot=true, category=Paragon2024.02 | 82.4 | 74.4 | 80.9 | 78.6 | 76.9 | 68.4 | 72.2 | 76.3 | — | |
| SimCSE-BERTTraining Paradigm=Contrastive-Trained Embedding Models2024.10 | 82.4 | 74.4 | 80.9 | 78.6 | 76.9 | 68.4 | 72.2 | 76.3 | — | |
| PromptEOL+CSEBackbone=OPT, Params=13B, Fine-tuning=Supervised NLI datasets2023.07 | 80.2 | 90.24 | 85.34 | 89.52 | 85.9 | 88.56 | 82.06 | 85.97 | — | |
| PromptEOL+CSEBackbone=OPT, Params=6.7B, Fine-tuning=Supervised NLI datasets2023.07 | 80.14 | 90.02 | 84.94 | 89.78 | 85.84 | 88.75 | 81.29 | 85.82 | — | |
| ST5-EncParams=4.8B, Fine-tuning=Community QA and NLI datasets2023.07 | 80.1 | 88.75 | 84.7 | 88.86 | 85.17 | 86.77 | 80.39 | 84.96 | — | |
| PromptEOL+CSEBackbone=OPT, Params=2.7B, Fine-tuning=Supervised NLI datasets2023.07 | 79.49 | 89.64 | 84.8 | 89.51 | 85.91 | 88.33 | 81.64 | 85.62 | — | |
| PromptEOL+CSEBackbone=LLaMA, Params=7B, Fine-tuning=Supervised NLI datasets2023.07 | 79.16 | 90.22 | 85.4 | 88.99 | 86.25 | 88.37 | 81.51 | 85.7 | — | |
| PromptEOL+CSEBackbone=OPT, Params=1.3B, Fine-tuning=Supervised NLI datasets2023.07 | 79.01 | 89.26 | 84.1 | 88.3 | 84.62 | 87.71 | 80.52 | 84.79 | — | |
| PromptEOLTraining dataset=manual dataset, Base model=LLaMA-2-7B2024.02 | 78.75 | 89.99 | 84.98 | 88.82 | 86.27 | 88.37 | 82.44 | 85.66 | — | |
| PromptEOL+CSEBackbone=LLaMA, Params=13B, Fine-tuning=Supervised NLI datasets2023.07 | 78.63 | 90.03 | 85.46 | 89.48 | 86.18 | 88.45 | 82.69 | 85.85 | — | |
| DebCSEBackbone=RoBERTa-large2025.02 | 77.68 | 87.17 | 80.53 | 85.9 | 83.57 | 85.36 | 73.89 | 82.01 | — | |
| ConSERTlarge joint-unsupBackbone=BERT-large, Training Supervision=NLI supervision and STS unlabeled texts, Strategy=joint unsupervised2021.05 | 77.47 | 85.45 | 79.41 | 85.59 | 80.39 | 83.42 | 77.26 | 81.28 | — | |
| SimCSE-RoBERTaParams=354M, Fine-tuning=Supervised datasets2023.07 | 77.46 | 87.27 | 82.36 | 86.66 | 83.93 | 86.7 | 81.95 | 83.76 | — | |
| MultiCSRBackbone=RoBERTa-base2025.02 | 77.03 | 84.72 | 79.71 | 85.8 | 82.68 | 84.24 | 80.64 | 82.12 | — | |
| DebCSEBackbone=BERT-large2025.02 | 76.82 | 86.36 | 79.81 | 85.8 | 80.83 | 83.45 | 74.67 | 81.11 | — | |
| MultiCSR-rBackbone=RoBERTa-base2025.02 | 76.79 | 85.03 | 80 | 86.05 | 82.65 | 84.79 | 81.14 | 82.35 | — | |
| PromptRoBERTaParams=123M, Fine-tuning=Supervised datasets2023.07 | 76.75 | 85.93 | 82.28 | 86.69 | 82.8 | 86.14 | 80.04 | 82.95 | — | |
| SimCSE-RoBERTaParams=123M, Fine-tuning=Supervised datasets2023.07 | 76.53 | 85.21 | 80.95 | 86.03 | 82.57 | 85.83 | 80.5 | 82.52 | — | |
| MultiCSR-rBackbone=BERT-base2025.02 | 76.37 | 82.5 | 78.37 | 85.38 | 82.15 | 84.01 | 80.55 | 81.33 | — | |
| SynCSE-rBackbone=BERT-large2025.02 | 76.32 | 85.17 | 79.29 | 85.78 | 82.76 | 84.76 | 82.51 | 82.37 | — | |
| DebCSEBackbone=BERT-base2025.02 | 76.15 | 84.67 | 78.91 | 85.41 | 80.55 | 82.99 | 73.6 | 80.33 | — | |
| SynCSEBackbone=RoBERTa-base2025.02 | 76.15 | 84.41 | 79.23 | 84.85 | 82.87 | 83.95 | 81.41 | 81.84 | — | |
| SynCSE-rBackbone=RoBERTa-base2025.02 | 76.01 | 83.18 | 79.13 | 85.51 | 83.03 | 84.66 | 80.93 | 81.78 | — | |
| SynCSEBackbone=RoBERTa-large2025.02 | 75.92 | 85.01 | 80.43 | 85.83 | 84.4 | 85.05 | 81.99 | 82.66 | — | |
| MultiCSRBackbone=BERT-base2025.02 | 75.88 | 82.39 | 78.8 | 84.42 | 80.54 | 82.23 | 80.03 | 80.61 | — | |
| SynCSE-rBackbone=BERT-base2025.02 | 75.82 | 83.24 | 78.61 | 84.75 | 81.68 | 83.45 | 80.67 | 81.17 | — | |
| MultiCSR-rBackbone=BERT-large2025.02 | 75.69 | 85.63 | 79.92 | 86.08 | 82.69 | 84.88 | 82.37 | 82.47 | — | |
| RankCSE_listMLEPLM=BERT_base2023.05 | 75.66 | 86.27 | 77.81 | 84.74 | 81.1 | 81.8 | 75.13 | 80.36 | — | |
| RankCSEBackbone=BERT-base2025.02 | 75.66 | 86.27 | 77.81 | 84.74 | 81.1 | 81.8 | 75.13 | 80.36 | — | |
| SynCSE-rBackbone=RoBERTa-large2025.02 | 75.64 | 84.53 | 80.36 | 85.88 | 84.47 | 85.82 | 83.24 | 82.85 | — | |
| MultiCSRBackbone=BERT-large2025.02 | 75.56 | 85.19 | 80.14 | 85.91 | 82.4 | 84.19 | 81.65 | 82.15 | — | |
| RankCSE_listMLEPLM=BERT_large2023.05 | 75.48 | 86.5 | 78.6 | 85.45 | 81.09 | 81.58 | 75.53 | 80.6 | — | |
| RankCSEBackbone=BERT-large2025.02 | 75.48 | 86.5 | 78.6 | 85.45 | 81.09 | 81.58 | 75.53 | 80.6 | — | |
| ConSERTlarge sup-unsupBackbone=BERT-large, Training Supervision=NLI supervision and STS unlabeled texts, Strategy=supervised then unsupervised2021.05 | 75.26 | 86.01 | 79 | 83.88 | 79.45 | 82.95 | 76.54 | 80.44 | — | |
| SynCSEBackbone=BERT-large2025.02 | 75.23 | 84.28 | 79.41 | 84.89 | 82.09 | 83.48 | 81.79 | 81.6 | — | |
| SBERT-base + Contrastive (SG-OPT)Backbone=SBERT-base, Pooling=CLS2021.06 | 75.16 | 81.27 | 76.31 | 84.71 | 80.33 | 81.46 | 76.64 | 79.41 | — | |
| CompCSEPLM=BERT-large, compositional objective=+ subsum.2023.07 | 75.1 | 86.57 | 77.7 | 84.72 | 80.25 | 80.17 | 73.21 | 79.67 | — | |
| PCLPLM=BERT-large2023.07 | 74.89 | 85.88 | 78.33 | 85.3 | 80.13 | 81.39 | 73.66 | 79.94 | — | |
| PCLPLM=BERT_large2023.05 | 74.87 | 86.11 | 78.29 | 85.65 | 80.52 | 81.62 | 73.94 | 80.14 | — | |
| PCLBackbone=BERT-large2025.02 | 74.87 | 86.11 | 78.29 | 85.65 | 80.52 | 81.62 | 73.94 | 80.14 | — | |
| SBERT-large + Contrastive (SG)Backbone=SBERT-large, Pooling=CLS2021.06 | 74.84 | 82.89 | 77.27 | 84.44 | 79.54 | 82.35 | 76.44 | 79.68 | — | |
| RankCSE_listNetPLM=BERT_large2023.05 | 74.75 | 86.46 | 78.52 | 85.41 | 80.62 | 81.4 | 76.12 | 80.47 | — | |
| SBERT-large + Contrastive (SG-OPT)Backbone=SBERT-large, Pooling=CLS2021.06 | 74.58 | 83.79 | 76.98 | 84.57 | 79.87 | 82.05 | 76.44 | 79.76 | — | |
| SROBERTa-NLI-largeBackbone=RoBERTa-large, Training data=NLI2019.08 | 74.53 | 77 | 73.18 | 81.85 | 76.82 | 79.1 | 74.29 | 76.68 | — | |
| SROBERTalarge-NLIBackbone=RoBERTa, Model Scale=large, Supervision=NLI2020.11 | 74.53 | 77 | 73.18 | 81.85 | 76.82 | 79.1 | 74.29 | — | — | |
| SynCSEBackbone=BERT-base2025.02 | 74.53 | 82.14 | 78.22 | 83.46 | 80.66 | 81.42 | 80.51 | 80.13 | — | |
| MoCoSEPLM=BERT-large2023.07 | 74.5 | 84.54 | 77.32 | 84.11 | 79.67 | 80.53 | 73.26 | 79.13 | — | |
| MultiCSRBackbone=RoBERTa-large2025.02 | 74.42 | 84.46 | 79.17 | 84.76 | 83.67 | 84.23 | 81.5 | 81.74 | — | |
| RankCSE_listNetPLM=BERT_base2023.05 | 74.38 | 85.97 | 77.51 | 84.46 | 81.31 | 81.46 | 75.26 | 80.05 | — | |
| CLAIF-BERTscaling=scaled2023.05 | 74.36 | 85.07 | 80.64 | 87.21 | 83.36 | 86.26 | 79.68 | 82.37 | — | |
| DebCSEBackbone=RoBERTa-base2025.02 | 74.29 | 85.54 | 79.46 | 85.68 | 81.2 | 83.96 | 74.04 | 80.6 | — | |
| SGPTParams=5.8B, Fine-tuning=Supervised datasets2023.07 | 74.28 | 85.35 | 79.21 | 85.52 | 82.54 | 85.5 | 79.53 | 81.7 | — | |
| MultiCSR-rBackbone=RoBERTa-large2025.02 | 74.28 | 84.81 | 79.2 | 85.26 | 83.93 | 84.4 | 81.62 | 81.93 | — | |
| PromptEOLTraining dataset=automatically generated (Auto_NLI), Base model=LLaMA-2-7B, Shots=20-shot2024.02 | 74.24 | 87.39 | 82.55 | 85.49 | 84.36 | 85.24 | 74.2 | 81.92 | — | |
| PromptEOLTraining dataset=automatically generated (Auto_NLI), Base model=LLaMA-2-7B, Shots=5-shot x 42024.02 | 74.16 | 87.75 | 82.65 | 85.95 | 84.97 | 85.26 | 76.44 | 82.45 | — | |
| PCLPLM=RoBERTa_large2023.05 | 74.08 | 84.36 | 76.42 | 85.49 | 81.76 | 82.79 | 71.51 | 79.49 | — | |
| PCLBackbone=RoBERTa-large2025.02 | 74.08 | 84.36 | 76.42 | 85.49 | 81.76 | 82.79 | 71.51 | 79.49 | — | |
| ConSERTbase joint-unsupBackbone=BERT-base, Training Supervision=NLI supervision and STS unlabeled texts, Strategy=joint unsupervised2021.05 | 74.07 | 83.93 | 77.05 | 83.66 | 78.76 | 81.36 | 76.77 | 79.37 | — | |
| ClusterNS-RoBERTabasePrompt-based=true, Backbone=RoBERTa, Model Size=base2023.05 | 74.02 | 85.12 | 77.96 | 84.47 | 82.84 | 83.28 | 70.47 | 79.74 | — | |
| ClusterNS-BERTlargePrompt-based=true, Backbone=BERT, Model Size=large2023.05 | 73.99 | 87.53 | 78.82 | 85.47 | 80.84 | 82.85 | 72.59 | 80.3 | — | |
| PromptRoBERTabasePrompt-based=true, Backbone=RoBERTa, Model Size=base2023.05 | 73.94 | 84.74 | 77.28 | 84.99 | 81.74 | 81.88 | 69.5 | 79.15 | — | |
| PromptRoBERTaParams=123M, Training Type=Unsupervised Contrastive Training2024.02 | 73.94 | 84.74 | 77.28 | 84.99 | 81.74 | 81.88 | 69.5 | 79.15 | — | |
| PromptROBERTaBackbone=RoBERTa-base2025.02 | 73.94 | 84.74 | 77.28 | 84.99 | 81.74 | 81.88 | 69.5 | 79.15 | — | |
| SBERT-base + Contrastive (SG)Backbone=SBERT-base, Pooling=CLS2021.06 | 73.76 | 80.08 | 75.58 | 83.52 | 79.1 | 81.05 | 75.78 | 78.41 | — | |
| PCLPLM=RoBERTa-large2023.07 | 73.76 | 84.59 | 76.81 | 85.37 | 81.66 | 82.89 | 70.33 | 79.34 | — | |
| PromptEOLTraining dataset=automatically generated (Auto_NLI), Base model=LLaMA-2-7B, Shots=5-shot2024.02 | 73.72 | 87.75 | 81.94 | 85.71 | 83.85 | 84.49 | 75.72 | 81.88 | — | |
| SNCSEPLM=RoBERTa-large2023.07 | 73.71 | 86.73 | 80.35 | 86.8 | 83.06 | 84.31 | 77.43 | 81.77 | — | |
| PromptRoBERTaTraining dataset=unsupervised2024.02 | 73.64 | 84.97 | 77.44 | 85.11 | 81.61 | 82.12 | 69.09 | 79.14 | — | |
| ConSERTbase sup-unsupBackbone=BERT-base, Training Supervision=NLI supervision and STS unlabeled texts, Strategy=supervised then unsupervised2021.05 | 73.51 | 84.86 | 77.44 | 83.11 | 77.98 | 81.8 | 74.29 | 79 | — | |
| RankCSE_listNetPLM=RoBERTa_large2023.05 | 73.47 | 85.77 | 78.07 | 85.65 | 82.51 | 84.12 | 73.73 | 80.47 | — | |
| ESimCSEBackbone=BERT-base, Embedding Strategy=[CLS], Discrete Augmentation=true2022.11 | 73.4 | 83.27 | 77.25 | 82.66 | 78.81 | 80.17 | 72.3 | 78.27 | — | |
| ESimCSEPLM=BERT-base2023.07 | 73.4 | 83.27 | 77.25 | 82.66 | 78.81 | 80.17 | 72.3 | 78.27 | — | |
| PromptEOLTraining dataset=automatically generated (Auto_NLI), Base model=LLaMA-2-7B, Shots=1-shot2024.02 | 73.3 | 87.61 | 81.52 | 85.35 | 83.85 | 83.63 | 76.86 | 81.73 | — | |
| PromptBERTlargePrompt-based=true, Backbone=BERT, Model Size=large2023.05 | 73.29 | 86.39 | 77.9 | 85.18 | 79.97 | 81.92 | 71.26 | 79.42 | — | |
| L2P-CSRPLM=RoBERTa-large2023.07 | 73.29 | 84.08 | 76.65 | 85.47 | 82.7 | 82.15 | 72.36 | 79.53 | — | |
| PromptEOLTraining dataset=automatically generated (Auto_NLI), Base model=LLaMA-2-7B, Shots=1-shot x 52024.02 | 73.27 | 87.9 | 81.74 | 85.72 | 84.11 | 84.66 | 76.45 | 81.98 | — | |
| ConSERTlarge jointBackbone=BERT-large, Training Supervision=NLI supervision, Optimization Strategy=joint2021.05 | 73.26 | 82.36 | 77.73 | 83.84 | 78.75 | 81.54 | 78.64 | 79.44 | — | |
| ESimCSEPLM=BERT-large2023.07 | 73.21 | 85.37 | 77.73 | 84.3 | 78.92 | 80.73 | 74.89 | 79.31 | — | |
| ConPVP-RoBERTabasePrompt-based=true, Backbone=RoBERTa, Model Size=base2023.05 | 73.2 | 83.22 | 76.24 | 83.37 | 81.49 | 82.18 | 74.59 | 79.18 | — | |
| RankCSE_listMLEPLM=RoBERTa_base2023.05 | 73.2 | 85.95 | 77.17 | 84.82 | 82.58 | 83.08 | 71.88 | 79.81 | — | |
| RankCSE_listMLEPLM=RoBERTa_large2023.05 | 73.2 | 85.83 | 78 | 85.63 | 82.67 | 84.19 | 73.64 | 80.45 | — | |
| ESimCSEPLM=RoBERTa-large2023.07 | 73.2 | 84.93 | 76.88 | 84.86 | 81.21 | 82.79 | 72.27 | 79.45 | — | |
| RankCSEBackbone=RoBERTa-base2025.02 | 73.2 | 85.95 | 77.17 | 84.82 | 82.58 | 83.08 | 71.88 | 79.81 | — | |
| RankCSEBackbone=RoBERTa-large2025.02 | 73.2 | 85.83 | 78 | 85.63 | 82.67 | 84.19 | 73.64 | 80.45 | — | |
| ArcCSEPLM=BERT_large2023.05 | 73.17 | 86.19 | 77.9 | 84.97 | 79.43 | 80.45 | 73.5 | 79.37 | — | |
| ArcCSEPLM=BERT-large2023.07 | 73.17 | 86.19 | 77.9 | 84.97 | 79.43 | 80.45 | 73.5 | 79.37 | — | |
| VisualCSE-RoBERTalargeBackbone=RoBERTa-large, Supervision Type=Visual + Text (unsupervised)2022.09 | 73.09 | 84.77 | 77.09 | 85.47 | 82.06 | 83.26 | 72.23 | 79.71 | — | |
| DCLRPLM=RoBERTa_large2023.05 | 73.09 | 84.57 | 76.13 | 85.15 | 81.99 | 82.35 | 71.8 | 79.3 | — | |
| DCLRPLM=RoBERTa-large2023.07 | 73.09 | 84.57 | 76.13 | 85.15 | 81.99 | 82.35 | 71.8 | 79.3 | — | |
| ClusterNS-BERTbasePrompt-based=true, Backbone=BERT, Model Size=base2023.05 | 72.92 | 84.86 | 77.38 | 84.52 | 80.23 | 81.58 | 69.53 | 78.72 | — | |
| RankCSE_listNetPLM=RoBERTa_base2023.05 | 72.91 | 85.72 | 76.94 | 84.52 | 82.59 | 83.46 | 71.94 | 79.73 | — | |
| SimCSE-RoBERTalargeBackbone=RoBERTa-large, Supervision Type=Text-only (unsupervised)2022.09 | 72.86 | 83.99 | 75.62 | 84.77 | 81.8 | 81.98 | 71.26 | 78.9 | — | |
| SimCSE-RoBERTalargeBackbone=RoBERTa-large, Supervision modality=Unsupervised text, Framework=SimCSE2022.09 | 72.86 | 83.99 | 75.62 | 84.77 | 81.8 | 81.98 | 71.26 | 78.9 | — | |
| SimCSEPLM=RoBERTa_large2023.05 | 72.86 | 83.99 | 75.62 | 84.77 | 81.8 | 81.98 | 71.26 | 78.9 | — | |
| SimCSEPLM=RoBERTa-large2023.07 | 72.86 | 83.99 | 75.62 | 84.77 | 81.8 | 81.98 | 71.26 | 78.9 | — | |
| SimCSEBackbone=RoBERTa-large2025.02 | 72.86 | 83.99 | 75.62 | 84.77 | 81.8 | 81.98 | 71.26 | 78.9 | — | |
| PCLPLM=BERT_base2023.05 | 72.84 | 83.81 | 76.52 | 83.06 | 79.32 | 80.01 | 73.38 | 78.42 | — | |
| PCLBackbone=BERT-base2025.02 | 72.84 | 83.81 | 76.52 | 83.06 | 79.32 | 80.01 | 73.38 | 78.42 | — | |
| PCLBackbone=BERT-base, Embedding Strategy=[CLS], Discrete Augmentation=true2022.11 | 72.74 | 83.36 | 76.05 | 83.07 | 79.26 | 79.72 | 72.75 | 78.14 | — | |
| PCLPLM=BERT-base2023.07 | 72.74 | 83.36 | 76.05 | 83.07 | 79.26 | 79.72 | 72.75 | 78.14 | — |