Sentence Embedding Evaluation on SentEval
89.44Average Score (Avg)CLAIFscaled-RoBERTa
Evaluation results
| Method | Links | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CLAIFscaled-RoBERTaBackbone=RoBERTa, Setting=scaled2023.05 | 89.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PromptRoBERTa supervised w/ CLHAIFBackbone=RoBERTa, Supervision=supervised, Framework=CLHAIF2023.05 | 89.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PromptRoBERTa supervisedBackbone=RoBERTa, Supervision=supervised2023.05 | 89.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SimCSE_RoBERTa supervised w/ CLHAIFBackbone=RoBERTa, Supervision=supervised, Framework=CLHAIF2023.05 | 88.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SimCSE_RoBERTa supervisedBackbone=RoBERTa, Supervision=supervised2023.05 | 88.08 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CLAIF_RoBERTaBackbone=RoBERTa2023.05 | 87.99 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PromptRoBERTaBackbone=RoBERTa2023.05 | 87.36 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CLAIFscaled-BERTBackbone=BERT, Setting=scaled2023.05 | 87.15 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PromptBERT supervised w/ CLHAIFBackbone=BERT, Supervision=supervised, Framework=CLHAIF2023.05 | 87.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DiffCSE-RoBERTa_baseBackbone=RoBERTa-base2022.04 | 87.04 | 82.82 | 88.61 | 94.32 | 87.71 | 88.63 | 90.4 | 76.81 | — | — | — | — | — | — | — | — | — | |
| DiffCSE_RoBERTaBackbone=RoBERTa2023.05 | 87.04 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PromptBERT supervisedBackbone=BERT, Supervision=supervised2023.05 | 86.98 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SimCSE-RoBERTa_base w/ MLMBackbone=RoBERTa-base, Objective=MLM2022.04 | 86.9 | 83.37 | 87.76 | 95.05 | 87.16 | 89.02 | 90.8 | 75.13 | — | — | — | — | — | — | — | — | — | |
| CMLM-BERT_baseBackbone=BERT-base, Training Data=1TB data2022.04 | 86.89 | 83.6 | 89.9 | 96.2 | 89.3 | 88.5 | 91 | 69.7 | — | — | — | — | — | — | — | — | — | |
| DiffCSE-BERT_baseBackbone=BERT-base2022.04 | 86.86 | 82.69 | 87.23 | 95.23 | 89.28 | 86.6 | 90.4 | 76.58 | — | — | — | — | — | — | — | — | — | |
| DiffCSE_BERTBackbone=BERT2023.05 | 86.86 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SimCSE_BERT supervised w/ CLHAIFBackbone=BERT, Supervision=supervised, Framework=CLHAIF2023.05 | 86.73 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SimCSE-BERT_base w/ MLMBackbone=BERT-base, Objective=MLM2022.04 | 86.64 | 82.92 | 87.23 | 95.71 | 88.73 | 86.81 | 87.01 | 78.07 | — | — | — | — | — | — | — | — | — | |
| CLAIF_BERTBackbone=BERT2023.05 | 86.62 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SimCSE_BERT supervisedBackbone=BERT, Supervision=supervised2023.05 | 86.51 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IS-BERT_baseBackbone=BERT-base2022.04 | 85.83 | 81.09 | 87.18 | 94.96 | 88.75 | 85.96 | 88.64 | 74.24 | — | — | — | — | — | — | — | — | — | |
| SimCSE-BERT_baseBackbone=BERT-base2022.04 | 85.81 | 81.18 | 86.46 | 94.45 | 88.88 | 85.5 | 89.8 | 74.43 | — | — | — | — | — | — | — | — | — | |
| SimCSE_BERTBackbone=BERT2023.05 | 85.81 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PromptBERTBackbone=BERT2023.05 | 85.49 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Avg. BERT embeddingsBackbone=BERT-base, Pooling=Average2022.04 | 84.94 | 78.66 | 86.25 | 94.37 | 88.66 | 84.4 | 92.8 | 69.54 | — | — | — | — | — | — | — | — | — | |
| SimCSE-RoBERTa_baseBackbone=RoBERTa-base2022.04 | 84.84 | 81.04 | 87.74 | 93.28 | 86.94 | 86.6 | 84.6 | 73.68 | — | — | — | — | — | — | — | — | — | |
| SimCSE_RoBERTaBackbone=RoBERTa2023.05 | 84.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-[CLS] embeddingBackbone=BERT-base, Pooling=[CLS]2022.04 | 84.66 | 78.68 | 84.85 | 94.21 | 88.23 | 84.13 | 91.4 | 71.13 | — | — | — | — | — | — | — | — | — | |
| Skip-thought2022.04 | 83.5 | 76.5 | 80.1 | 93.6 | 87.1 | 82 | 92.2 | 73 | — | — | — | — | — | — | — | — | — | |
| GloVe embeddings (avg.)Backbone=GloVe, Pooling=Average2022.04 | 81.52 | 77.25 | 78.3 | 91.17 | 87.85 | 80.18 | 83 | 72.87 | — | — | — | — | — | — | — | — | — | |
| MLM+ subs+ del-spanPooling strategy=[CLS], Objective=2-CL, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 61.8 | — | — | — | — | — | — | — | — | 75.6 | — | 72.5 | 49 | 48.9 | 57.4 | 63.6 | 65.6 | |
| MLM+ del-span+ reorderPooling strategy=[CLS], Objective=2-CL, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 61.5 | — | — | — | — | — | — | — | — | 75 | — | 68.7 | 49.4 | 54.3 | 57.6 | 64 | 61.4 | |
| MLM+ del-span+ reorderPooling strategy=mean, Objective=2-CL, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 59.1 | — | — | — | — | — | — | — | — | 75.4 | — | 67.8 | 48.3 | 50.3 | 54.9 | 60.4 | 56.8 | |
| MLM+ subs+ del-spanPooling strategy=mean, Objective=2-CL, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 58.5 | — | — | — | — | — | — | — | — | 75.5 | — | 67 | 48.3 | 45 | 54.6 | 60.9 | 58.5 | |
| MLM+ del-word+ reorderPooling strategy=mean, Objective=2-CL, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 58.4 | — | — | — | — | — | — | — | — | 75.8 | — | 66.2 | 51.1 | 45.7 | 51.8 | 61.3 | 57 | |
| MLM+ del-wordPooling strategy=[CLS], Objective=1-CL, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 58.1 | — | — | — | — | — | — | — | — | 77.1 | — | 71.6 | 50.6 | 44.5 | 48.3 | 58.4 | 56.1 | |
| MLM+ del-wordPooling strategy=mean, Objective=1-CL, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 56.7 | — | — | — | — | — | — | — | — | 75.9 | — | 69 | 50.6 | 40 | 50.2 | 58.9 | 52.4 | |
| RoBERTa-basePooling strategy=[CLS], Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 56.1 | — | — | — | — | — | — | — | — | 75.9 | — | 71.9 | 47.4 | 37.5 | 47.9 | 55.1 | 57.6 | |
| MLM+ del-spanPooling strategy=mean, Objective=1-CL, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 54.8 | — | — | — | — | — | — | — | — | 71 | — | 62.6 | 49.3 | 41.7 | 48.9 | 58.1 | 52.3 | |
| MLM+ subs+ del-wordPooling strategy=[CLS], Objective=2-CL, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 54.3 | — | — | — | — | — | — | — | — | 73.6 | — | 62.9 | 44.5 | 35.8 | 47.6 | 55.8 | 59.6 | |
| RoBERTa-basePooling strategy=mean, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 53.8 | — | — | — | — | — | — | — | — | 74.1 | — | 65.6 | 47.2 | 38.3 | 46.7 | 55 | 49.5 | |
| MLM+ subs+ del-wordPooling strategy=mean, Objective=2-CL, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 53.8 | — | — | — | — | — | — | — | — | 73.6 | — | 63.4 | 44.6 | 39.8 | 50.1 | 55.5 | 49.6 | |
| MLM+ del-word+ reorderPooling strategy=[CLS], Objective=2-CL, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 49.6 | — | — | — | — | — | — | — | — | 71.9 | — | 63.8 | 41.9 | 30.9 | 37.4 | 48.9 | 52.1 | |
| MLM+ del-spanPooling strategy=[CLS], Objective=1-CL, Pre-training corpus=BookCorpus and English Wikipedia, Pre-training steps=500k2020.12 | 37.5 | — | — | — | — | — | — | — | — | 62.7 | — | 57.4 | 34.4 | 20.4 | 24.3 | 32 | 31.5 | |
| BiLSTM (InferSent)Training Data=AllNLI, Embedding Dimension=40962018.05 | — | 81.1 | 86.3 | 92.4 | 90.2 | 84.6 | 88.2 | 76.2 | 83.1 | 0.884 | 86.3 | 0.758 | — | — | — | — | — | |
| BiLSTM-2layerTraining Data=AllNLI, Embedding Dimension=40962018.05 | — | 81.3 | 86.2 | 92 | 90.2 | 85.3 | 88.4 | 75.7 | 82.9 | 0.884 | 86.3 | 0.763 | — | — | — | — | — | |
| Byte mLSTMTraining=Unsupervised/Supervised2018.05 | — | 86.9 | 91.4 | 94.6 | 88.5 | — | — | 75 | 82.8 | 0.792 | — | — | — | — | — | — | — | |
| CNN-LSTMTraining=Unsupervised/Supervised2018.05 | — | 77.8 | 82.1 | 93.6 | 89.4 | — | 92.6 | 76.5 | 83.8 | 0.862 | — | — | — | — | — | — | — | |
| DisSentTraining=Unsupervised/Supervised2018.05 | — | 80.1 | 84.9 | 93.6 | 90.1 | 84.1 | 93.6 | 75 | — | 0.849 | 83.7 | — | — | — | — | — | — | |
| FastSent+AETraining=Unsupervised/Supervised2018.05 | — | 71.8 | 76.7 | 88.8 | 81.5 | — | 80.4 | 71.2 | 79.1 | — | — | — | — | — | — | — | — | |
| MultiTaskTraining=Unsupervised/Supervised2018.05 | — | 82.5 | 87.7 | 94 | 90.9 | 83.2 | 93 | 78.6 | 84.4 | 0.888 | 87.8 | 0.789 | — | — | — | — | — | |
| QuickThoughtsTraining=Unsupervised/Supervised2018.05 | — | 82.4 | 86 | 94.8 | 90.2 | 87.6 | 92.4 | 76.9 | 84 | 0.874 | — | — | — | — | — | — | — | |
| SkipThought-LNTraining=Unsupervised/Supervised2018.05 | — | 79.4 | 83.1 | 93.7 | 89.3 | 82.9 | 88.4 | — | — | 0.858 | 79.5 | — | — | — | — | — | — | |
| SuBiLSTMTraining Data=AllNLI, Embedding Dimension=40962018.05 | — | 81.4 | 86.4 | 93.2 | 90.7 | 85 | 89.8 | 76.3 | 83.4 | 0.886 | 86.7 | 0.77 | — | — | — | — | — | |
| SuBiLSTM-TiedTraining Data=AllNLI, Embedding Dimension=40962018.05 | — | 81.6 | 86.5 | 93 | 90.5 | 85.1 | 90.4 | 76.3 | 83.3 | 0.885 | 86.3 | 0.771 | — | — | — | — | — |