Sentence Classification Transfer Tasks on SentEval (test)
90.53MRPromptEOL
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| PromptEOLbase_model=LLaMA-2-7B, fine_tuning=None (base model)2024.02 | 90.53 | 92.45 | 96.22 | 91.24 | 95.39 | 96.2 | 74.96 | 91 | — | |
| PromptEOLbase_model=LLaMA-2-7B, setting=1-shot x 5, dataset_size=40002024.02 | 90.46 | 48.67 | 50 | 90.63 | 67.98 | 57.84 | 33.51 | 62.73 | — | |
| PromptEOLbase_model=LLaMA-2-7B, fine_tuning_source=automatically generated dataset, setting=1-shot x 5, sentence_pairs=2560002024.02 | 90.38 | 92.75 | 95.49 | 90.61 | 94.53 | 95.28 | 72.79 | 90.26 | — | |
| PromptEOLbase_model=LLaMA-2-7B, setting=0-shot, dataset_size=40002024.02 | 90.08 | 36.24 | 54.58 | 90.87 | 94.47 | 1.8 | 33.51 | 57.36 | — | |
| PromptEOLbase_model=LLaMA-2-7B, fine_tuning_source=automatically generated dataset, setting=0-shot, sentence_pairs=2560002024.02 | 90 | 92.58 | 95.23 | 90.56 | 94.07 | 94 | 73.39 | 89.97 | — | |
| PromptEOLbase_model=LLaMA-2-7B, fine_tuning_source=manual dataset, sentence_pairs=2560002024.02 | 89.94 | 93.22 | 96.05 | 90.83 | 94.89 | 95.4 | 74.26 | 90.65 | — | |
| PromptEOLbase_model=LLaMA-2-7B, fine_tuning_source=automatically generated dataset, setting=1-shot, sentence_pairs=2560002024.02 | 89.93 | 92.63 | 95.28 | 90.62 | 94.32 | 94.76 | 72.17 | 89.96 | — | |
| PromptEOLbase_model=LLaMA-2-7B, fine_tuning_source=automatically generated dataset, setting=5-shot, sentence_pairs=2560002024.02 | 89.63 | 92.52 | 94.74 | 90.68 | 93.84 | 95.16 | 73.77 | 90.05 | — | |
| PromptEOLbase_model=LLaMA-2-7B, fine_tuning_source=automatically generated dataset, setting=5-shot x 4, sentence_pairs=2560002024.02 | 89.63 | 92.52 | 94.74 | 90.68 | 93.84 | 95.16 | 73.77 | 90.05 | — | |
| PromptEOLbase_model=LLaMA-2-7B, fine_tuning_source=automatically generated dataset, setting=20-shot, sentence_pairs=2560002024.02 | 89.33 | 92.76 | 94.71 | 91.19 | 93.66 | 93.65 | 74.2 | 89.93 | — | |
| MultiCSR-rModel=RoBERTa-large2025.02 | 87.45 | 92.29 | 94.56 | 90.45 | — | 90.8 | 78.61 | 89.45 | 91.98 | |
| SynCSE*Model=RoBERTa-large2025.02 | 87.42 | 92.21 | 94.19 | 90.82 | — | 85 | 76.87 | 88.3 | 91.6 | |
| SynCSE-rModel=RoBERTa-large2025.02 | 87.24 | 92.29 | 94.65 | 90.52 | — | 91.4 | 79.01 | 89.64 | 92.37 | |
| MultiCSR*Model=RoBERTa-large2025.02 | 87.05 | 91.87 | 94.07 | 90.53 | — | 88.8 | 78.26 | 88.88 | 91.6 | |
| PromptEOLbase_model=LLaMA-2-7B, setting=1-shot, dataset_size=40002024.02 | 85.75 | 48.61 | 55.01 | 88.57 | 58.99 | 56.98 | 33.51 | 61.06 | — | |
| SimCSEModel=BERT-large2025.02 | 85.36 | 89.38 | 95.39 | 89.63 | — | 91.8 | 76.41 | 88.34 | 90.44 | |
| MultiCSR*Model=RoBERTa-base2025.02 | 84.99 | 91.23 | 93.07 | 89.42 | — | 84.6 | 77.28 | 87.38 | 91.1 | |
| MultiCSR*Model=BERT-large2025.02 | 84.95 | 89.86 | 94.42 | 89.88 | — | 84.6 | 76.52 | 87.22 | 90.33 | |
| MultiCSR-rModel=RoBERTa-base2025.02 | 84.89 | 90.7 | 93.62 | 89.5 | — | 85.4 | 78.38 | 87.51 | 90.06 | |
| MultiCSR-rModel=BERT-large2025.02 | 84.86 | 90.17 | 95 | 89.88 | — | 88 | 76.29 | 87.7 | 89.68 | |
| SynCSE*Model=RoBERTa-base2025.02 | 84.82 | 91.31 | 93.18 | 89.7 | — | 84.8 | 76.7 | 87.26 | 90.28 | |
| SynCSE-rModel=BERT-large2025.02 | 84.74 | 90.15 | 94.99 | 89.82 | — | 87.8 | 77.57 | 87.95 | 90.61 | |
| SynCSE*Model=BERT-large2025.02 | 84.66 | 89.96 | 94.49 | 90.08 | — | 86.4 | 76.75 | 87.54 | 90.44 | |
| RankCSEModel=BERT-large2025.02 | 84.63 | 89.51 | 95.5 | 90.08 | — | 93.2 | 76.99 | 88.65 | 90.61 | |
| RankCSEModel=RoBERTa-large2025.02 | 84.61 | 89.27 | 94.47 | 89.99 | — | 92.6 | 74.43 | 87.87 | 89.73 | |
| PCLModel=RoBERTa-large2025.02 | 84.47 | 89.06 | 94.6 | 89.26 | — | 94.2 | 74.96 | 87.94 | 89.02 | |
| PromptBERTModel=RoBERTa-base2025.02 | 83.82 | 88.72 | 93.19 | 90.36 | — | 90.6 | 76.75 | 87.36 | 88.08 | |
| SynCSE-rModel=RoBERTa-base2025.02 | 83.78 | 91.15 | 92.98 | 89.5 | — | 85.8 | 77.33 | 87.21 | 89.95 | |
| RankCSEModel=RoBERTa-base2025.02 | 83.32 | 88.61 | 94.03 | 88.88 | — | 90.8 | 76.46 | 87.31 | 89.07 | |
| RankCSEModel=BERT-base2025.02 | 83.07 | 88.27 | 95.06 | 89.9 | — | 89.4 | 76.23 | 87.09 | 87.7 | |
| PromptRoBERTafine_tuning_source=unsupervised dataset2024.02 | 82.88 | 88.14 | 94.13 | 87.22 | 87.97 | 88.6 | 74.63 | 86.22 | — | |
| SimCSEModel=RoBERTa-large2025.02 | 82.74 | 87.87 | 93.66 | 88.22 | — | 92 | 69.68 | 86.11 | 88.58 | |
| PCLModel=BERT-large2025.02 | 82.47 | 87.87 | 95.04 | 89.59 | — | 93 | 76 | 87.39 | 87.75 | |
| DiffCSEModel=RoBERTa-base2025.02 | 82.42 | 88.34 | 93.51 | 87.28 | — | 86.6 | 76.35 | 86.03 | 87.7 | |
| PCLModel=RoBERTa-base2025.02 | 81.83 | 87.55 | 92.92 | 87.21 | — | 85.2 | 76.46 | 85.49 | 87.26 | |
| DiffCSEModel=BERT-base2025.02 | 81.76 | 86.2 | 94.76 | 89.21 | — | 87.6 | 75.54 | 85.8 | 86 | |
| PromptEOLbase_model=LLaMA-2-7B, setting=5-shot, dataset_size=40002024.02 | 81.68 | 81.45 | 68.82 | 86.71 | 85.06 | 57.76 | 58.04 | 74.22 | — | |
| MultiCSR*Model=BERT-base2025.02 | 81.64 | 87.79 | 93.83 | 89.91 | — | 80.2 | 75.25 | 85.11 | 87.15 | |
| MultiCSR-rModel=BERT-base2025.02 | 81.47 | 87.53 | 93.99 | 89.68 | — | 83.8 | 76 | 85.57 | 86.55 | |
| SimCSEModel=BERT-base2025.02 | 81.18 | 86.46 | 94.45 | 88.88 | — | 89.8 | 74.43 | 85.81 | 85.5 | |
| SynCSE-rModel=BERT-base2025.02 | 81.13 | 87.82 | 94.07 | 89.87 | — | 83.8 | 77.86 | 86 | 87.42 | |
| SynCSE*Model=BERT-base2025.02 | 81.09 | 88.29 | 93.53 | 90.02 | — | 84.4 | 75.3 | 85.6 | 86.6 | |
| SimCSEModel=RoBERTa-base2025.02 | 81.04 | 87.74 | 93.28 | 86.94 | — | 84.6 | 73.68 | 84.84 | 86.6 | |
| PromptBERTModel=BERT-base2025.02 | 80.74 | 85.49 | 93.65 | 89.32 | — | 88.2 | 76.06 | 85.49 | 84.95 | |
| PCLModel=BERT-base2025.02 | 80.11 | 85.25 | 94.22 | 89.15 | — | 87.4 | 76.12 | 85.34 | 85.12 | |
| PromptEOLbase_model=LLaMA-2-7B, setting=20-shot, dataset_size=40002024.02 | 79.61 | 36.24 | 50 | 80.78 | 50.08 | 1.8 | 33.51 | 47.43 | — | |
| PromptEOLbase_model=LLaMA-2-7B, setting=5-shot x 4, dataset_size=40002024.02 | 79.6 | 50.3 | 50 | 85.72 | 61.22 | 25.2 | 33.51 | 55.08 | — | |
| PromptEOLBackbone=LLAMA, Params=65B2023.07 | 0.9213 | 0.9343 | 0.9716 | 0.9191 | 0.9533 | 0.974 | 0.7728 | 0.9209 | — | |
| PromptEOLBackbone=LLAMA, Params=13B2023.07 | 0.9202 | 0.9322 | 0.9729 | 0.914 | 0.9566 | 0.958 | 0.7646 | 0.9169 | — | |
| PromptEOLBackbone=LLAMA, Params=30B2023.07 | 0.9164 | 0.9327 | 0.971 | 0.9186 | 0.9599 | 0.958 | 0.7843 | 0.9201 | — | |
| PromptEOL+CSEBackbone=LLAMA, Params=13B2023.07 | 0.9122 | 0.9322 | 0.9683 | 0.9152 | 0.9489 | 0.958 | 0.7426 | 0.9111 | — | |
| PromptEOLBackbone=OPT, Params=66B2023.07 | 0.9096 | 0.934 | 0.9701 | 0.9193 | 0.9522 | 0.964 | 0.7525 | 0.9145 | — | |
| PromptEOLBackbone=OPT, Params=30B2023.07 | 0.9095 | 0.9277 | 0.9699 | 0.9179 | 0.9528 | 0.97 | 0.7397 | 0.9125 | — | |
| PromptEOLBackbone=OPT, Params=13B2023.07 | 0.9073 | 0.929 | 0.9669 | 0.9148 | 0.9401 | 0.968 | 0.7559 | 0.9117 | — | |
| PromptEOL+ICLBackbone=OPT, Params=66B2023.07 | 0.904 | 0.925 | 0.9708 | 0.9124 | 0.9434 | 0.974 | 0.7501 | 0.9114 | — | |
| PromptEOLBackbone=LLAMA, Params=7B2023.07 | 0.904 | 0.929 | 0.9688 | 0.9157 | 0.9511 | 0.954 | 0.7513 | 0.9106 | — | |
| PromptEOL+CSEBackbone=LLAMA, Params=7B2023.07 | 0.9028 | 0.9327 | 0.9667 | 0.9145 | 0.9473 | 0.956 | 0.7554 | 0.9108 | — | |
| PromptEOL+ICLBackbone=OPT, Params=30B2023.07 | 0.9027 | 0.9282 | 0.9646 | 0.9176 | 0.9434 | 0.97 | 0.7629 | 0.9128 | — | |
| PromptEOLBackbone=OPT, Params=6.7B2023.07 | 0.9026 | 0.925 | 0.9667 | 0.9139 | 0.9467 | 0.96 | 0.7791 | 0.9134 | — | |
| PromptEOL+ICLBackbone=OPT, Params=6.7B2023.07 | 0.898 | 0.9327 | 0.9632 | 0.9146 | 0.9379 | 0.954 | 0.7443 | 0.9064 | — | |
| PromptEOL+CSEBackbone=OPT, Params=6.7B2023.07 | 0.896 | 0.9205 | 0.9591 | 0.9109 | 0.9478 | 0.958 | 0.7571 | 0.9071 | — | |
| PromptEOL+ICLBackbone=OPT, Params=13B2023.07 | 0.8945 | 0.9298 | 0.9623 | 0.9128 | 0.9451 | 0.954 | 0.7571 | 0.9079 | — | |
| PromptEOL+CSEBackbone=OPT, Params=13B2023.07 | 0.892 | 0.924 | 0.9592 | 0.9086 | 0.9374 | 0.954 | 0.731 | 0.9009 | — | |
| PromptEOLBackbone=OPT, Params=2.7B2023.07 | 0.8883 | 0.9229 | 0.9593 | 0.9176 | 0.9462 | 0.96 | 0.7594 | 0.9077 | — | |
| PromptEOL+ICLBackbone=OPT, Params=2.7B2023.07 | 0.8873 | 0.9179 | 0.9544 | 0.9154 | 0.9352 | 0.952 | 0.753 | 0.9022 | — | |
| PromptEOL+CSEBackbone=OPT, Params=1.3B2023.07 | 0.8862 | 0.9189 | 0.9549 | 0.9164 | 0.9429 | 0.948 | 0.7322 | 0.8999 | — | |
| PromptEOL+CSEBackbone=OPT, Params=2.7B2023.07 | 0.884 | 0.9216 | 0.9557 | 0.9151 | 0.9412 | 0.952 | 0.7409 | 0.9015 | — | |
| PromptEOLBackbone=OPT, Params=1.3B2023.07 | 0.8806 | 0.9155 | 0.959 | 0.9155 | 0.9308 | 0.95 | 0.7397 | 0.8987 | — | |
| PromptEOL+ICLBackbone=OPT, Params=1.3B2023.07 | 0.8705 | 0.9049 | 0.9534 | 0.9154 | 0.9072 | 0.958 | 0.7264 | 0.8908 | — | |
| PromptEOLBackbone=OPT, Params=350M2023.07 | 0.8414 | 0.8808 | 0.9317 | 0.8977 | 0.8973 | 0.912 | 0.7136 | 0.8678 | — | |
| PromptEOL+ICLBackbone=OPT, Params=350M2023.07 | 0.822 | 0.8665 | 0.9321 | 0.897 | 0.8786 | 0.876 | 0.7252 | 0.8568 | — | |
| PromptEOLBackbone=OPT, Params=125M2023.07 | 0.8086 | 0.8766 | 0.9319 | 0.8977 | 0.8731 | 0.922 | 0.7264 | 0.8623 | — | |
| PromptEOL+ICLBackbone=OPT, Params=125M2023.07 | 0.8086 | 0.871 | 0.9308 | 0.8955 | 0.871 | 0.92 | 0.7328 | 0.8614 | — |