Semantic Textual Similarity on STS Tasks (test)
72.38STS12 ScorePretended CoT w/ ReverseEOL
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Pretended CoT w/ ReverseEOLBackbone=OLMoE-1B-7B2026.06 | 72.38 | 82.71 | 74.05 | 81.63 | 80.98 | 78.04 | 70.57 | 77.19 | |
| Pretended CoT w/ ReverseEOLBackbone=LLaMA-3-8B2026.06 | 72.14 | 83.62 | 74.96 | 82.27 | 81.23 | 79.38 | 74.15 | 78.25 | |
| Pretended CoT w/ ReverseEOLBackbone=LLaMA-2-13B2026.06 | 71.22 | 81.58 | 74.13 | 81.89 | 82.19 | 80.96 | 72.23 | 77.74 | |
| Pretended CoT w/ ReverseEOLBackbone=Mistral-7B2026.06 | 71.04 | 83.28 | 74.91 | 81.45 | 81.51 | 79.3 | 73.86 | 77.91 | |
| Pretended CoT w/ ReverseEOLBackbone=DeepSeek-MoE-16B2026.06 | 70.11 | 82.57 | 72.36 | 81.37 | 80.81 | 77.99 | 74.59 | 77.11 | |
| Pretended CoT w/ ReverseEOLBackbone=Qwen2-7B2026.06 | 68.43 | 82.54 | 74.18 | 81.23 | 79.14 | 79.3 | 74.45 | 77.04 | |
| Pretended CoT w/ ReverseEOLBackbone=MiniCPM-MoE-8x2B2026.06 | 68.04 | 81.56 | 71.54 | 77.73 | 77.26 | 76.16 | 72.15 | 74.92 | |
| Pretended CoT w/ ReverseEOLBackbone=Gemma-2-2B2026.06 | 67.2 | 81.3 | 71.35 | 80.23 | 78.55 | 72.43 | 67.16 | 74.03 | |
| Pretended CoTBackbone=LLaMA-3-8B2026.06 | 66.65 | 82.6 | 72.4 | 79.36 | 80.86 | 77.09 | 73.66 | 76.09 | |
| Pretended CoT w/ ReverseEOLBackbone=Qwen1.5-MoE-A2.7B2026.06 | 66.52 | 83.25 | 73.66 | 81.15 | 77.43 | 78.19 | 72.36 | 76.08 | |
| Pretended CoTBackbone=Mistral-7B2026.06 | 66.45 | 82.04 | 72.24 | 77.93 | 79.36 | 76.66 | 71.06 | 75.11 | |
| Pretended CoTBackbone=DeepSeek-MoE-16B2026.06 | 65.81 | 81.71 | 69.83 | 77.6 | 78.6 | 75.47 | 71.91 | 74.42 | |
| Pretended CoTBackbone=LLaMA-2-13B2026.06 | 64.27 | 78.61 | 69.93 | 76.37 | 79.28 | 75.88 | 69.04 | 73.34 | |
| Pretended CoTBackbone=OLMoE-1B-7B2026.06 | 64.09 | 81.28 | 69.93 | 78.28 | 76.7 | 72.99 | 68.06 | 73.05 | |
| Pretended CoTBackbone=Qwen2-7B2026.06 | 61.64 | 78.24 | 70.14 | 74.44 | 76.63 | 76.22 | 73.3 | 72.94 | |
| Pretended CoTBackbone=Qwen1.5-MoE-A2.7B2026.06 | 61.61 | 81.26 | 69.07 | 75.34 | 74.52 | 75.1 | 70.54 | 72.49 | |
| Pretended CoT w/ ReverseEOLBackbone=Mixtral-8x7B2026.06 | 60.11 | 75.86 | 67.29 | 74.95 | 77.09 | 75.64 | 66.91 | 71.12 | |
| Pretended CoTBackbone=MiniCPM-MoE-8x2B2026.06 | 59.24 | 80.53 | 68.55 | 73.14 | 73.95 | 68.38 | 68.55 | 70.33 | |
| Pretended CoTBackbone=Gemma-2-2B2026.06 | 57.89 | 78.04 | 65.02 | 75.24 | 76.89 | 68.3 | 64.81 | 69.46 | |
| Pretended CoTBackbone=Mixtral-8x7B2026.06 | 53.15 | 70.97 | 60.46 | 73.35 | 76.67 | 73.66 | 63.75 | 67.43 |