Semantic Similarity Evaluation on Insurance Tasks N=1500 (test)
0.869Mean BERT Cosine SimilarityDeepSeek-R1 + Fine-tune
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DeepSeek-R1 + Fine-tuneMode=Fine-tuned, Number of parameters=8B2026.02 | 0.869 | 0.159 | 0.929 | 1 | 79.1 | |
| Gemini-2.5-Flash2026.02 | 0.799 | 0.16 | 0.857 | 0.977 | 68.5 | |
| GPT-4o-mini2026.02 | 0.787 | 0.163 | 0.844 | 0.974 | 65.6 | |
| Claude-Haiku-4.52026.02 | 0.757 | 0.17 | 0.803 | 0.974 | 57.5 | |
| GPT-4.12026.02 | 0.749 | 0.163 | 0.793 | 0.977 | 56.1 | |
| GPT-5.22026.02 | 0.719 | 0.167 | 0.762 | 1 | 47.2 |