Empathetic Dialogue on KardiaBench
3.751RelevanceKardia-R1
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Kardia-R1Backbone=Qwen2.5-7B-Instruct2025.12 | 3.751 | 66.53 | 4.947 | 3.786 | 4.638 | 4.657 | |
| Kardia-R1Backbone=Gemma-7B2025.12 | 3.683 | 64.48 | 4.943 | 3.753 | 4.521 | 4.747 | |
| SFT OnlyBackbone=Qwen2.5-7B-Instruct2025.12 | 3.673 | 66.41 | 4.936 | 3.736 | 4.39 | 4.543 | |
| w/ Embedding RewardBackbone=Qwen2.5-7B-Instruct2025.12 | 3.653 | 66.31 | 4.95 | 3.743 | 4.42 | 4.556 | |
| SFT OnlyBackbone=Gemma-7B2025.12 | 3.65 | 63.87 | 4.96 | 3.676 | 4.496 | 4.67 | |
| Kardia-R1Backbone=Qwen2.5-3B-Instruct2025.12 | 3.63 | 65.78 | 4.945 | 3.65 | 4.406 | 4.653 | |
| SFT OnlyBackbone=Gemma-2B2025.12 | 3.61 | 64.07 | 4.877 | 3.653 | 4.353 | 4.582 | |
| w/ Embedding RewardBackbone=Gemma-7B2025.12 | 3.606 | 62.91 | 4.92 | 3.731 | 4.437 | 4.593 | |
| SFT OnlyBackbone=Qwen2.5-3B-Instruct2025.12 | 3.59 | 65.45 | 4.926 | 3.616 | 4.36 | 4.616 | |
| w/ Embedding RewardBackbone=Qwen2.5-3B-Instruct2025.12 | 3.583 | 65.61 | 4.933 | 3.597 | 4.33 | 4.513 | |
| Kardia-R1Backbone=Gemma-2B2025.12 | 3.579 | 64.27 | 4.938 | 3.677 | 4.426 | 4.53 | |
| w/ RLHF RewardBackbone=Gemma-2B2025.12 | 3.576 | 43.18 | 4.893 | 3.66 | 4.33 | 4.523 | |
| w/ Embedding RewardBackbone=Gemma-2B2025.12 | 3.567 | 64.12 | 4.903 | 3.641 | 4.343 | 4.5 | |
| w/ RLHF RewardBackbone=Qwen2.5-7B-Instruct2025.12 | 3.433 | 44.9 | 4.943 | 3.1 | 4.117 | 4.513 | |
| w/ RLHF RewardBackbone=Gemma-7B2025.12 | 3.333 | 29.37 | 4.936 | 2.823 | 3.903 | 4.68 | |
| w/ RLHF RewardBackbone=Qwen2.5-3B-Instruct2025.12 | 3.303 | 35.08 | 4.95 | 2.91 | 4.156 | 4.53 | |
| Base ModelBackbone=Qwen2.5-7B-Instruct2025.12 | 3.216 | 9.54 | 4.861 | 2.588 | 3.75 | 4.488 | |
| PsyLLMCategory=Specialized Empathetic Dialogue Baselines2025.12 | 3.155 | — | 4.747 | 3.407 | 3.912 | 3.466 | |
| HarnessingCategory=Specialized Empathetic Dialogue Baselines2025.12 | 3.146 | 7.79 | 4.971 | 2.453 | 3.826 | 3.382 | |
| GPT-4oCategory=Large Foundation Models (Raw)2025.12 | 3.082 | 15.14 | 4.947 | 3.291 | 4.304 | 4.91 | |
| Base ModelBackbone=Qwen2.5-3B-Instruct2025.12 | 3.07 | 9.56 | 4.667 | 2.635 | 3.891 | 4.335 | |
| DeepSeek-R1Category=Large Foundation Models (Raw)2025.12 | 3.068 | 14.37 | 4.807 | 2.446 | 3.697 | 3.146 | |
| Base ModelBackbone=Gemma-7B2025.12 | 2.821 | 2.46 | 4.142 | 2.409 | 3.344 | 4.159 | |
| Gemini-2.0-flashCategory=Large Foundation Models (Raw)2025.12 | 2.669 | 12.93 | 4.264 | 2.162 | 3.384 | 4.018 | |
| Base ModelBackbone=Gemma-2B2025.12 | 2.458 | 1.21 | 3.701 | 2.302 | 2.827 | 4.431 | |
| ReflectDiffuCategory=Specialized Empathetic Dialogue Baselines2025.12 | 1.753 | 30.43 | 2.886 | 1.813 | 1.157 | 3.62 | |
| DeepSeek-V3Category=Large Foundation Models (Raw)2025.12 | 1.262 | 3.18 | 3.149 | 1.207 | 1.383 | 4.79 | |
| Qwen2.5-72B-InstructCategory=Large Foundation Models (Raw)2025.12 | 1.209 | 4.19 | 3.102 | 1.175 | 1.282 | 4.893 |