Patient simulation dialogue generation on 11 OSCE stations baseline configuration
52.24Information RecallClaude-Haiku-4.5
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Claude-Haiku-4.5Backbone=Claude-Haiku-4.5, Configuration=baseline, LLM-as-a-Judge=GPT-4o-mini, Temperature=0, Top-p=0.92026.06 | 52.24 | 4.97 | 87.48 | 4.76 | 3.42 | 4.35 | 4.28 | |
| GPT-4o-miniBackbone=GPT-4o-mini, Configuration=baseline, LLM-as-a-Judge=GPT-4o-mini, Temperature=0, Top-p=0.92026.06 | 47.76 | 5 | 74.81 | 4.83 | 3.6 | 4.31 | 4.42 | |
| Ministral-14BBackbone=Ministral-14B-2512, Configuration=baseline, LLM-as-a-Judge=GPT-4o-mini, Temperature=0, Top-p=0.92026.06 | 47.44 | 4.97 | 82.55 | 4.7 | 2.83 | 3.98 | 4.24 | |
| Gemini-3.1-Flash-LiteBackbone=Gemini-3.1-Flash-Lite, Configuration=baseline, LLM-as-a-Judge=GPT-4o-mini, Temperature=0, Top-p=0.92026.06 | 47.26 | 4.03 | 85.96 | 4.67 | 3.4 | 4.07 | 4.2 | |
| Recorded dialoguesType=Human-recorded2026.06 | 43.58 | 4.14 | 68.57 | 4.03 | 3.21 | 3.79 | 3.91 |