Patient Simulation Fidelity on Dementia Personas Simulation
4.06Auth Fidelity ScoreDemMA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| DemMAEvaluator=Gemini-2.5-pro2026.01 | 4.06 | 4.44 | 4.12 | 4.75 | 3.75 | 4.22 | |
| DemMAEvaluator=Qwen3-32B2026.01 | 4 | 4.42 | 4.25 | 4.15 | 3.95 | 4.15 | |
| DemMAEvaluator=GPT-5.2-pro2026.01 | 3.78 | 4.33 | 4.44 | 4.89 | 4 | 4.29 | |
| DemMAEvaluator=Expert Evaluation2026.01 | 3.5 | 4 | 3.5 | 4 | 3.5 | 3.7 | |
| SFT-UtteranceEvaluator=Expert Evaluation2026.01 | 2.5 | 2.5 | 3 | 3.5 | 4 | 3.1 | |
| SFT-UtteranceEvaluator=Gemini-2.5-pro2026.01 | 2.33 | 2.44 | 3.11 | 3.33 | 3.78 | 3 | |
| SFT-UtteranceEvaluator=Qwen3-32B2026.01 | 2.3 | 2.35 | 3.1 | 3 | 3.75 | 2.9 | |
| Clinical-Profile PromptEvaluator=Gemini-2.5-pro2026.01 | 2.26 | 2.33 | 1.78 | 2.78 | 1.22 | 2.07 | |
| SFT-UtteranceEvaluator=GPT-5.2-pro2026.01 | 2.22 | 2.11 | 2.78 | 4 | 3.85 | 2.99 | |
| Clinical-Profile PromptEvaluator=Qwen3-32B2026.01 | 2.21 | 2.3 | 1.65 | 2.25 | 1.35 | 1.95 | |
| Clinical-Profile PromptEvaluator=GPT-5.2-pro2026.01 | 2.14 | 2.11 | 2.78 | 3.56 | 1.31 | 2.38 | |
| Clinical-Profile PromptEvaluator=Expert Evaluation2026.01 | 2 | 2.5 | 2 | 3 | 1.5 | 2.2 | |
| VanillaEvaluator=Gemini-2.5-pro2026.01 | 1.6 | 2 | 2 | 2.6 | 1 | 1.84 | |
| VanillaEvaluator=Qwen3-32B2026.01 | 1.58 | 2.15 | 1.57 | 2.1 | 0.95 | 1.67 | |
| VanillaEvaluator=GPT-5.2-pro2026.01 | 1.5 | 2.39 | 2.28 | 3.17 | 0.72 | 2.01 | |
| VanillaEvaluator=Expert Evaluation2026.01 | 1.5 | 2 | 2 | 2.5 | 1 | 1.8 |