Medical Question Answering on VitalBench Tier A (635) (test)
86Overall AccuracyVitalAgent
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| VitalAgentSetting=Leakage-free2026.05 | 86 | 87.7 | 72.9 | 99.2 | |
| Health-LLMSetting=Oracle, Prompting Strategy=few-shot2026.05 | 82 | 72.8 | 87.6 | 100 | |
| LifeAgent*Setting=Oracle2026.05 | 81.9 | 72.5 | 87.6 | 100 | |
| Health-LLMSetting=Oracle, Prompting Strategy=zero-shot2026.05 | 78.4 | 67.8 | 84.1 | 100 | |
| Health-LLMSetting=Oracle, Prompting Strategy=few-shot CoT2026.05 | 77.6 | 64.3 | 88.2 | 100 | |
| LifeAgent*Setting=Leakage-free2026.05 | 54.5 | 47.4 | 61.2 | 65 | |
| Health-LLMSetting=Leakage-free, Prompting Strategy=zero-shot2026.05 | 49.9 | 38.9 | 61.2 | 65 | |
| Health-LLMSetting=Leakage-free, Prompting Strategy=few-shot2026.05 | 48.8 | 36.5 | 62.4 | 64.2 | |
| Health-LLMSetting=Leakage-free, Prompting Strategy=few-shot CoT2026.05 | 48.3 | 34.8 | 63.5 | 65 | |
| PHIASetting=Oracle2026.05 | 39.5 | 24.6 | 61.2 | 51.2 | |
| PHIASetting=Leakage-free2026.05 | 20.2 | 8.2 | 46.5 | 17.1 |