Medical Question Answering on VitalBench Tier B (670) (test)
65.2Overall AccuracyLifeAgent*
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LifeAgent*Setting=Oracle2026.05 | 65.2 | 80 | 41.8 | 68.8 | |
| Health-LLMSetting=Oracle, Prompting Strategy=zero-shot2026.05 | 58.8 | 77.3 | 23.2 | 68.8 | |
| Health-LLMSetting=Oracle, Prompting Strategy=few-shot2026.05 | 55.8 | 72.9 | 40.2 | 49.8 | |
| VitalAgentSetting=Leakage-free2026.05 | 55.7 | 74.1 | 44.3 | 44.3 | |
| Health-LLMSetting=Oracle, Prompting Strategy=few-shot CoT2026.05 | 49.6 | 62.7 | 41.8 | 41.2 | |
| Health-LLMSetting=Leakage-free, Prompting Strategy=few-shot2026.05 | 30 | 56.1 | 19.1 | 9.5 | |
| Health-LLMSetting=Leakage-free, Prompting Strategy=few-shot CoT2026.05 | 29.6 | 54.1 | 20.1 | 9.5 | |
| LifeAgent*Setting=Leakage-free2026.05 | 28.7 | 54.5 | 16.5 | 9.5 | |
| Health-LLMSetting=Leakage-free, Prompting Strategy=zero-shot2026.05 | 28.2 | 55.3 | 13.9 | 9.5 | |
| PHIASetting=Oracle2026.05 | 22.7 | 31 | 27.8 | 8.6 | |
| PHIASetting=Leakage-free2026.05 | 14.8 | 24.3 | 19.1 | 0 |