Clinical Diagnosis on RareBench HMS subset n=88
51.14Recall@1Mixed-Vendor MAC
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Mixed-Vendor MACSetup=Mixed-Vendor MAC2026.02 | 51.14 | 60.23 | 68.18 | 77.27 | |
| Gemini (Gemini-2.5-Pro)Setup=Single-LLM2026.02 | 47.73 | 59.09 | 64.77 | 70.45 | |
| OpenAI (o4-mini)Setup=Single-Vendor MAC2026.02 | 47.73 | 56.82 | 62.5 | 77.27 | |
| OpenAI (o4-mini)Setup=Single-LLM2026.02 | 43.18 | 57.95 | 68.18 | 76.14 | |
| Gemini (Gemini-2.5-Pro)Setup=Single-Vendor MAC2026.02 | 43.18 | 51.14 | 56.82 | 59.09 | |
| Claude (Claude-4.5-Sonnet)Setup=Single-Vendor MAC2026.02 | 43.18 | 56.82 | 67.05 | 72.73 | |
| Claude (Claude-4.5-Sonnet)Setup=Single-LLM2026.02 | 39.77 | 56.82 | 65.91 | 75 |