Clinical Diagnosis Retrieval on RareBench MME (n=40)
40R@1Mixed-Vendor MAC
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Mixed-Vendor MACSetup=Mixed-Vendor2026.02 | 40 | 52.5 | 60 | 65 | |
| Single-Vendor MACVendor=Gemini (Gemini-2.5-Pro)2026.02 | 35 | 45 | 47.5 | 47.5 | |
| Single-Vendor MACVendor=Claude (Claude-4.5-Sonnet)2026.02 | 35 | 45 | 50 | 57.5 | |
| Single-LLMVendor=Gemini (Gemini-2.5-Pro)2026.02 | 30 | 37.5 | 42.5 | 47.5 | |
| Single-Vendor MACVendor=OpenAI (o4-mini)2026.02 | 15 | 25 | 30 | 35 | |
| Single-LLMVendor=OpenAI (o4-mini)2026.02 | 10 | 22.5 | 25 | 27.5 | |
| Single-LLMVendor=Claude (Claude-4.5-Sonnet)2026.02 | 2.5 | 22.5 | 30 | 42.5 |