Clinical Diagnosis on RareBench LIRICAL (R@1, R@3, R@5, R@10)
36.24Recall@1Single-LLM (Gemini)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Single-LLM (Gemini)Setup=Single-LLM, Backbone=Gemini-2.5-Pro2026.02 | 36.24 | 45.78 | 48.5 | 53.95 | |
| Mixed-Vendor MACSetup=Mixed-Vendor MAC2026.02 | 35.69 | 46.59 | 51.77 | 57.22 | |
| Single-Vendor MAC (Gemini)Setup=Single-Vendor MAC, Backbone=Gemini-2.5-Pro2026.02 | 33.79 | 41.42 | 43.87 | 50.95 | |
| Single-Vendor MAC (Claude)Setup=Single-Vendor MAC, Backbone=Claude-4.5-Sonnet2026.02 | 31.88 | 43.6 | 46.59 | 52.59 | |
| Single-Vendor MAC (OpenAI)Setup=Single-Vendor MAC, Backbone=o4-mini2026.02 | 30.25 | 38.96 | 41.96 | 47.41 | |
| Single-LLM (Claude)Setup=Single-LLM, Backbone=Claude-4.5-Sonnet2026.02 | 29.43 | 40.33 | 43.87 | 52.32 | |
| Single-LLM (OpenAI)Setup=Single-LLM, Backbone=o4-mini2026.02 | 28.61 | 36.51 | 41.14 | 48.5 |