Data-Driven Structured EHR Understanding and Reasoning on Synthea
82D-R2 AccuracyGemini 2.5
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Gemini 2.5Model Type=General LLMs, Setting=Zero-shot2025.11 | 82 | 98 | 58 | 92 | 83 | 100 | 100 | |
| GPT-4oModel Type=General LLMs, Setting=Zero-shot2025.11 | 56 | 79 | 51 | 52 | 48 | 70 | 84 | |
| DeepSeek-V2.5Model Type=General LLMs, Setting=Zero-shot2025.11 | 51 | 72 | 41 | 18 | 14 | 44 | 52 | |
| Qwen-72BModel Type=General LLMs, Setting=Zero-shot2025.11 | 48 | 15 | 6 | 27 | 20 | 41 | 29 | |
| Gemini 1.5Model Type=General LLMs, Setting=Zero-shot2025.11 | 41 | 29 | 34 | 32 | 21 | 19 | 16 | |
| DeepSeek-V3Model Type=General LLMs, Setting=Zero-shot2025.11 | 37 | 72 | 41 | 8 | 12 | 72 | 90 | |
| Gemini-2.0Model Type=General LLMs, Setting=Zero-shot2025.11 | 30 | 64 | 43 | 21 | 24 | 54 | 67 | |
| CancerLLMModel Type=Medical LLMs, Setting=Zero-shot2025.11 | 28 | 10 | 16 | 20 | 15 | 33 | 25 | |
| Qwen-32BModel Type=General LLMs, Setting=Zero-shot2025.11 | 26 | 25 | 25 | 24 | 15 | 47 | 10 | |
| Qwen-7BModel Type=General LLMs, Setting=Zero-shot2025.11 | 24 | 1 | 7 | 4 | 1 | — | — | |
| GPT-3.5 TurboModel Type=General LLMs, Setting=Zero-shot2025.11 | 18 | 6 | 15 | 14 | 7 | 7 | 24 | |
| Qwen-14BModel Type=General LLMs, Setting=Zero-shot2025.11 | 17 | 4 | 30 | 19 | 11 | 16 | 4 | |
| Med42-70BModel Type=Medical LLMs, Setting=Zero-shot2025.11 | 17 | 13 | 3 | 18 | 11 | 27 | 18 | |
| PMC_LLaMA_13BModel Type=Medical LLMs, Setting=Zero-shot2025.11 | 13 | 6 | 6 | 15 | 10 | 8 | — | |
| ApolloModel Type=Medical LLMs, Setting=Zero-shot2025.11 | 12 | 11 | 5 | 17 | 6 | 20 | 11 | |
| JMLRModel Type=Medical LLMs, Setting=Zero-shot2025.11 | 10 | 1 | 3 | 11 | 6 | 7 | 3 | |
| HEALModel Type=Medical LLMs, Setting=Zero-shot2025.11 | 8 | — | — | 1 | — | — | — | |
| Meditron-7BModel Type=Medical LLMs, Setting=Zero-shot2025.11 | 6 | — | 3 | — | — | — | — | |
| MedAlpaca-13BModel Type=Medical LLMs, Setting=Zero-shot2025.11 | 4 | 2 | 11 | 6 | 2 | 10 | — |