Medical Data and Knowledge Processing on EHRStruct eICU
95D-U1 AccuracyGemini 1.0 Pro
Evaluation Results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 1.0 Prosetting=zero-shot (eICU), Model Category=General LLMs2025.11 | 95 | 84 | 95 | 47 | 36 | 97 | 53 | 53.2 | 61.1 | 60.3 | 62.1 | |
| DeepSeek-V2.5setting=zero-shot (eICU), Model Category=General LLMs2025.11 | 72 | 41 | 18 | 47 | 14 | 44 | 52 | 53.2 | — | — | — | |
| DeepSeek-V3setting=zero-shot (eICU), Model Category=General LLMs2025.11 | 72 | 41 | 8 | 61 | 12 | 72 | 59 | — | 52.8 | — | — | |
| GPT-4osetting=zero-shot (eICU), Model Category=General LLMs2025.11 | 62 | 52 | 52 | 67 | 36 | 96 | 11 | 50.7 | 63.3 | 56 | 61.4 | |
| Gemini 1.5 Prosetting=zero-shot (eICU), Model Category=General LLMs2025.11 | 53 | 27 | 37 | 63 | 25 | 74 | 15 | — | — | 56.1 | — | |
| GPT-3.5 Turbosetting=zero-shot (eICU), Model Category=General LLMs2025.11 | 36 | 5 | 6 | 36 | 2 | 21 | 3 | — | 52.9 | 55.8 | 55.8 | |
| Gemini 1.5 Flashsetting=zero-shot (eICU), Model Category=General LLMs2025.11 | 34 | 39 | 35 | 83 | 24 | 59 | 45 | — | 57.6 | 57.2 | 59.9 | |
| Qwen-32Bsetting=zero-shot (eICU), Model Category=General LLMs2025.11 | 25 | 25 | 24 | 26 | 15 | 47 | 10 | — | 58.3 | 51 | — | |
| Apollosetting=zero-shot (eICU), Model Category=Medical LLMs2025.11 | 21 | — | 5 | 24 | 1 | 2 | — | — | — | — | — | |
| Med42-70Bsetting=zero-shot (eICU), Model Category=Medical LLMs2025.11 | 20 | — | 11 | 25 | 2 | 2 | — | — | — | — | — | |
| JMLRsetting=zero-shot (eICU), Model Category=Medical LLMs2025.11 | 17 | — | 2 | 11 | — | 2 | — | — | — | — | — | |
| CancerLLMsetting=zero-shot (eICU), Model Category=Medical LLMs2025.11 | 16 | — | 10 | 27 | 2 | 3 | — | — | — | — | — | |
| Qwen-72Bsetting=zero-shot (eICU), Model Category=General LLMs2025.11 | 15 | 6 | 27 | 48 | 20 | 41 | 29 | — | — | — | 52.2 | |
| MedAlpaca-13Bsetting=zero-shot (eICU), Model Category=Medical LLMs2025.11 | 11 | — | — | 10 | — | 1 | — | — | — | — | — | |
| HEALsetting=zero-shot (eICU), Model Category=Medical LLMs2025.11 | 10 | — | — | 7 | — | — | — | — | — | — | — | |
| PMC-LLaMA-13Bsetting=zero-shot (eICU), Model Category=Medical LLMs2025.11 | 10 | — | — | 12 | 1 | 1 | — | — | — | — | — | |
| Qwen-14Bsetting=zero-shot (eICU), Model Category=General LLMs2025.11 | 4 | 30 | 19 | 17 | 11 | 16 | 4 | — | — | — | — | |
| Meditron-7Bsetting=zero-shot (eICU), Model Category=Medical LLMs2025.11 | 3 | — | — | 11 | 3 | — | — | — | — | — | — | |
| Huatuosetting=zero-shot (eICU), Model Category=Medical LLMs2025.11 | 2 | — | — | — | — | — | — | — | — | — | — | |
| Qwen-7Bsetting=zero-shot (eICU), Model Category=General LLMs2025.11 | 1 | 7 | 4 | 24 | 1 | — | — | — | — | — | — |