# Medical LLM Evaluation on LLMEval Med Source: [Medical LLM Evaluation on LLMEval Med]() Collection index: [llms.txt]() Data updated: 2026-03-23T11:46:38.455Z Evaluation of Large Language Model performance specifically on medical domain knowledge and reasoning, covering aspects like safety, ethics, and core understanding. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Medical LLM Evaluation | | Dataset (including any stated split) | LLMEval Med | | Metric | Reasoning Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 31 of 31 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Reasoning Score](): Higher is better; unit: PERCENT. - [Safety & Ethics Score](): Higher is better; unit: PERCENT. - [Understanding Score](): Higher is better; unit: PERCENT. - [Overall Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Reasoning Score | Result date | Report paper | | --- | --- | --- | --- | | Gemini-2.5-Pro | 73.5 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | Qwen3-235B-Instruct | 65.7 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | Baichuan-M2-32B | 64.2 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | o3 | 63.8 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | Deepseek-R1 | 63.4 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | GPT-4.1 | 60.4 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | Qwen3-30B-A3B-Instruct + More Query Rubrics | 59.8 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | Qwen3-32B | 59.1 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | Claude-3.7-Sonnet | 57.8 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | HuatuoGPT-o1-72B | 56.9 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | Qwen3-30B-A3B-Instruct | 54.9 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | Qwen3-4B-Instruct + Doctor Rubrics | 46.1 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | Qwen3-4B-Instruct + More Query Rubrics | 44.7 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | Qwen3-4B-Instruct + Principle Rubrics | 42.2 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | Qwen3-4B-Instruct | 39.2 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | | Qwen3-4B-Instruct + Draft Rubrics | 39.2 | Not recorded | 698c0d57c97a0b3e3b5e13a3 | ### Gemini-2.5-Pro Reasoning Score: 73.5. Result ID: 6995f6c85698665ec42ab24e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Closed-source LLM"}. This record does not assert independent reproduction. ### Qwen3-235B-Instruct Reasoning Score: 65.7. Result ID: 6995f6c85698665ec42ab24a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Open-source LLM"}. This record does not assert independent reproduction. ### Baichuan-M2-32B Reasoning Score: 64.2. Result ID: 6995f6c85698665ec42ab250. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Specialized LLM"}. This record does not assert independent reproduction. ### o3 Reasoning Score: 63.8. Result ID: 6995f6c85698665ec42ab24c. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed-source LLM"}. This record does not assert independent reproduction. ### Deepseek-R1 Reasoning Score: 63.4. Result ID: 6995f6c85698665ec42ab249. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Open-source LLM"}. This record does not assert independent reproduction. ### GPT-4.1 Reasoning Score: 60.4. Result ID: 6995f6c85698665ec42ab24f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Closed-source LLM"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + More Query Rubrics Reasoning Score: 59.8. Result ID: 6995f6c85698665ec42ab258. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"backbone":"Qwen3-30B-A3B-Instruct","supervision":"More Query Rubrics"}. This record does not assert independent reproduction. ### Qwen3-32B Reasoning Score: 59.1. Result ID: 6995f6c85698665ec42ab24b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Open-source LLM"}. This record does not assert independent reproduction. ### Claude-3.7-Sonnet Reasoning Score: 57.8. Result ID: 6995f6c85698665ec42ab24d. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed-source LLM"}. This record does not assert independent reproduction. ### HuatuoGPT-o1-72B Reasoning Score: 56.9. Result ID: 6995f6c85698665ec42ab251. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Specialized LLM"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct Reasoning Score: 54.9. Result ID: 6995f6c85698665ec42ab257. [Reporting paper]() Recorded evaluation configuration: {"Type":"Our Method baseline"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + Doctor Rubrics Reasoning Score: 46.1. Result ID: 6995f6c85698665ec42ab254. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"backbone":"Qwen3-4B-Instruct","supervision":"Doctor Rubrics"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + More Query Rubrics Reasoning Score: 44.7. Result ID: 6995f6c85698665ec42ab256. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"backbone":"Qwen3-4B-Instruct","supervision":"More Query Rubrics"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + Principle Rubrics Reasoning Score: 42.2. Result ID: 6995f6c85698665ec42ab255. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"backbone":"Qwen3-4B-Instruct","supervision":"Principle Rubrics"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Reasoning Score: 39.2. Result ID: 6995f6c85698665ec42ab252. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Our Method baseline"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + Draft Rubrics Reasoning Score: 39.2. Result ID: 6995f6c85698665ec42ab253. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"backbone":"Qwen3-4B-Instruct","supervision":"Draft Rubrics"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.