# Medical Reasoning on MedCaseReasoning (test) Source: [Medical Reasoning on MedCaseReasoning (test)]() Collection index: [llms.txt]() Data updated: 2026-04-02T09:28:57.833Z Evaluation of model performance on the test split of the MedCaseReasoning dataset, focusing on medical reasoning capabilities measured by Accuracy (ACC), AUROC, Brier score, and Expected Calibration Error (ECE). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Medical Reasoning | | Dataset (including any stated split) | MedCaseReasoning (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [AUROC](): Higher is better; unit: PERCENT. - [Brier Score](): Lower is better; unit: SCALAR. - [Expected Calibration Error](): Lower is better; unit: SCALAR. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | GPT-5-mini | 72.5 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-5-mini + Verbalized Conf. | 72 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-5-mini + Verbalized Top-k | 70.2 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-5-mini + Verbalized Probability Distribution | 69.1 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-4.1 + Verbalized Conf. | 66.5 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-4.1 + Verbalized Distrib. | 65.5 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-4.1 | 64.4 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-4.1 + Verbalized Top-k | 62.3 | Not recorded | 691d4f16d6b0dd334292d799 | | DeepSeek-V3 + Verbalized Conf. | 57.5 | Not recorded | 691d4f16d6b0dd334292d799 | | DeepSeek-V3 | 56.8 | Not recorded | 691d4f16d6b0dd334292d799 | | DeepSeek-V3 + Verbalized Distrib. | 55.1 | Not recorded | 691d4f16d6b0dd334292d799 | | DeepSeek-V3 + Verbalized Top-k | 54.3 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-30B-A3B-Instruct + Verbalized Top-k | 44.9 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-30B-A3B-Instruct + Verbalized Conf. | 44.3 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-30B-A3B-Instruct + Verbalized Distrib. | 42.9 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-30B-A3B-Instruct | 40.4 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-30B-A3B-Instruct + Logit | 40.4 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-30B-A3B-Instruct + p(True) | 40.4 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Thinking | 31.7 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Instruct + Verbalized Conf. | 31.5 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Thinking + Verbalized Conf. | 31.5 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Instruct + Verbalized Distrib. | 31.1 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Thinking + Verbalized Top-k | 30.9 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Instruct + Verbalized Top-k | 30.1 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Instruct | 29.8 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Instruct + Logit | 29.8 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Instruct + p(True) | 29.8 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Thinking + Verbalized Probability Distribution | 29 | Not recorded | 691d4f16d6b0dd334292d799 | ### GPT-5-mini Accuracy: 72.5. Result ID: 69ce36d647f34ce924b7b092. [Reporting paper]() Recorded evaluation configuration: {"Backbone Model":"GPT-5-mini","Confidence Estimation Strategy":"Base"}. This record does not assert independent reproduction. ### GPT-5-mini + Verbalized Conf. Accuracy: 72. Result ID: 69ce36d647f34ce924b7b093. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"GPT-5-mini","Confidence Estimation Strategy":"Verbalized Conf."}. This record does not assert independent reproduction. ### GPT-5-mini + Verbalized Top-k Accuracy: 70.2. Result ID: 69ce36d647f34ce924b7b094. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"GPT-5-mini","Confidence Estimation Strategy":"Verbalized Top-k"}. This record does not assert independent reproduction. ### GPT-5-mini + Verbalized Probability Distribution Accuracy: 69.1. Result ID: 69ce36d647f34ce924b7b095. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"GPT-5-mini","Confidence Estimation Strategy":"Verbalized Probability Distribution"}. This record does not assert independent reproduction. ### GPT-4.1 + Verbalized Conf. Accuracy: 66.5. Result ID: 69ce36d647f34ce924b7b077. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4.1","Inference Strategy":"+ Verbalized Conf."}. This record does not assert independent reproduction. ### GPT-4.1 + Verbalized Distrib. Accuracy: 65.5. Result ID: 69ce36d647f34ce924b7b079. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4.1","Inference Strategy":"+ Verbalized Probability Distribution"}. This record does not assert independent reproduction. ### GPT-4.1 Accuracy: 64.4. Result ID: 69ce36d647f34ce924b7b076. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4.1","Inference Strategy":"Base"}. This record does not assert independent reproduction. ### GPT-4.1 + Verbalized Top-k Accuracy: 62.3. Result ID: 69ce36d647f34ce924b7b078. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4.1","Inference Strategy":"+ Verbalized Top-k"}. This record does not assert independent reproduction. ### DeepSeek-V3 + Verbalized Conf. Accuracy: 57.5. Result ID: 69ce36d647f34ce924b7b07b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3","Inference Strategy":"+ Verbalized Conf."}. This record does not assert independent reproduction. ### DeepSeek-V3 Accuracy: 56.8. Result ID: 69ce36d647f34ce924b7b07a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3","Inference Strategy":"Base"}. This record does not assert independent reproduction. ### DeepSeek-V3 + Verbalized Distrib. Accuracy: 55.1. Result ID: 69ce36d647f34ce924b7b07d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3","Inference Strategy":"+ Verbalized Probability Distribution"}. This record does not assert independent reproduction. ### DeepSeek-V3 + Verbalized Top-k Accuracy: 54.3. Result ID: 69ce36d647f34ce924b7b07c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3","Inference Strategy":"+ Verbalized Top-k"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + Verbalized Top-k Accuracy: 44.9. Result ID: 69ce36d647f34ce924b7b074. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B-Instruct","Inference Strategy":"+ Verbalized Top-k"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + Verbalized Conf. Accuracy: 44.3. Result ID: 69ce36d647f34ce924b7b073. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B-Instruct","Inference Strategy":"+ Verbalized Conf."}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + Verbalized Distrib. Accuracy: 42.9. Result ID: 69ce36d647f34ce924b7b075. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B-Instruct","Inference Strategy":"+ Verbalized Probability Distribution"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct Accuracy: 40.4. Result ID: 69ce36d647f34ce924b7b070. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B-Instruct","Inference Strategy":"Base"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + Logit Accuracy: 40.4. Result ID: 69ce36d647f34ce924b7b071. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B-Instruct","Inference Strategy":"+ Logit"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + p(True) Accuracy: 40.4. Result ID: 69ce36d647f34ce924b7b072. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B-Instruct","Inference Strategy":"+ p(True)"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking Accuracy: 31.7. Result ID: 69ce36d647f34ce924b7b08e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"Qwen3-4B-Thinking","Confidence Estimation Strategy":"Base"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + Verbalized Conf. Accuracy: 31.5. Result ID: 69ce36d647f34ce924b7b06d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct","Inference Strategy":"+ Verbalized Conf."}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking + Verbalized Conf. Accuracy: 31.5. Result ID: 69ce36d647f34ce924b7b08f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"Qwen3-4B-Thinking","Confidence Estimation Strategy":"Verbalized Conf."}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + Verbalized Distrib. Accuracy: 31.1. Result ID: 69ce36d647f34ce924b7b06f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct","Inference Strategy":"+ Verbalized Probability Distribution"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking + Verbalized Top-k Accuracy: 30.9. Result ID: 69ce36d647f34ce924b7b090. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"Qwen3-4B-Thinking","Confidence Estimation Strategy":"Verbalized Top-k"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + Verbalized Top-k Accuracy: 30.1. Result ID: 69ce36d647f34ce924b7b06e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct","Inference Strategy":"+ Verbalized Top-k"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Accuracy: 29.8. Result ID: 69ce36d647f34ce924b7b06a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct","Inference Strategy":"Base"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + Logit Accuracy: 29.8. Result ID: 69ce36d647f34ce924b7b06b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct","Inference Strategy":"+ Logit"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + p(True) Accuracy: 29.8. Result ID: 69ce36d647f34ce924b7b06c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct","Inference Strategy":"+ p(True)"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking + Verbalized Probability Distribution Accuracy: 29. Result ID: 69ce36d647f34ce924b7b091. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"Qwen3-4B-Thinking","Confidence Estimation Strategy":"Verbalized Probability Distribution"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.