# Beyond MedQA: Towards Real-world Clinical Decision Making in the Era of LLMs

Source: [Beyond MedQA: Towards Real-world Clinical Decision Making in the Era of LLMs](<https://www.sota2.com/research/paper/68fc59563ef7ccd74e452456>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T11:01:11.336Z

Authors: Yunpeng Xiao, Carl Yang, Mark Mai, Xiao Hu, Kai Shu

Published: 2025-10-22T20:06:10.000Z

## Abstract

Large language models (LLMs) show promise for clinical use. They are often evaluated using datasets such as MedQA. However, Many medical datasets, such as MedQA, rely on simplified Question-Answering (Q\\A) that underrepresents real-world clinical decision-making. Based on this, we propose a unifying paradigm that characterizes clinical decision-making tasks along two dimensions: Clinical Backgrounds and Clinical Questions. As the background and questions approach the real clinical environment, the difficulty increases. We summarize the settings of existing datasets and benchmarks along two dimensions. Then we review methods to address clinical decision-making, including training-time and test-time techniques, and summarize when they help. Next, we extend evaluation beyond accuracy to include efficiency, explainability. Finally, we highlight open challenges. Our paradigm clarifies assumptions, standardizes comparisons, and guides the development of clinically meaningful LLMs.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2510.20001>)

[PDF](<https://arxiv.org/pdf/2510.20001.pdf>)
