# ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoning

Source: [ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoning](<https://www.sota2.com/research/paper/68fa7c021e209131cea8bd65>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:54:15.806Z

Authors: Yu Sun, Xingyu Qian, Weiwen Xu, Hao Zhang, Chenghao Xiao, Long Li, Deli Zhao, Wenbing Huang, Tingyang Xu, Qifeng Bai, Yu Rong

Published: 2025-06-11T08:36:55.000Z

## Abstract

Reasoning-based large language models have excelled in mathematics and programming, yet their potential in knowledge-intensive medical question answering remains underexplored and insufficiently validated in clinical contexts. To bridge this gap, we introduce ReasonMed, the largest medical reasoning dataset to date, comprising 370k high-quality examples distilled from 1.75 million initial reasoning paths generated by complementary LLMs and curated through a cost-efficient easy-medium-difficult (EMD) pipeline. ReasonMed is built through a multi-agent generation, verification, and refinement process, in which an Error Refiner improves reasoning paths by correcting error-prone steps identified by a verifier. Using ReasonMed, we investigate effective strategies for training medical reasoning models and find that integrating detailed CoT reasoning with concise answer summaries yields the most robust fine-tuning results. Models trained on ReasonMed set a new benchmark: ReasonMed-7B surpasses the prior best sub-10B models by 4.17% and even exceeds LLaMA3.1-70B on PubMedQA by 4.60%. When scaled to ReasonMed-14B, it remains highly competitive, underscoring consistent scaling potential. The codes and datasets are available at https://github.com/YuSun-Work/ReasonMed.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Medical Reasoning on MMLU (val)](<https://www.sota2.com/research/sota/medical-reasoning-on-mmlu-val.md>): Medical Reasoning; MMLU (val); Anatomy (AN) Accuracy: 75.6.

- [Medical Question Answering on MedMCQA](<https://www.sota2.com/research/sota/medical-question-answering-on-medmcqa.md>): Medical Question Answering; MedMCQA; Accuracy: 65.1.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2506.09513>)

[PDF](<https://arxiv.org/pdf/2506.09513.pdf>)
