# Medical Question Answering on HealthBench Hard Source: [Medical Question Answering on HealthBench Hard]() Collection index: [llms.txt]() Data updated: 2026-04-14T12:41:28.198Z Evaluation of model performance on the HealthBench Hard dataset for Medical Question Answering. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Medical Question Answering | | Dataset (including any stated split) | HealthBench Hard | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 39 of 39 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [Score](): Higher is better; unit: SCALAR. - [Pass@1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Pass | 39.02 | Not recorded | 69ddca827d0172140ec7a018 | | Pass | 30.62 | Not recorded | 69ddca827d0172140ec7a018 | | AggAgent | 24.46 | Not recorded | 69ddca827d0172140ec7a018 | | AggAgent | 24.27 | Not recorded | 69ddca827d0172140ec7a018 | | SolAgg | 21.84 | Not recorded | 69ddca827d0172140ec7a018 | | SolAgg | 21.25 | Not recorded | 69ddca827d0172140ec7a018 | | Pass | 21.09 | Not recorded | 69ddca827d0172140ec7a018 | | AggAgent | 19.25 | Not recorded | 69ddca827d0172140ec7a018 | | SolAgg | 17.3 | Not recorded | 69ddca827d0172140ec7a018 | | SummAgg | 16.92 | Not recorded | 69ddca827d0172140ec7a018 | | SummAgg | 12.37 | Not recorded | 69ddca827d0172140ec7a018 | | BoN | 10.58 | Not recorded | 69ddca827d0172140ec7a018 | | BoN | 9.74 | Not recorded | 69ddca827d0172140ec7a018 | | SummAgg | 9.74 | Not recorded | 69ddca827d0172140ec7a018 | | Pass | 9.67 | Not recorded | 69ddca827d0172140ec7a018 | | FewTool | 9.02 | Not recorded | 69ddca827d0172140ec7a018 | | BoN | 8.79 | Not recorded | 69ddca827d0172140ec7a018 | | FewTool | 8.04 | Not recorded | 69ddca827d0172140ec7a018 | | FewTool | 5.34 | Not recorded | 69ddca827d0172140ec7a018 | ### Pass Accuracy: 39.02. Result ID: 69de35e447f34ce924c71b9a. [Reporting paper]() Recorded evaluation configuration: {"K":"8"}. This record does not assert independent reproduction. ### Pass Accuracy: 30.62. Result ID: 69de35e447f34ce924c71b94. [Reporting paper]() Recorded evaluation configuration: {"K":"4"}. This record does not assert independent reproduction. ### AggAgent Accuracy: 24.46. Result ID: 69de35e447f34ce924c71b9f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"8"}. This record does not assert independent reproduction. ### AggAgent Accuracy: 24.27. Result ID: 69de35e447f34ce924c71b99. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"4"}. This record does not assert independent reproduction. ### SolAgg Accuracy: 21.84. Result ID: 69de35e447f34ce924c71b9d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"8"}. This record does not assert independent reproduction. ### SolAgg Accuracy: 21.25. Result ID: 69de35e447f34ce924c71b97. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"4"}. This record does not assert independent reproduction. ### Pass Accuracy: 21.09. Result ID: 69de35e447f34ce924c71b8e. [Reporting paper]() Recorded evaluation configuration: {"K":"2"}. This record does not assert independent reproduction. ### AggAgent Accuracy: 19.25. Result ID: 69de35e447f34ce924c71b93. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"2"}. This record does not assert independent reproduction. ### SolAgg Accuracy: 17.3. Result ID: 69de35e447f34ce924c71b91. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"2"}. This record does not assert independent reproduction. ### SummAgg Accuracy: 16.92. Result ID: 69de35e447f34ce924c71b9e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"8"}. This record does not assert independent reproduction. ### SummAgg Accuracy: 12.37. Result ID: 69de35e447f34ce924c71b98. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"4"}. This record does not assert independent reproduction. ### BoN Accuracy: 10.58. Result ID: 69de35e447f34ce924c71b95. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"4"}. This record does not assert independent reproduction. ### BoN Accuracy: 9.74. Result ID: 69de35e447f34ce924c71b8f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"2"}. This record does not assert independent reproduction. ### SummAgg Accuracy: 9.74. Result ID: 69de35e447f34ce924c71b92. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"2"}. This record does not assert independent reproduction. ### Pass Accuracy: 9.67. Result ID: 69de35e447f34ce924c71b8d. [Reporting paper]() Recorded evaluation configuration: {"K":"1"}. This record does not assert independent reproduction. ### FewTool Accuracy: 9.02. Result ID: 69de35e447f34ce924c71b90. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"2"}. This record does not assert independent reproduction. ### BoN Accuracy: 8.79. Result ID: 69de35e447f34ce924c71b9b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"8"}. This record does not assert independent reproduction. ### FewTool Accuracy: 8.04. Result ID: 69de35e447f34ce924c71b96. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"4"}. This record does not assert independent reproduction. ### FewTool Accuracy: 5.34. Result ID: 69de35e447f34ce924c71b9c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"K":"8"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.