# Medical Question Answering on MedAgentsBench Source: [Medical Question Answering on MedAgentsBench]() Collection index: [llms.txt]() Data updated: 2026-03-24T17:08:17.157Z Evaluation of model performance on the MedAgentsBench dataset for Medical Question Answering, reporting scores across sub-components including MedBullets, MedQA, MedMCQA, MedExQA, and MedXpertQA, along with the overall average. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Medical Question Answering | | Dataset (including any stated split) | MedAgentsBench | | Metric | MedBullets Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MedBullets Score](): Higher is better; unit: PERCENT. - [MedQA Score](): Higher is better; unit: PERCENT. - [MedMCQA Score](): Higher is better; unit: PERCENT. - [MedExQA Score](): Higher is better; unit: PERCENT. - [MedXpertQA Score](): Higher is better; unit: PERCENT. - [Overall Average Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | MedBullets Score | Result date | Report paper | | --- | --- | --- | --- | | GSEM | 59.55 | Not recorded | 69c21ad30636e5bf4afc4d18 | | Mem0g | 58.43 | Not recorded | 69c21ad30636e5bf4afc4d18 | | Vanilla | 55.06 | Not recorded | 69c21ad30636e5bf4afc4d18 | | Mem0 | 53.93 | Not recorded | 69c21ad30636e5bf4afc4d18 | | GraphRAG | 52.81 | Not recorded | 69c21ad30636e5bf4afc4d18 | | A-Mem | 52.81 | Not recorded | 69c21ad30636e5bf4afc4d18 | | ReMe | 51.69 | Not recorded | 69c21ad30636e5bf4afc4d18 | | Mem0g | 50.56 | Not recorded | 69c21ad30636e5bf4afc4d18 | | Naïve RAG | 50.56 | Not recorded | 69c21ad30636e5bf4afc4d18 | | GSEM | 49.43 | Not recorded | 69c21ad30636e5bf4afc4d18 | | Vanilla | 48.31 | Not recorded | 69c21ad30636e5bf4afc4d18 | | A-Mem | 48.31 | Not recorded | 69c21ad30636e5bf4afc4d18 | | Naïve RAG | 47.19 | Not recorded | 69c21ad30636e5bf4afc4d18 | | ReMe | 47.19 | Not recorded | 69c21ad30636e5bf4afc4d18 | | FLEX | 43.82 | Not recorded | 69c21ad30636e5bf4afc4d18 | | GraphRAG | 42.7 | Not recorded | 69c21ad30636e5bf4afc4d18 | | FLEX | 40.45 | Not recorded | 69c21ad30636e5bf4afc4d18 | | Mem0 | 39.32 | Not recorded | 69c21ad30636e5bf4afc4d18 | ### GSEM MedBullets Score: 59.55. Result ID: 69c2c4e947f34ce924ac820e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3.5-35B"}. This record does not assert independent reproduction. ### Mem0g MedBullets Score: 58.43. Result ID: 69c2c4e947f34ce924ac820a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3.5-35B"}. This record does not assert independent reproduction. ### Vanilla MedBullets Score: 55.06. Result ID: 69c2c4e947f34ce924ac8206. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3.5-35B"}. This record does not assert independent reproduction. ### Mem0 MedBullets Score: 53.93. Result ID: 69c2c4e947f34ce924ac8209. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3.5-35B"}. This record does not assert independent reproduction. ### GraphRAG MedBullets Score: 52.81. Result ID: 69c2c4e947f34ce924ac8208. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3.5-35B"}. This record does not assert independent reproduction. ### A-Mem MedBullets Score: 52.81. Result ID: 69c2c4e947f34ce924ac820b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3.5-35B"}. This record does not assert independent reproduction. ### ReMe MedBullets Score: 51.69. Result ID: 69c2c4e947f34ce924ac820c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3.5-35B"}. This record does not assert independent reproduction. ### Mem0g MedBullets Score: 50.56. Result ID: 69c2c4e947f34ce924ac8201. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3.2"}. This record does not assert independent reproduction. ### Naïve RAG MedBullets Score: 50.56. Result ID: 69c2c4e947f34ce924ac8207. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3.5-35B"}. This record does not assert independent reproduction. ### GSEM MedBullets Score: 49.43. Result ID: 69c2c4e947f34ce924ac8205. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3.2"}. This record does not assert independent reproduction. ### Vanilla MedBullets Score: 48.31. Result ID: 69c2c4e947f34ce924ac81fd. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3.2"}. This record does not assert independent reproduction. ### A-Mem MedBullets Score: 48.31. Result ID: 69c2c4e947f34ce924ac8202. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3.2"}. This record does not assert independent reproduction. ### Naïve RAG MedBullets Score: 47.19. Result ID: 69c2c4e947f34ce924ac81fe. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3.2"}. This record does not assert independent reproduction. ### ReMe MedBullets Score: 47.19. Result ID: 69c2c4e947f34ce924ac8203. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3.2"}. This record does not assert independent reproduction. ### FLEX MedBullets Score: 43.82. Result ID: 69c2c4e947f34ce924ac8204. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3.2"}. This record does not assert independent reproduction. ### GraphRAG MedBullets Score: 42.7. Result ID: 69c2c4e947f34ce924ac81ff. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3.2"}. This record does not assert independent reproduction. ### FLEX MedBullets Score: 40.45. Result ID: 69c2c4e947f34ce924ac820d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3.5-35B"}. This record does not assert independent reproduction. ### Mem0 MedBullets Score: 39.32. Result ID: 69c2c4e947f34ce924ac8200. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3.2"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.