# Medical Question Answering on MedMCQA (Pass@1) Source: [Medical Question Answering on MedMCQA (Pass@1)]() Collection index: [llms.txt]() Data updated: 2026-05-28T10:21:12.955Z Evaluation of model performance on the MedMCQA dataset for Medical Question Answering, specifically measured using the Pass@1 metric. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Medical Question Answering | | Dataset (including any stated split) | MedMCQA | | Metric | Pass@1 Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1 Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Qwen2.5-7B + PPO | 53.6 | Not recorded | 68fa7bc11e209131cea8a35b | | Qwen2.5-7B + PPO w/ VERL. | 52.9 | Not recorded | 68fa7bc11e209131cea8a35b | | Qwen2.5-7B + GRPO | 52.1 | Not recorded | 68fa7bc11e209131cea8a35b | | Qwen2.5-7B + GRPO w/ VERL. | 52 | Not recorded | 68fa7bc11e209131cea8a35b | | Qwen2.5-7B | 50.6 | Not recorded | 68fa7bc11e209131cea8a35b | | Qwen3-1.7B | 48.54 | Not recorded | 6a17cc53dbd7150506f815fc | | SHIFT | 48.47 | Not recorded | 6a17cc53dbd7150506f815fc | | SHIFT | 48.3 | Not recorded | 6a17cc53dbd7150506f815fc | | SC-Entropy | 47.37 | Not recorded | 6a17cc53dbd7150506f815fc | | Cluster | 47.3 | Not recorded | 6a17cc53dbd7150506f815fc | | Mathstral-7B-v0.1 + GRPO w/ VERL. | 47.1 | Not recorded | 68fa7bc11e209131cea8a35b | | Random (avg) | 46.97 | Not recorded | 6a17cc53dbd7150506f815fc | | A-PPL | 46.52 | Not recorded | 6a17cc53dbd7150506f815fc | | CoreSet | 46.48 | Not recorded | 6a17cc53dbd7150506f815fc | | SC-Entropy | 46.38 | Not recorded | 6a17cc53dbd7150506f815fc | | CoT Similarity | 46.09 | Not recorded | 6a17cc53dbd7150506f815fc | | Random (avg) | 45.91 | Not recorded | 6a17cc53dbd7150506f815fc | | Cluster | 45.88 | Not recorded | 6a17cc53dbd7150506f815fc | | Q-PPL | 45.88 | Not recorded | 6a17cc53dbd7150506f815fc | | CoT Similarity | 45.85 | Not recorded | 6a17cc53dbd7150506f815fc | | Q-PPL | 45.53 | Not recorded | 6a17cc53dbd7150506f815fc | | A-PPL | 45.17 | Not recorded | 6a17cc53dbd7150506f815fc | | Mathstral-7B-v0.1 + GRPO | 44.9 | Not recorded | 68fa7bc11e209131cea8a35b | | CoreSet | 44.89 | Not recorded | 6a17cc53dbd7150506f815fc | | Mathstral-7B-v0.1 + PPO w/ VERL. | 44.8 | Not recorded | 68fa7bc11e209131cea8a35b | | Mathstral-7B-v0.1 + PPO | 44.2 | Not recorded | 68fa7bc11e209131cea8a35b | | Mathstral-7B-v0.1 | 42.9 | Not recorded | 68fa7bc11e209131cea8a35b | | Qwen3-1.7B | 40.98 | Not recorded | 6a17cc53dbd7150506f815fc | ### Qwen2.5-7B + PPO Pass@1 Accuracy: 53.6. Result ID: 69ddfb2147f34ce924c562ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","Training":"PPO"}. This record does not assert independent reproduction. ### Qwen2.5-7B + PPO w/ VERL. Pass@1 Accuracy: 52.9. Result ID: 69ddfb2147f34ce924c562ae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","Training":"PPO w/ VERL."}. This record does not assert independent reproduction. ### Qwen2.5-7B + GRPO Pass@1 Accuracy: 52.1. Result ID: 69ddfb2147f34ce924c562ab. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","Training":"GRPO"}. This record does not assert independent reproduction. ### Qwen2.5-7B + GRPO w/ VERL. Pass@1 Accuracy: 52. Result ID: 69ddfb2147f34ce924c562ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","Training":"GRPO w/ VERL."}. This record does not assert independent reproduction. ### Qwen2.5-7B Pass@1 Accuracy: 50.6. Result ID: 69ddfb2147f34ce924c562aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B"}. This record does not assert independent reproduction. ### Qwen3-1.7B Pass@1 Accuracy: 48.54. Result ID: 6a181713fbc373592c127b55. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Train Size":"100%"}. This record does not assert independent reproduction. ### SHIFT Pass@1 Accuracy: 48.47. Result ID: 6a181713fbc373592c127b65. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Train Size":"0.2%"}. This record does not assert independent reproduction. ### SHIFT Pass@1 Accuracy: 48.3. Result ID: 6a181713fbc373592c127b5d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Train Size":"0.1%"}. This record does not assert independent reproduction. ### SC-Entropy Pass@1 Accuracy: 47.37. Result ID: 6a181713fbc373592c127b61. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Train Size":"0.2%"}. This record does not assert independent reproduction. ### Cluster Pass@1 Accuracy: 47.3. Result ID: 6a181713fbc373592c127b5f. [Reporting paper]() Recorded evaluation configuration: {"Train Size":"0.2%"}. This record does not assert independent reproduction. ### Mathstral-7B-v0.1 + GRPO w/ VERL. Pass@1 Accuracy: 47.1. Result ID: 69ddfb2147f34ce924c562a7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Mathstral-7B-v0.1","Training":"GRPO w/ VERL."}. This record does not assert independent reproduction. ### Random (avg) Pass@1 Accuracy: 46.97. Result ID: 6a181713fbc373592c127b5e. [Reporting paper]() Recorded evaluation configuration: {"Train Size":"0.2%"}. This record does not assert independent reproduction. ### A-PPL Pass@1 Accuracy: 46.52. Result ID: 6a181713fbc373592c127b64. [Reporting paper]() Recorded evaluation configuration: {"Train Size":"0.2%"}. This record does not assert independent reproduction. ### CoreSet Pass@1 Accuracy: 46.48. Result ID: 6a181713fbc373592c127b60. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Train Size":"0.2%"}. This record does not assert independent reproduction. ### SC-Entropy Pass@1 Accuracy: 46.38. Result ID: 6a181713fbc373592c127b59. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Train Size":"0.1%"}. This record does not assert independent reproduction. ### CoT Similarity Pass@1 Accuracy: 46.09. Result ID: 6a181713fbc373592c127b5a. [Reporting paper]() Recorded evaluation configuration: {"Train Size":"0.1%"}. This record does not assert independent reproduction. ### Random (avg) Pass@1 Accuracy: 45.91. Result ID: 6a181713fbc373592c127b56. [Reporting paper]() Recorded evaluation configuration: {"Train Size":"0.1%"}. This record does not assert independent reproduction. ### Cluster Pass@1 Accuracy: 45.88. Result ID: 6a181713fbc373592c127b57. [Reporting paper]() Recorded evaluation configuration: {"Train Size":"0.1%"}. This record does not assert independent reproduction. ### Q-PPL Pass@1 Accuracy: 45.88. Result ID: 6a181713fbc373592c127b63. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Train Size":"0.2%"}. This record does not assert independent reproduction. ### CoT Similarity Pass@1 Accuracy: 45.85. Result ID: 6a181713fbc373592c127b62. [Reporting paper]() Recorded evaluation configuration: {"Train Size":"0.2%"}. This record does not assert independent reproduction. ### Q-PPL Pass@1 Accuracy: 45.53. Result ID: 6a181713fbc373592c127b5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Train Size":"0.1%"}. This record does not assert independent reproduction. ### A-PPL Pass@1 Accuracy: 45.17. Result ID: 6a181713fbc373592c127b5c. [Reporting paper]() Recorded evaluation configuration: {"Train Size":"0.1%"}. This record does not assert independent reproduction. ### Mathstral-7B-v0.1 + GRPO Pass@1 Accuracy: 44.9. Result ID: 69ddfb2147f34ce924c562a6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Mathstral-7B-v0.1","Training":"GRPO"}. This record does not assert independent reproduction. ### CoreSet Pass@1 Accuracy: 44.89. Result ID: 6a181713fbc373592c127b58. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Train Size":"0.1%"}. This record does not assert independent reproduction. ### Mathstral-7B-v0.1 + PPO w/ VERL. Pass@1 Accuracy: 44.8. Result ID: 69ddfb2147f34ce924c562a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Mathstral-7B-v0.1","Training":"PPO w/ VERL."}. This record does not assert independent reproduction. ### Mathstral-7B-v0.1 + PPO Pass@1 Accuracy: 44.2. Result ID: 69ddfb2147f34ce924c562a8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Mathstral-7B-v0.1","Training":"PPO"}. This record does not assert independent reproduction. ### Mathstral-7B-v0.1 Pass@1 Accuracy: 42.9. Result ID: 69ddfb2147f34ce924c562a5. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Mathstral-7B-v0.1"}. This record does not assert independent reproduction. ### Qwen3-1.7B Pass@1 Accuracy: 40.98. Result ID: 6a181713fbc373592c127b54. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Train Size":"N/A"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.