List your company
ResearchFormerly WizWand

Medical Question Answering on PubMedQA (Accuracy)

81.4AccuracyHuatuoGPT-o1-70B
47.28856.1446573.856Apr 1, 2025Jun 13, 2025Aug 26, 2025Nov 8, 2025Jan 20, 2026Apr 4, 2026Jun 17, 2026
Updated 4mo ago

Evaluation results

MethodLinks
HuatuoGPT-o1-70BParameter Scale=70B2025.0481.4
HuatuoGPT-o1-8BParameter Scale=8B2025.0480.1
UltraMedical-70B-3Parameter Scale=70B2025.0480
o3Size category=Large Models2025.0780
HuatuoGPT-o1-72BParameter Scale=72B2025.0479.9
OpenBioLLM-70BParameter Scale=70B2025.0479.3
UltraMedical-8B-3.1Parameter Scale=8B2025.0479.2
OpenBioLLM 70BSize category=Large Models2025.0779
RewriteBackbone=Llama-3.1-8B, Method Type=Model-based conversion2026.0378.8
HuatuoGPT-o1-7BParameter Scale=7B2025.0478.6
FilterBackbone=Llama-3.1-8B, Method Type=Filtering non-convertible questions2026.0378.5
GPT-4oSize category=Large Models2025.0778.4
DirectBackbone=Llama-3.1-8B, Method Type=Direct conversion to short-answer2026.0378.3
Med42-70BParameter Scale=70B2025.0478.1
BaseBackbone=Llama-3.1-8B, Method Type=Base instruct model2026.0378
Orig.Backbone=Llama-3.1-8B, Method Type=Standard RLVR on original MCQs2026.0377.8
BioMistral DARE 7BSize category=Small Models2025.0777.7
m1-32B-1KParameter Scale=32B, Thinking Budget=1K2025.0477.6
IDCBackbone=Llama-3.1-8B, Method Type=Iterative Distractor Curation2026.0377.6
MMedS-8BParameter Scale=8B2025.0477.5
m1-7B-1KParameter Scale=7B, Thinking Budget=1K2025.0477.5
DeepSeek R1Size category=Large Models2025.0777.2
MedGemma 27BSize category=Small Models, test-time scaling=true2025.0776.8
IQVIA Med-R1 8BSize category=Small Models2025.0776.4
Gemini 2.5 FlashSize category=Large Models2025.0776.2
Qwen3-8B + GRPOBackbone=Qwen3-8B, Training=GRPO2026.0676.2
Med42-8BParameter Scale=8B2025.0476
m1-7B-23KParameter Scale=7B, Thinking Budget=23K2025.0475.8
Gemini 2.5 ProSize category=Large Models2025.0775.8
MEDIC-ADSize=7B2026.0375.6
MedLlama3-8B-v2Parameter Scale=8B2025.0475.5
LingshuSize=7B2026.0375.4
Qwen3-8B + RCSDBackbone=Qwen3-8B, Training=RCSD2026.0675.1
Citrus-VSize=8B2026.0374.8
Qwen3-8B + OPSDBackbone=Qwen3-8B, Training=OPSD2026.0674.4
JSL-MedLlama 3 8B v2.0Size category=Small Models2025.0774.2
Qwen3-8BBackbone=Qwen3-8B2026.0674.2
Qwen3-8B + GRPO-RubricsBackbone=Qwen3-8B, Training=GRPO-Rubrics2026.0674.2
OpenBioLLM 8BSize category=Small Models2025.0774.1
IDCBackbone=Qwen2-7B, Method Type=Iterative Distractor Curation2026.0374
MedGemma 4BSize category=Small Models2025.0773.4
Gemma 3 27BSize category=Small Models2025.0773.4
FilterBackbone=Qwen2-7B, Method Type=Filtering non-convertible questions2026.0372.9
Qwen2.5-7B-InstructParameter Scale=7B, Chain-of-Thought (CoT) Prompting=true2025.0472.6
Orig.Backbone=Qwen2-7B, Method Type=Standard RLVR on original MCQs2026.0372.5
DirectBackbone=Qwen2-7B, Method Type=Direct conversion to short-answer2026.0372
Qwen2.5-7B-InstructParameter Scale=7B, Chain-of-Thought (CoT) Prompting=false2025.0471.3
Qwen2.5-72B-InstructParameter Scale=72B, Chain-of-Thought (CoT) Prompting=true2025.0471.3
RewriteBackbone=Qwen2-7B, Method Type=Model-based conversion2026.0371.2
UltraMedical-8B-3Parameter Scale=8B2025.0471.1
Qwen2.5-72B-InstructParameter Scale=72B, Chain-of-Thought (CoT) Prompting=false2025.0470.8
OpenBioLLM-8BParameter Scale=8B2025.0470.1
BaseBackbone=Qwen2-7B, Method Type=Base instruct model2026.0369.5
ReConcileBase Model=Gemma-3-4B2025.0869.3
Qwen2.5-32B-InstructParameter Scale=32B, Chain-of-Thought (CoT) Prompting=true2025.0468.9
TMA-AllComponBase Model=Gemma-3-4B2025.0868.7
Gemma 3 4BSize category=Small Models2025.0768.4
Qwen2.5-32B-InstructParameter Scale=32B, Chain-of-Thought (CoT) Prompting=false2025.0468
Single-Agent BestBase Model=Gemma-3-4B2025.0866.6
MMed-8B-EnInsParameter Scale=8B2025.0463.8
MMed-8BParameter Scale=8B2025.0463.4
ORMModel Size=8B2026.0362
OVM2026.0359.4
SEMA-RAGBackbone=deepseek-v3.12026.0559.2
SEMA-RAGBackbone=gemini-2.0-flash2026.0559.2
MCNIGModel Size=8B2026.0357.7
ReFilterBackbone=Qwen2.5-14B-Instruct, Zero-shot evaluation=true2026.0256.8
DyLanBase Model=Gemma-3-4B2025.0856.7
MedAgentsBase Model=Gemma-3-4B2025.0856.1
ReFilterBackbone=LLaMA-3-8B-Instruct, Zero-shot evaluation=true2026.0256
SEMA-RAGBackbone=qwen3-coder-plus2026.0556
SEMA-RAGBackbone=kimi-k22026.0555.8
ReFilterBackbone=Qwen2.5-1.5B-Instruct, Zero-shot evaluation=true2026.0255.2
S-RAGBackbone=Qwen2.5-14B-Instruct, Zero-shot evaluation=true2026.0255.2
DyPRAGBackbone=Qwen2.5-14B-Instruct, Zero-shot evaluation=true2026.0255.2
DyPRAGBackbone=LLaMA-3-8B-Instruct, Zero-shot evaluation=true2026.0254.8
i-MedRAGBackbone=kimi-k22026.0554.6
S-RAGBackbone=LLaMA-3-8B-Instruct, Zero-shot evaluation=true2026.0254.2
QwenPRMModel Size=7B2026.0354.1
PRAGBackbone=LLaMA-3-8B-Instruct, Zero-shot evaluation=true2026.0254
VanillaBackbone=Qwen2.5-14B-Instruct, Zero-shot evaluation=true2026.0254
PRAGBackbone=Qwen2.5-14B-Instruct, Zero-shot evaluation=true2026.0254
CoTBackbone=kimi-k22026.0553.6
MedLlama3-8B-v1Parameter Scale=8B2025.0452.7
MedRAGBackbone=kimi-k22026.0552.6
VanillaBackbone=LLaMA-3-8B-Instruct, Zero-shot evaluation=true2026.0251.8
PRAGBackbone=Qwen2.5-1.5B-Instruct, Zero-shot evaluation=true2026.0251.8
S-RAGBackbone=Qwen2.5-1.5B-Instruct, Zero-shot evaluation=true2026.0251.6
i-MedRAGBackbone=gemini-2.0-flash2026.0551.2
SEMA-RAGBackbone=glm-4.0-flash2026.0551.2
i-MedRAGBackbone=deepseek-v3.12026.0550.6
MedCPTBackbone=kimi-k22026.0550.2
DyPRAGBackbone=Qwen2.5-1.5B-Instruct, Zero-shot evaluation=true2026.0250
VanillaBackbone=Qwen2.5-1.5B-Instruct, Zero-shot evaluation=true2026.0249.8
Granite PRM v22026.0349.8
MathShepherd2026.0349.7
MedRAGBackbone=qwen3-coder-plus2026.0549.2
IGModel Size=8B2026.0349
MDAgentsBase Model=Gemma-3-4B2025.0848.7
i-MedRAGBackbone=qwen3-coder-plus2026.0548.6