List your company
ResearchFormerly WizWand

Medical Question Answering on MMLU-Pro Health

84AccuracyClaude Sonnet 4.5
41.77652.73863.774.662Mar 13, 2026Mar 31, 2026Apr 18, 2026May 7, 2026May 25, 2026Jun 12, 2026Jul 1, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Claude Sonnet 4.5Backbone=Claude Sonnet 4.52026.0784
DeepSeek-R1Backbone=DeepSeek-R12026.0774.2
MedGemma 27BBackbone=MedGemma 27B2026.0772
Qwen3 8BBackbone=Qwen3 8B2026.0769.7
Agentic Search + GRPO (Qwen3 4B)Backbone=Qwen3 4B2026.0766.1
FaithMed (Qwen3 4B)Backbone=Qwen3 4B2026.0763.5
Base (Qwen3 4B)Backbone=Qwen3 4B2026.0763
IDCBackbone=Llama-3.1-8B, Method Type=Iterative Distractor Curation2026.0360.76
FilterBackbone=Llama-3.1-8B, Method Type=Filtering non-convertible questions2026.0359.17
Gemma 3 12BBackbone=Gemma 3 12B2026.0759
BaseBackbone=Llama-3.1-8B, Method Type=Base instruct model2026.0358.56
RewriteBackbone=Llama-3.1-8B, Method Type=Model-based conversion2026.0358.19
DirectBackbone=Llama-3.1-8B, Method Type=Direct conversion to short-answer2026.0357.95
Orig.Backbone=Llama-3.1-8B, Method Type=Standard RLVR on original MCQs2026.0357.46
Llama 3.1 8B InstructBackbone=Llama 3.1 8B2026.0756
MedRAG (Qwen3 4B)Backbone=Qwen3 4B2026.0755.5
Agentic Search (Qwen3 4B)Backbone=Qwen3 4B2026.0755.4
FaithMed (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.0752
IDCBackbone=Qwen2-7B, Method Type=Iterative Distractor Curation2026.0349.51
MedRAG (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.0748.5
Agentic Search + GRPO (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.0748.1
DirectBackbone=Qwen2-7B, Method Type=Direct conversion to short-answer2026.0347.07
Agentic Search (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.0746.4
Orig.Backbone=Qwen2-7B, Method Type=Standard RLVR on original MCQs2026.0346.21
FilterBackbone=Qwen2-7B, Method Type=Filtering non-convertible questions2026.0346.09
RewriteBackbone=Qwen2-7B, Method Type=Model-based conversion2026.0345.72
BaseBackbone=Qwen2-7B, Method Type=Base instruct model2026.0344.38
Base (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.0743.4