List your company
ResearchFormerly WizWand

Question Answering on MedQA (EM Accuracy Only)

55.6EM AccuracyProMed(S#1+2)
26.355233.947641.5449.1324Aug 19, 2025
Updated 4mo ago

Evaluation results

MethodLinks
ProMed(S#1+2)Backbone=LLaMA3.1-8B, Method Type=SFT+RL2025.0855.6
ProMed(S#1)+GRPOBackbone=LLaMA3.1-8B, Method Type=SFT+RL2025.0854.6
ProMed(S#1)+DPOBackbone=LLaMA3.1-8B, Method Type=SFT+RL2025.0852.8
ProMed(S#1)Backbone=LLaMA3.1-8B, Method Type=SFT2025.0852.63
SFTBackbone=LLaMA3.1-8B, Method Type=SFT2025.0849.39
SFT-GTBackbone=LLaMA3.1-8B, Method Type=SFT2025.0848.74
DirectBackbone=LLaMA3.1-8B, Method Type=Prompt2025.0848.32
ProMed(S#1+2)Backbone=LLaMA3.2-3B, Method Type=SFT+RL2025.0847.38
ProMed(S#1)+GRPOBackbone=LLaMA3.2-3B, Method Type=SFT+RL2025.0846.32
MEDIQBackbone=LLaMA3.1-8B, Method Type=Prompt2025.0844.78
ProMed(S#1)Backbone=LLaMA3.2-3B, Method Type=SFT2025.0843.69
ProMed(S#1)+DPOBackbone=LLaMA3.2-3B, Method Type=SFT+RL2025.0843.44
CoTBackbone=LLaMA3.1-8B, Method Type=Prompt2025.0843.12
SFT-GTBackbone=LLaMA3.2-3B, Method Type=SFT2025.0842.76
MCTS-MVBackbone=LLaMA3.1-8B, Method Type=Prompt2025.0842.5
SFTBackbone=LLaMA3.2-3B, Method Type=SFT2025.0841.83
VanillaBackbone=LLaMA3.1-8B, Method Type=Prompt2025.0840.11
ProMed(S#1+2)Backbone=Qwen3-1.7B, Method Type=SFT+RL2025.0839.93
ProMed(S#1)+DPOBackbone=Qwen3-1.7B, Method Type=SFT+RL2025.0838.06
ProMed(S#1)+GRPOBackbone=Qwen3-1.7B, Method Type=SFT+RL2025.0837.62
ProMed(S#1)Backbone=Qwen3-1.7B, Method Type=SFT2025.0837.61
MEDIQBackbone=Qwen3-1.7B, Method Type=Prompt2025.0837.13
UoTBackbone=Qwen3-1.7B, Method Type=Prompt2025.0836.94
SFTBackbone=Qwen3-1.7B, Method Type=SFT2025.0836.85
UoTBackbone=LLaMA3.2-3B, Method Type=Prompt2025.0836.71
UoTBackbone=LLaMA3.1-8B, Method Type=Prompt2025.0836.68
DirectBackbone=Qwen3-1.7B, Method Type=Prompt2025.0836.34
MEDIQBackbone=LLaMA3.2-3B, Method Type=Prompt2025.0835.73
CoTBackbone=LLaMA3.2-3B, Method Type=Prompt2025.0835.61
MCTS-BTBackbone=LLaMA3.1-8B, Method Type=Prompt2025.0834.41
SFT-GTBackbone=Qwen3-1.7B, Method Type=SFT2025.0833.49
DialogTBackbone=LLaMA3.1-8B, Method Type=SFT2025.0833.42
MCTS-MVBackbone=Qwen3-1.7B, Method Type=Prompt2025.0833.39
VanillaBackbone=Qwen3-1.7B, Method Type=Prompt2025.0831.05
VanillaBackbone=LLaMA3.2-3B, Method Type=Prompt2025.0830.81
MCTS-MVBackbone=LLaMA3.2-3B, Method Type=Prompt2025.0830.64
MCTS-BTBackbone=LLaMA3.2-3B, Method Type=Prompt2025.0830.24
MCTS-BTBackbone=Qwen3-1.7B, Method Type=Prompt2025.0829.77
CoTBackbone=Qwen3-1.7B, Method Type=Prompt2025.0829.54
DialogTBackbone=LLaMA3.2-3B, Method Type=SFT2025.0828.83
DialogTBackbone=Qwen3-1.7B, Method Type=SFT2025.0828.54
DirectBackbone=LLaMA3.2-3B, Method Type=Prompt2025.0827.48