List your company
ResearchFormerly WizWand

Medical Question Answering on MedQA (Accuracy, Tokens)

66.61AccuracyHSIR-DPO
45.539651.009856.4861.9502May 24, 2026
Updated 5mo ago

Evaluation results

MethodLinks
HSIR-DPOBackbone=Qwen2.5-3B, Initial Model Setup=SFT-Oracle (M0)2026.0566.611,116
HSIR-SFTBackbone=Qwen2.5-3B, Initial Model Setup=SFT-Oracle (M0)2026.0563.711,116
ReSTEMBackbone=Qwen2.5-3B, Initial Model Setup=SFT-Oracle (M0)2026.0561.811,243
IRPOBackbone=Qwen2.5-3B, Initial Model Setup=SFT-Oracle (M0)2026.0559.461,585
SFT-OracleBackbone=Qwen2.5-3B2026.0558.681,448
HSIR-DPOBackbone=Qwen2.5-1.5B, Initial Model Setup=SFT-Oracle (M0)2026.0553.491,047
HSIR-SFTBackbone=Qwen2.5-1.5B, Initial Model Setup=SFT-Oracle (M0)2026.0550.91,156
ReSTEMBackbone=Qwen2.5-1.5B, Initial Model Setup=SFT-Oracle (M0)2026.0547.211,423
SFT-OracleBackbone=Qwen2.5-1.5B2026.0546.581,678
IRPOBackbone=Qwen2.5-1.5B, Initial Model Setup=SFT-Oracle (M0)2026.0546.351,683