Medical Question Answering on MedQA (Accuracy, Tokens)
66.61AccuracyHSIR-DPO
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| HSIR-DPOBackbone=Qwen2.5-3B, Initial Model Setup=SFT-Oracle (M0)2026.05 | 66.61 | 1,116 | |
| HSIR-SFTBackbone=Qwen2.5-3B, Initial Model Setup=SFT-Oracle (M0)2026.05 | 63.71 | 1,116 | |
| ReSTEMBackbone=Qwen2.5-3B, Initial Model Setup=SFT-Oracle (M0)2026.05 | 61.81 | 1,243 | |
| IRPOBackbone=Qwen2.5-3B, Initial Model Setup=SFT-Oracle (M0)2026.05 | 59.46 | 1,585 | |
| SFT-OracleBackbone=Qwen2.5-3B2026.05 | 58.68 | 1,448 | |
| HSIR-DPOBackbone=Qwen2.5-1.5B, Initial Model Setup=SFT-Oracle (M0)2026.05 | 53.49 | 1,047 | |
| HSIR-SFTBackbone=Qwen2.5-1.5B, Initial Model Setup=SFT-Oracle (M0)2026.05 | 50.9 | 1,156 | |
| ReSTEMBackbone=Qwen2.5-1.5B, Initial Model Setup=SFT-Oracle (M0)2026.05 | 47.21 | 1,423 | |
| SFT-OracleBackbone=Qwen2.5-1.5B2026.05 | 46.58 | 1,678 | |
| IRPOBackbone=Qwen2.5-1.5B, Initial Model Setup=SFT-Oracle (M0)2026.05 | 46.35 | 1,683 |