Question Answering on MedQA (EM Accuracy Only)
55.6EM AccuracyProMed(S#1+2)
Evaluation Results
| Method | Links | |
|---|---|---|
| ProMed(S#1+2)Backbone=LLaMA3.1-8B, Method Type=SFT+RL2025.08 | 55.6 | |
| ProMed(S#1)+GRPOBackbone=LLaMA3.1-8B, Method Type=SFT+RL2025.08 | 54.6 | |
| ProMed(S#1)+DPOBackbone=LLaMA3.1-8B, Method Type=SFT+RL2025.08 | 52.8 | |
| ProMed(S#1)Backbone=LLaMA3.1-8B, Method Type=SFT2025.08 | 52.63 | |
| SFTBackbone=LLaMA3.1-8B, Method Type=SFT2025.08 | 49.39 | |
| SFT-GTBackbone=LLaMA3.1-8B, Method Type=SFT2025.08 | 48.74 | |
| DirectBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 48.32 | |
| ProMed(S#1+2)Backbone=LLaMA3.2-3B, Method Type=SFT+RL2025.08 | 47.38 | |
| ProMed(S#1)+GRPOBackbone=LLaMA3.2-3B, Method Type=SFT+RL2025.08 | 46.32 | |
| MEDIQBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 44.78 | |
| ProMed(S#1)Backbone=LLaMA3.2-3B, Method Type=SFT2025.08 | 43.69 | |
| ProMed(S#1)+DPOBackbone=LLaMA3.2-3B, Method Type=SFT+RL2025.08 | 43.44 | |
| CoTBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 43.12 | |
| SFT-GTBackbone=LLaMA3.2-3B, Method Type=SFT2025.08 | 42.76 | |
| MCTS-MVBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 42.5 | |
| SFTBackbone=LLaMA3.2-3B, Method Type=SFT2025.08 | 41.83 | |
| VanillaBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 40.11 | |
| ProMed(S#1+2)Backbone=Qwen3-1.7B, Method Type=SFT+RL2025.08 | 39.93 | |
| ProMed(S#1)+DPOBackbone=Qwen3-1.7B, Method Type=SFT+RL2025.08 | 38.06 | |
| ProMed(S#1)+GRPOBackbone=Qwen3-1.7B, Method Type=SFT+RL2025.08 | 37.62 | |
| ProMed(S#1)Backbone=Qwen3-1.7B, Method Type=SFT2025.08 | 37.61 | |
| MEDIQBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 37.13 | |
| UoTBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 36.94 | |
| SFTBackbone=Qwen3-1.7B, Method Type=SFT2025.08 | 36.85 | |
| UoTBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 36.71 | |
| UoTBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 36.68 | |
| DirectBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 36.34 | |
| MEDIQBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 35.73 | |
| CoTBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 35.61 | |
| MCTS-BTBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 34.41 | |
| SFT-GTBackbone=Qwen3-1.7B, Method Type=SFT2025.08 | 33.49 | |
| DialogTBackbone=LLaMA3.1-8B, Method Type=SFT2025.08 | 33.42 | |
| MCTS-MVBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 33.39 | |
| VanillaBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 31.05 | |
| VanillaBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 30.81 | |
| MCTS-MVBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 30.64 | |
| MCTS-BTBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 30.24 | |
| MCTS-BTBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 29.77 | |
| CoTBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 29.54 | |
| DialogTBackbone=LLaMA3.2-3B, Method Type=SFT2025.08 | 28.83 | |
| DialogTBackbone=Qwen3-1.7B, Method Type=SFT2025.08 | 28.54 | |
| DirectBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 27.48 |