Question Answering on CMB Exam
59.33Accuracy (EM)ProMed(S#1+2)
Evaluation Results
| Method | Links | |
|---|---|---|
| ProMed(S#1+2)Backbone=LLaMA3.1-8B, Method Type=SFT+RL2025.08 | 59.33 | |
| ProMed(S#1+2)Backbone=Qwen3-1.7B, Method Type=SFT+RL2025.08 | 51.98 | |
| ProMed(S#1)Backbone=LLaMA3.1-8B, Method Type=SFT2025.08 | 51.78 | |
| ProMed(S#1)+GRPOBackbone=LLaMA3.1-8B, Method Type=SFT+RL2025.08 | 51.43 | |
| MCTS-MVBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 50.44 | |
| SFT-GTBackbone=LLaMA3.1-8B, Method Type=SFT2025.08 | 49.93 | |
| ProMed(S#1)+DPOBackbone=LLaMA3.1-8B, Method Type=SFT+RL2025.08 | 47.87 | |
| SFTBackbone=LLaMA3.1-8B, Method Type=SFT2025.08 | 47.15 | |
| ProMed(S#1)+GRPOBackbone=Qwen3-1.7B, Method Type=SFT+RL2025.08 | 46.61 | |
| ProMed(S#1+2)Backbone=LLaMA3.2-3B, Method Type=SFT+RL2025.08 | 46.25 | |
| ProMed(S#1)Backbone=Qwen3-1.7B, Method Type=SFT2025.08 | 45.69 | |
| ProMed(S#1)Backbone=LLaMA3.2-3B, Method Type=SFT2025.08 | 45.07 | |
| ProMed(S#1)+GRPOBackbone=LLaMA3.2-3B, Method Type=SFT+RL2025.08 | 44.76 | |
| CoTBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 44.75 | |
| VanillaBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 44.49 | |
| SFTBackbone=Qwen3-1.7B, Method Type=SFT2025.08 | 44.3 | |
| DirectBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 44.1 | |
| UoTBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 44.04 | |
| UoTBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 43.79 | |
| SFT-GTBackbone=Qwen3-1.7B, Method Type=SFT2025.08 | 43.65 | |
| SFTBackbone=LLaMA3.2-3B, Method Type=SFT2025.08 | 43.6 | |
| DirectBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 43.1 | |
| MCTS-BTBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 42.95 | |
| ProMed(S#1)+DPOBackbone=LLaMA3.2-3B, Method Type=SFT+RL2025.08 | 42.87 | |
| ProMed(S#1)+DPOBackbone=Qwen3-1.7B, Method Type=SFT+RL2025.08 | 42.42 | |
| SFT-GTBackbone=LLaMA3.2-3B, Method Type=SFT2025.08 | 42.37 | |
| UoTBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 40.57 | |
| DialogTBackbone=LLaMA3.1-8B, Method Type=SFT2025.08 | 38.37 | |
| MCTS-MVBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 37.93 | |
| CoTBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 34.79 | |
| DialogTBackbone=LLaMA3.2-3B, Method Type=SFT2025.08 | 34.07 | |
| VanillaBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 33.45 | |
| MEDIQBackbone=LLaMA3.1-8B, Method Type=Prompt2025.08 | 33.16 | |
| DialogTBackbone=Qwen3-1.7B, Method Type=SFT2025.08 | 32.43 | |
| MEDIQBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 31.13 | |
| CoTBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 31.04 | |
| MCTS-BTBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 30.28 | |
| MCTS-MVBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 30.03 | |
| VanillaBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 29.01 | |
| MCTS-BTBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 21.5 | |
| DirectBackbone=Qwen3-1.7B, Method Type=Prompt2025.08 | 19.34 | |
| MEDIQBackbone=LLaMA3.2-3B, Method Type=Prompt2025.08 | 15.97 |