Medical Reasoning on Medicine MedQA M-Med
75.2MedQA ScoreSearch-o1
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Search-o1Backbone=GPT-4o-mini, Category=Advanced RAG Methods2025.08 | 75.2 | 66.6 | 70.9 | |
| RATBackbone=GPT-4o-mini, Category=Advanced RAG Methods2025.08 | 74.4 | 70.6 | 72.5 | |
| Self-AskBackbone=GPT-4o-mini, Category=Advanced RAG Methods2025.08 | 72.4 | 68 | 70.2 | |
| CoTBackbone=GPT-4o-mini2025.08 | 71.4 | 67 | 69.2 | |
| Standard RAGBackbone=GPT-4o-mini2025.08 | 70.6 | 64.2 | 67.4 | |
| UR2 (GPT Summary)Backbone=Qwen-2.5-7B2025.08 | 69.6 | 62.8 | 66.2 | |
| UR2 (GPT summary)Backbone=LLaMA-3.1-8B2025.08 | 68.6 | 58.4 | 63.5 | |
| UR2Model=LLaMA-3.1-8B2025.08 | 68.6 | 58.4 | 63.5 | |
| UR2 (qw Summary)Backbone=Qwen-2.5-7B2025.08 | 67.2 | 60.8 | 64 | |
| Vanilla RLBackbone=LLaMA-3.1-8B2025.08 | 66.8 | 57.4 | 62.1 | |
| Vanilla RLModel=LLaMA-3.1-8B2025.08 | 66.8 | 57.4 | 62.1 | |
| UR2 (qw summary)Backbone=LLaMA-3.1-8B2025.08 | 65.8 | 56.2 | 61 | |
| Vanilla RLBackbone=Qwen-2.5-7B2025.08 | 64.2 | 57.4 | 60.8 | |
| RATModel=LLaMA-3.1-8B2025.08 | 63.8 | 56 | 59.9 | |
| Open-Reasoner-ZeroBackbone=Qwen-2.5-7B, Category=CoT-RL Methods2025.08 | 61.6 | 58.8 | 60.2 | |
| RATModel=Qwen-2.5-7B2025.08 | 60 | 53.2 | 56.6 | |
| UR2 (GPT summary)Backbone=Qwen-2.5-3B2025.08 | 59.8 | 56.8 | 58.3 | |
| UR2Model=Qwen-2.5-3B2025.08 | 59.8 | 56.8 | 58.3 | |
| CoTBackbone=LLaMA-3.1-8B2025.08 | 59.6 | 52.6 | 56.1 | |
| CoTModel=LLaMA-3.1-8B2025.08 | 59.6 | 52.6 | 56.1 | |
| General ReasonerBackbone=Qwen-2.5-7B, Category=CoT-RL Methods2025.08 | 58.4 | 54.4 | 56.4 | |
| Search-o1Model=Qwen-2.5-7B2025.08 | 58.2 | 52.8 | 55.6 | |
| CoTBackbone=Qwen-2.5-7B2025.08 | 57.2 | 52 | 54.6 | |
| SimpleRL-ZooBackbone=Qwen-2.5-7B, Category=CoT-RL Methods2025.08 | 57.2 | 51 | 54.1 | |
| Standard RAGBackbone=LLaMA-3.1-8B2025.08 | 56.4 | 53.2 | 54.8 | |
| Standard RAGModel=LLaMA-3.1-8B2025.08 | 56.4 | 53.2 | 54.8 | |
| Search-o1Model=LLaMA-3.1-8B2025.08 | 56 | 46 | 56.6 | |
| UR2 (qw summary)Backbone=Qwen-2.5-3B2025.08 | 55.2 | 54 | 54.6 | |
| Standard RAGBackbone=Qwen-2.5-7B2025.08 | 54.2 | 53.2 | 53.7 | |
| Self-AskModel=LLaMA-3.1-8B2025.08 | 53 | 42.8 | 47.9 | |
| Vanilla RLBackbone=Qwen-2.5-3B2025.08 | 51.8 | 47.6 | 49.7 | |
| Self-AskModel=Qwen-2.5-7B2025.08 | 51.8 | 47.8 | 49.8 | |
| Vanilla RLModel=Qwen-2.5-3B2025.08 | 51.8 | 47.6 | 49.7 | |
| RATModel=Qwen-2.5-3B2025.08 | 47.4 | 42.6 | 45 | |
| Standard RAGBackbone=Qwen-2.5-3B2025.08 | 45.6 | 40 | 42.8 | |
| Standard RAGModel=Qwen-2.5-3B2025.08 | 45.6 | 40 | 42.8 | |
| Search-o1Model=Qwen-2.5-3B2025.08 | 44.6 | 37.4 | 41 | |
| Self-AskModel=Qwen-2.5-3B2025.08 | 39.6 | 36 | 37.8 | |
| CoTBackbone=Qwen-2.5-3B2025.08 | 39.4 | 36.8 | 38.1 | |
| CoTModel=Qwen-2.5-3B2025.08 | 39.4 | 36.8 | 38.1 |