Question Answering on HeadQA
93.4AccuracyDeepSeek-R1
Evaluation Results
| Method | Links | |
|---|---|---|
| DeepSeek-R1Backbone=DeepSeek-R12026.07 | 93.4 | |
| Claude Sonnet 4.5Backbone=Claude Sonnet 4.52026.07 | 91 | |
| FaithMed (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 85.7 | |
| Agentic Search + GRPO (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 84.4 | |
| MedGemma 27BBackbone=MedGemma 27B2026.07 | 84 | |
| MedRAG (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 83.9 | |
| Qwen3 8BBackbone=Qwen3 8B2026.07 | 83.4 | |
| Agentic Search (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 81.5 | |
| Base (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 81.3 | |
| AttributionIteration=1, Selection Ratio=0.22026.04 | 80.56 | |
| DiversityIteration=2, Selection Ratio=0.52026.04 | 80.45 | |
| DiversityIteration=2, Selection Ratio=0.22026.04 | 80.38 | |
| AllIteration=12026.04 | 80.23 | |
| EVOSELECTIteration=1, Selection Ratio=0.22026.04 | 80.2 | |
| EVOSELECTIteration=2, Selection Ratio=0.52026.04 | 80.16 | |
| EVOSELECTIteration=1, Selection Ratio=0.52026.04 | 80.05 | |
| Attr-DivIteration=2, Selection Ratio=0.22026.04 | 80.05 | |
| DiversityIteration=1, Selection Ratio=0.22026.04 | 80.01 | |
| EVOSELECTIteration=2, Selection Ratio=0.22026.04 | 79.98 | |
| RandomIteration=1, Selection Ratio=0.22026.04 | 79.94 | |
| AttributionIteration=1, Selection Ratio=0.52026.04 | 79.94 | |
| AttributionIteration=2, Selection Ratio=0.52026.04 | 79.94 | |
| TSDSIteration=2, Selection Ratio=0.22026.04 | 79.91 | |
| Attr-DivIteration=1, Selection Ratio=0.52026.04 | 79.87 | |
| Attr-DivIteration=1, Selection Ratio=0.22026.04 | 79.8 | |
| DiversityIteration=1, Selection Ratio=0.52026.04 | 79.76 | |
| Attr-DivIteration=2, Selection Ratio=0.52026.04 | 79.76 | |
| RandomIteration=2, Selection Ratio=0.52026.04 | 79.69 | |
| AttributionIteration=2, Selection Ratio=0.22026.04 | 79.69 | |
| TSDSIteration=1, Selection Ratio=0.52026.04 | 79.65 | |
| RandomIteration=2, Selection Ratio=0.22026.04 | 79.58 | |
| AllIteration=22026.04 | 79.5 | |
| TSDSIteration=2, Selection Ratio=0.52026.04 | 79.5 | |
| TSDSIteration=1, Selection Ratio=0.22026.04 | 79.47 | |
| RandomIteration=1, Selection Ratio=0.52026.04 | 79.43 | |
| Base2026.04 | 78.01 | |
| FaithMed (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 75.6 | |
| Gemma 3 12BBackbone=Gemma 3 12B2026.07 | 73 | |
| Agentic Search + GRPO (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 68.5 | |
| Llama 3.1 8B InstructBackbone=Llama 3.1 8B2026.07 | 67.6 | |
| MedRAG (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 67.5 | |
| Attr-DivIteration=1, Selection Ratio=0.2, Backbone=3b base model2026.04 | 66.59 | |
| TSDSIteration=2, Selection Ratio=0.2, Backbone=3b base model2026.04 | 66.56 | |
| AllIteration=2, Selection Ratio=All, Backbone=3b base model2026.04 | 66.52 | |
| AttributionIteration=2, Selection Ratio=0.2, Backbone=3b base model2026.04 | 66.48 | |
| EVOSELECTIteration=2, Selection Ratio=0.2, Backbone=3b base model2026.04 | 66.45 | |
| EVOSELECTIteration=2, Selection Ratio=0.5, Backbone=3b base model2026.04 | 66.41 | |
| AttributionIteration=2, Selection Ratio=0.5, Backbone=3b base model2026.04 | 66.37 | |
| Base (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 66.2 | |
| EVOSELECTIteration=1, Selection Ratio=0.2, Backbone=3b base model2026.04 | 66.19 | |
| AttributionIteration=1, Selection Ratio=0.2, Backbone=3b base model2026.04 | 66.08 | |
| TSDSIteration=1, Selection Ratio=0.2, Backbone=3b base model2026.04 | 66.01 | |
| RandomIteration=1, Selection Ratio=0.2, Backbone=3b base model2026.04 | 65.97 | |
| EVOSELECTIteration=1, Selection Ratio=0.5, Backbone=3b base model2026.04 | 65.9 | |
| RandomIteration=2, Selection Ratio=0.2, Backbone=3b base model2026.04 | 65.9 | |
| RandomIteration=1, Selection Ratio=0.5, Backbone=3b base model2026.04 | 65.79 | |
| TSDSIteration=2, Selection Ratio=0.5, Backbone=3b base model2026.04 | 65.75 | |
| TSDSIteration=1, Selection Ratio=0.5, Backbone=3b base model2026.04 | 65.68 | |
| Attr-DivIteration=2, Selection Ratio=0.2, Backbone=3b base model2026.04 | 65.68 | |
| Attr-DivIteration=1, Selection Ratio=0.5, Backbone=3b base model2026.04 | 65.57 | |
| Agentic Search (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 65.5 | |
| DiversityIteration=2, Selection Ratio=0.2, Backbone=3b base model2026.04 | 65.43 | |
| AttributionIteration=1, Selection Ratio=0.5, Backbone=3b base model2026.04 | 65.32 | |
| DiversityIteration=1, Selection Ratio=0.2, Backbone=3b base model2026.04 | 65.24 | |
| RandomIteration=2, Selection Ratio=0.5, Backbone=3b base model2026.04 | 65.21 | |
| BaseIteration=Base, Backbone=3b base model2026.04 | 65.1 | |
| DiversityIteration=1, Selection Ratio=0.5, Backbone=3b base model2026.04 | 65.03 | |
| Attr-DivIteration=2, Selection Ratio=0.5, Backbone=3b base model2026.04 | 64.92 | |
| Full FTBase model=gpt-oss 20B, Trainable Parameters=100.0%2026.04 | 64.9 | |
| AllIteration=1, Selection Ratio=All, Backbone=3b base model2026.04 | 64.77 | |
| DiversityIteration=2, Selection Ratio=0.5, Backbone=3b base model2026.04 | 64.77 | |
| MPTBase model=gpt-oss 20B, Trainable Parameters=<0.05%2026.04 | 64.4 | |
| LoRABase model=gpt-oss 20B, Trainable Parameters=~2.50%2026.04 | 62.9 | |
| Full FTBase model=Meditron3 8B, Trainable Parameters=100.0%2026.04 | 60.3 | |
| MPTBase model=Meditron3 8B, Trainable Parameters=<0.05%2026.04 | 59.5 | |
| PTBase model=gpt-oss 20B, Trainable Parameters=<0.05%2026.04 | 58.5 | |
| Full FTBase model=LLaMA 3.1 8B, Trainable Parameters=100.0%2026.04 | 58.4 | |
| LoRABase model=Meditron3 8B, Trainable Parameters=~2.50%2026.04 | 58.1 | |
| MPTBase model=LLaMA 3.1 8B, Trainable Parameters=<0.05%2026.04 | 57.3 | |
| LoRABase model=LLaMA 3.1 8B, Trainable Parameters=~2.50%2026.04 | 56 | |
| PTBase model=Meditron3 8B, Trainable Parameters=<0.05%2026.04 | 53.5 | |
| PTBase model=LLaMA 3.1 8B, Trainable Parameters=<0.05%2026.04 | 51.2 | |
| OLTQAFull Config=True2023.05 | 34.48 | |
| UnifiedQA2023.05 | 29.14 |