Medical Question Answering on MedQA, MedMCQA, and PubMedQA Aggregate
60.3MedQA AccuracyMed42-v2-Evontree (implicit)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Med42-v2-Evontree (implicit)Setting=SFT (on evaluation)2025.10 | 60.3 | 62.4 | 78.9 | 67.2 | |
| OntoTune_sftSetting=SFT (on evaluation)2025.10 | 58.4 | 60.4 | 78.6 | 65.8 | |
| OntoTune_dpoSetting=SFT (on evaluation)2025.10 | 58.3 | 60.7 | 79.4 | 66.1 | |
| OntoTune_sft+dpoSetting=SFT (on evaluation)2025.10 | 58.2 | 60.5 | 78.9 | 65.9 | |
| Med42-v2Setting=SFT (on evaluation)2025.10 | 57.8 | 58.1 | 74.6 | 63.5 | |
| Med42-v2-Evontree (mix)Setting=SFT (on evaluation)2025.10 | 57.2 | 57.7 | 74.9 | 63.3 | |
| Med42-v2-Evontree (explicit)Setting=SFT (on evaluation)2025.10 | 57.1 | 58.6 | 71.1 | 62.3 | |
| LLaMA3 8BSetting=SFT (on evaluation)2025.10 | 56.4 | 53.9 | 77.2 | 62.5 | |
| TaxoLLaMASetting=SFT (on evaluation)2025.10 | 55.9 | 57.5 | 77.6 | 63.7 | |
| OntoTune_dpoSetting=zero-shot2025.10 | 53.3 | 57.2 | 65.5 | 58.7 | |
| LLaMA3 8B-Evontree (implicit)Setting=zero-shot2025.10 | 52.7 | 54.4 | 72.9 | 60 | |
| OntoTune_sft+dpoSetting=zero-shot2025.10 | 51.9 | 56.7 | 66.3 | 58.3 | |
| LLaMA3 8BSetting=zero-shot2025.10 | 51.7 | 51.7 | 70.3 | 57.9 | |
| OntoTune_sftSetting=zero-shot2025.10 | 51.5 | 56.7 | 72 | 60.1 | |
| AloeSetting=SFT (on evaluation)2025.10 | 51.1 | 56.8 | 75.4 | 61.1 | |
| LLaMA3 8B-Evontree (mix)Setting=zero-shot2025.10 | 51 | 57.4 | 74.7 | 61 | |
| TaxoLLaMASetting=zero-shot2025.10 | 50.5 | 46.1 | 73.4 | 56.7 | |
| LLaMA3 8B-Evontree (explicit)Setting=zero-shot2025.10 | 43.4 | 48.6 | 76.4 | 56.1 |