Medical Knowledge Question Answering on MMLU Clinical Knowledge
92AccuracyGPT-4o
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-4oPrompting (shots)=0-shot2024.10 | 92 | |
| GPT-4oPrompting (shots)=5-shot2024.10 | 92 | |
| GPT-4T (May 2024)Prompting (shots)=5-shot2024.10 | 87 | |
| GPT-4T (May 2024)Prompting (shots)=0-shot2024.10 | 85 | |
| Dense ModelBase Model=Llama 3.1-8B, Fine-tuning=Full Fine-tuning, Training Data=MedMCQA2026.01 | 73.5 | |
| GradPrunerBase Model=Llama 3.1-8B, Fine-tuning=Full Fine-tuning, Training Data=MedMCQA2026.01 | 71.7 | |
| LacoBase Model=Llama 3.1-8B, Fine-tuning=Full Fine-tuning, Training Data=MedMCQA2026.01 | 71.1 | |
| SATBase Model=Llama 3.1-8B, Fine-tuning=Full Fine-tuning, Training Data=MedMCQA2026.01 | 70.9 | |
| MINITRONBase Model=Llama 3.1-8B, Fine-tuning=Full Fine-tuning, Training Data=MedMCQA2026.01 | 70.3 | |
| LLMPrunerBase Model=Llama 3.1-8B, Fine-tuning=Full Fine-tuning, Training Data=MedMCQA2026.01 | 68.8 |