Question Answering on MedMCQA (val)
90AccuracyHuman (expert score)
Evaluation results
| Method | Links | |
|---|---|---|
| Human (expert score)2022.07 | 90 | |
| GPT-4Date=20232022.07 | 73.7 | |
| MedPalm v2Date=20232022.07 | 72.3 | |
| Codex 5-shot CoTDate=2022, Few-shot shots=5, Chain-of-Thought (CoT)=true, Ensemble samples (k)=1002022.07 | 59.7 | |
| Llama-2 5-shot CoTDate=2023, Model Parameters=70B, Few-shot shots=5, Chain-of-Thought (CoT)=true, Ensemble samples (k)=502022.07 | 53.6 | |
| GalacticaDate=2022, Evaluation Protocol=Finetuned SOTA2022.07 | 52.9 | |
| Human (passing score)2022.07 | 50 |