List your company
ResearchFormerly WizWand

Question Answering on MedMCQA (dev)

0.791AccuracyGPT-4 (Medprompt)
0.27620.409850.54350.67715Nov 16, 2022Mar 14, 2023Jul 10, 2023Nov 5, 2023Mar 2, 2024Jun 28, 2024Oct 25, 2024
Updated 8mo ago

Evaluation results

MethodLinks
GPT-4 (Medprompt)Prompting Strategy=Medprompt, Few-shot Count (k)=5, Ensemble steps=5x, Fine-tuning=false2023.110.791
GPT-4oPrompting (shots)=5-shot2024.100.79
GPT-4oPrompting (shots)=0-shot2024.100.77
GPT-4Prompting Strategy=5-shot, Fine-tuning=false2023.110.724
Med-PaLM 2Strategy Selection=choose best, Fine-tuning=true2023.110.723
GPT-4T (May 2024)Prompting (shots)=5-shot2024.100.72
GPT-4T (May 2024)Prompting (shots)=0-shot2024.100.7
Flan-PaLM 540BStrategy Selection=choose best, Fine-tuning=true2023.110.576
GalacticaModel Size=120B, Shots=0, Domain=in-domain2022.110.529
BLOOMShots=5, Domain=in-domain2022.110.325
OPTShots=5, Domain=in-domain2022.110.296