List your company
ResearchFormerly WizWand

Multiple Choice Question Answering on MedQA 5 opts

87AccuracyGPT-4o
17.52835.56453.671.636Feb 15, 2024Mar 28, 2024May 9, 2024Jun 20, 2024Aug 1, 2024Sep 12, 2024Oct 25, 2024
Updated 8mo ago

Evaluation results

MethodLinks
GPT-4oPrompting (shots)=5-shot2024.1087
GPT-4oPrompting (shots)=0-shot2024.1086
GPT-4T (May 2024)Prompting (shots)=5-shot2024.1078
GPT-4T (May 2024)Prompting (shots)=0-shot2024.1075
GPT-3.5 Turbo 1106Evaluation Protocol=Few-shot, Few-shot settings=3-shot2024.0250.82
GPT-3.5 Turbo 1106shot=3-shot, options=52024.0250.82
BioMistral 7B DAREEvaluation Protocol=SFT, Few-shot settings=3-shot, Merging Strategy=DARE2024.0245.2
BioMistral 7B SLERPEvaluation Protocol=SFT, Few-shot settings=3-shot, Merging Strategy=SLERP2024.0244.3
BioMistral 7B EnsembleEvaluation Protocol=SFT, Few-shot settings=3-shot, Merging Strategy=Ensemble2024.0243.6
BioMistral 7B TIESEvaluation Protocol=SFT, Few-shot settings=3-shot, Merging Strategy=TIES2024.0243.2
BioMistral 7BEvaluation Protocol=SFT, Few-shot settings=3-shot2024.0242.8
Mistral 7B InstructEvaluation Protocol=SFT, Few-shot settings=3-shot2024.0240.9
BioMistral 7B SLERPshot=3-shot, options=5, merging_strategy=SLERP2024.0238.9
BioMistral 7B DAREshot=3-shot, options=5, merging_strategy=DARE2024.0238.8
BioMistral 7B TIESshot=3-shot, options=5, merging_strategy=TIES2024.0237.7
BioMistral 7Bshot=3-shot, options=52024.0237.4
BioMistral 7B Ensembleshot=3-shot, options=5, merging_strategy=Ensemble2024.0237.1
BioMedGPT-LM-7Bshot=3-shot, options=52024.0234.9
Mistral 7B Instructshot=3-shot, options=52024.0234.5
BioMedGPT-LM-7BEvaluation Protocol=SFT, Few-shot settings=3-shot2024.0233.9
MedAlpaca 7BEvaluation Protocol=SFT, Few-shot settings=3-shot2024.0233.7
MedAlpaca 7Bshot=3-shot, options=52024.0230.4
MediTron-7BEvaluation Protocol=SFT, Few-shot settings=3-shot2024.0228.1
MediTron-7Bshot=3-shot, options=52024.0226.3
PMC-LLAMA 7Bshot=3-shot, options=52024.0221.1
PMC-LLaMA 7BEvaluation Protocol=SFT, Few-shot settings=3-shot2024.0220.2