Multiple Choice Question Answering on MedQA 5 opts
87AccuracyGPT-4o
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-4oPrompting (shots)=5-shot2024.10 | 87 | |
| GPT-4oPrompting (shots)=0-shot2024.10 | 86 | |
| GPT-4T (May 2024)Prompting (shots)=5-shot2024.10 | 78 | |
| GPT-4T (May 2024)Prompting (shots)=0-shot2024.10 | 75 | |
| GPT-3.5 Turbo 1106Evaluation Protocol=Few-shot, Few-shot settings=3-shot2024.02 | 50.82 | |
| GPT-3.5 Turbo 1106shot=3-shot, options=52024.02 | 50.82 | |
| BioMistral 7B DAREEvaluation Protocol=SFT, Few-shot settings=3-shot, Merging Strategy=DARE2024.02 | 45.2 | |
| BioMistral 7B SLERPEvaluation Protocol=SFT, Few-shot settings=3-shot, Merging Strategy=SLERP2024.02 | 44.3 | |
| BioMistral 7B EnsembleEvaluation Protocol=SFT, Few-shot settings=3-shot, Merging Strategy=Ensemble2024.02 | 43.6 | |
| BioMistral 7B TIESEvaluation Protocol=SFT, Few-shot settings=3-shot, Merging Strategy=TIES2024.02 | 43.2 | |
| BioMistral 7BEvaluation Protocol=SFT, Few-shot settings=3-shot2024.02 | 42.8 | |
| Mistral 7B InstructEvaluation Protocol=SFT, Few-shot settings=3-shot2024.02 | 40.9 | |
| BioMistral 7B SLERPshot=3-shot, options=5, merging_strategy=SLERP2024.02 | 38.9 | |
| BioMistral 7B DAREshot=3-shot, options=5, merging_strategy=DARE2024.02 | 38.8 | |
| BioMistral 7B TIESshot=3-shot, options=5, merging_strategy=TIES2024.02 | 37.7 | |
| BioMistral 7Bshot=3-shot, options=52024.02 | 37.4 | |
| BioMistral 7B Ensembleshot=3-shot, options=5, merging_strategy=Ensemble2024.02 | 37.1 | |
| BioMedGPT-LM-7Bshot=3-shot, options=52024.02 | 34.9 | |
| Mistral 7B Instructshot=3-shot, options=52024.02 | 34.5 | |
| BioMedGPT-LM-7BEvaluation Protocol=SFT, Few-shot settings=3-shot2024.02 | 33.9 | |
| MedAlpaca 7BEvaluation Protocol=SFT, Few-shot settings=3-shot2024.02 | 33.7 | |
| MedAlpaca 7Bshot=3-shot, options=52024.02 | 30.4 | |
| MediTron-7BEvaluation Protocol=SFT, Few-shot settings=3-shot2024.02 | 28.1 | |
| MediTron-7Bshot=3-shot, options=52024.02 | 26.3 | |
| PMC-LLAMA 7Bshot=3-shot, options=52024.02 | 21.1 | |
| PMC-LLaMA 7BEvaluation Protocol=SFT, Few-shot settings=3-shot2024.02 | 20.2 |