Medical Question Answering on MIRAGE (test)
89.44MMLU-MedCoT
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| CoTBackbone=GPT-4 (-32k-0613)2024.02 | 89.44 | 83.97 | 69.88 | 39.6 | 84.3 | 73.44 | |
| MEDRAGBackbone=GPT-4 (-32k-0613)2024.02 | 87.24 | 82.8 | 66.65 | 70.6 | 92.56 | 79.97 | |
| MEDRAGBackbone=Mixtral (8×7B)2024.02 | 75.85 | 60.02 | 56.42 | 67.6 | 87.54 | 69.48 | |
| MEDRAGBackbone=GPT-3.5 (-16k-0613)2024.02 | 75.48 | 66.61 | 58.04 | 67.4 | 90.29 | 71.57 | |
| CoTBackbone=Mixtral (8×7B)2024.02 | 74.01 | 64.1 | 56.28 | 35.2 | 77.51 | 61.42 | |
| CoTBackbone=GPT-3.5 (-16k-0613)2024.02 | 72.91 | 65.04 | 55.25 | 36 | 74.27 | 60.69 | |
| MEDRAGBackbone=MEDITRON (70B)2024.02 | 65.38 | 49.57 | 52.67 | 56.4 | 76.86 | 60.18 | |
| CoTBackbone=MEDITRON (70B)2024.02 | 64.92 | 51.69 | 46.74 | 53.4 | 68.45 | 57.04 | |
| CoTBackbone=Llama2 (70B)2024.02 | 57.39 | 47.84 | 42.6 | 42.2 | 61.17 | 50.24 | |
| MEDRAGBackbone=Llama2 (70B)2024.02 | 54.55 | 44.93 | 43.08 | 50.4 | 73.95 | 53.38 | |
| MEDRAGBackbone=PMC-LLaMA (13B)2024.02 | 52.53 | 42.58 | 48.29 | 56 | 65.21 | 52.92 | |
| CoTBackbone=PMC-LLaMA (13B)2024.02 | 52.16 | 44.38 | 46.55 | 55.8 | 63.11 | 52.4 |