Biomedical Retrieval-Augmented Generation on Mirage
87.24MMLU-med AccuracyGPT-4-0613
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GPT-4-0613organization=OpenAI, zero-shot=true2024.07 | 87.24 | 70.6 | 92.56 | 82.8 | 66.65 | 79.97 | |
| Llama3-RankRAG 70Bbackbone=Llama3, parameters=70B, zero-shot=true2024.07 | 81.44 | 79.8 | 90.76 | 69.21 | 69.11 | 78.06 | |
| Llama3-ChatQA-1.5 70Bbackbone=Llama3, version=1.5, parameters=70B, zero-shot=true2024.07 | 80.51 | 74.8 | 83.17 | 68.89 | 62.54 | 73.98 | |
| Mixtral 8*7Bbackbone=Mixtral 8x7B, zero-shot=true2024.07 | 75.85 | 67.6 | 87.54 | 60.02 | 56.42 | 69.49 | |
| GPT-3.5organization=OpenAI, zero-shot=true2024.07 | 75.48 | 67.4 | 90.29 | 66.61 | 58.04 | 71.56 | |
| Meditron 70Bbackbone=Meditron, parameters=70B, zero-shot=true2024.07 | 65.38 | 56.4 | 76.86 | 49.57 | 52.67 | 60.18 | |
| Llama3-RankRAG 8Bbackbone=Llama3, parameters=8B, zero-shot=true2024.07 | 64.55 | 65 | 84.44 | 48.86 | 56.9 | 63.95 | |
| Llama3-ChatQA-1.5 8Bbackbone=Llama3, version=1.5, parameters=8B, zero-shot=true2024.07 | 61.4 | 66.4 | 82.69 | 42.36 | 46.97 | 59.96 | |
| Llama2 70Bbackbone=Llama2, parameters=70B, zero-shot=true2024.07 | 54.55 | 50.4 | 73.95 | 44.93 | 43.08 | 53.38 | |
| PMC-llama 13Bbackbone=PMC-llama, parameters=13B, zero-shot=true2024.07 | 52.53 | 42.58 | 48.29 | 56 | 65.21 | 52.92 |