Biomedical Question Answering on PubMedQA PQA-L In-Domain (test)
78AccuracyHuman (expert)
Evaluation Results
| Method | Links | |
|---|---|---|
| Human (expert)Setting=Manual2023.08 | 78 | |
| BioMedGPT-10BSetting=Fine-tuning2023.08 | 76.1 | |
| Llama2-ChatSetting=Fine-tuning2023.08 | 75.5 | |
| LlamaSetting=Fine-tuning2023.08 | 73.4 | |
| InstructGPTSetting=Zero-shot2023.08 | 73.2 | |
| PMC-LlamaSetting=Fine-tuning2023.08 | 69.5 | |
| ChatGPTSetting=Zero-shot2023.08 | 63.9 | |
| Human (pass)Setting=Manual2023.08 | 60 | |
| Llama2-ChatSetting=Zero-shot2023.08 | 21.9 | |
| LlamaSetting=Zero-shot2023.08 | 5.2 | |
| Llama2Setting=Zero-shot2023.08 | 3.7 |