Question Answering on UMLS Biomedical QA (PubMedQA, BioASQ, ProcBank) (20% test)
57PubMedQA AccuracySAKE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SAKEModel=Qwen2.5 7B-Instruct2025.05 | 57 | 95.2 | 80 | 75.4 | |
| GIVEModel=GPT-3.5 Turbo2025.05 | 53.6 | 88.2 | 73.4 | 70.1 | |
| SAKEModel=Qwen2.5 3B-Instruct2025.05 | 51 | 82.1 | 70 | 66.1 | |
| SFTModel=Qwen2.5 7B-Instruct2025.05 | 42 | 82.1 | 75 | 62.9 | |
| GRPOModel=Qwen2.5 7B-Instruct2025.05 | 42 | 66.7 | 72.5 | 56.7 | |
| SFTModel=Qwen2.5 3B-Instruct2025.05 | 33 | 76.2 | 55 | 53.1 | |
| CoTModel=Qwen2.5 7B-Instruct2025.05 | 29 | 77.4 | 75 | 55.4 | |
| Search-R1Model=Qwen2.5 7B-Instruct2025.05 | 24 | 78.6 | 75 | 53.6 | |
| GraphRAGModel=GPT-3.5 Turbo2025.05 | 23.4 | 10.3 | 71.3 | 27 | |
| I/O PromptModel=Qwen2.5 7B-Instruct2025.05 | 23 | 78.5 | 70 | 52.2 | |
| RAGModel=Qwen2.5 3B-Instruct2025.05 | 22 | 20.2 | 52.5 | 26.8 | |
| GRPOModel=Qwen2.5 3B-Instruct2025.05 | 19 | 69.1 | 67.5 | 46.5 | |
| ToGModel=GPT-3.5 Turbo2025.05 | 17.6 | 18 | 66.8 | 26.6 | |
| Search-R1Model=Qwen2.5 3B-Instruct2025.05 | 16 | 39.3 | 42.5 | 29.5 | |
| RAGModel=Qwen2.5 7B-Instruct2025.05 | 15 | 46.4 | 70 | 36.6 | |
| CoTModel=Qwen2.5 3B-Instruct2025.05 | 12 | 25 | 65 | 26.3 | |
| I/O PromptModel=Qwen2.5 3B-Instruct2025.05 | 11 | 28.6 | 67.5 | 27.7 |