Question Answering on PubMedQA PQA-L (test)
87.08AccuracyGyan-4.4
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gyan-4.4Category=Correct2026.05 | 87.08 | 418 | |
| BioGPT2022.10 | 78.2 | — | |
| MediSwift-XLSize=1.21B, Sparsity=dense2024.03 | 76.8 | — | |
| ClinicalAgentsCategory=Multi-Agents2026.03 | 76.6 | — | |
| ReConcileCategory=Multi-Agents2026.03 | 76.4 | — | |
| MediSwift-XLSize=605M, Sparsity=50%2024.03 | 76.3 | — | |
| Intern-S1Category=Single LLM2026.03 | 75.8 | — | |
| BioGPT-largeSize=1.54B2024.03 | 75.5 | — | |
| MediSwift-XLSize=302M, Sparsity=75%2024.03 | 75.4 | — | |
| GPT-4Shot-count=0-shot2024.03 | 75.2 | — | |
| MediSwift-LargeSize=510M, Sparsity=dense2024.03 | 75.1 | — | |
| Few-shot + CoTCategory=Single Agent2026.03 | 74.6 | — | |
| BioMedLMSize=2.70B2024.03 | 74.4 | — | |
| MediSwift-MedSize=302M, Sparsity=dense2024.03 | 74.2 | — | |
| MediSwift-LargeSize=255M, Sparsity=50%2024.03 | 74.2 | — | |
| MDAgentsCategory=Multi-Agents2026.03 | 74.2 | — | |
| MedChain-AgentsCategory=Multi-Agents2026.03 | 74.2 | — | |
| MediSwift-MedSize=151M, Sparsity=50%2024.03 | 73.8 | — | |
| BioGPT-medSize=345M2024.03 | 73.6 | — | |
| DRAGONSize=360M2024.03 | 73.4 | — | |
| PMC-LlamaSize=7.00B2024.03 | 73.4 | — | |
| MediSwift-LargeSize=128M, Sparsity=75%2024.03 | 73.4 | — | |
| ReActCategory=Single Agent2026.03 | 73.2 | — | |
| MediSwift-MedSize=76M, Sparsity=75%2024.03 | 72.4 | — | |
| MedAgentsCategory=Multi-Agents2026.03 | 72.4 | — | |
| ColaCareCategory=Multi-Agents2026.03 | 72.4 | — | |
| BioLinkBERTModel Variant=large2022.10 | 72.2 | — | |
| BioLinkBERT-largeSize=340M2024.03 | 72.2 | — | |
| RAGCategory=Single Agent2026.03 | 71.8 | — | |
| GPT-3.5Shot-count=0-shot2024.03 | 71.6 | — | |
| AutoGenCategory=Multi-Agents2026.03 | 71.4 | — | |
| BioLinkBERTModel Variant=base2022.10 | 70.2 | — | |
| BioLinkBERT-baseSize=110M2024.03 | 70.2 | — | |
| Llama-4-Maverick-17BCategory=Single LLM2026.03 | 68.8 | — | |
| GPT-5.2Category=Single LLM2026.03 | 68.2 | — | |
| MedGemma-4BCategory=Single LLM2026.03 | 66.8 | — | |
| FineMedLM-o1-8BCategory=Single LLM2026.03 | 64.4 | — | |
| BioELECTRA2022.10 | 64.2 | — | |
| BioELECTRASize=110M2024.03 | 64.2 | — | |
| DeepSeek-R1Category=Single LLM2026.03 | 62 | — | |
| Qwen3-VL-235BCategory=Single LLM2026.03 | 56.2 | — | |
| PubMedBERT2022.10 | 55.8 | — | |
| PubMedBERTSize=110M2024.03 | 55.8 | — | |
| HuaTuoGPT-o1-7BCategory=Single LLM2026.03 | 55.2 | — | |
| Gyan-4.4Category=Incorrect2026.05 | 12.92 | 62 |