Question Answering on BioASQ
98.32AccuracyFine-Tuned GPT-4o + MedBioRAG
Evaluation results
| Method | Links | |
|---|---|---|
| Fine-Tuned GPT-4o + MedBioRAGBackbone=GPT-4o, Fine-tuned=true, RAG=MedBioRAG2025.12 | 98.32 | |
| GPT-4 + MedBioRAGBackbone=GPT-4, Fine-tuned=false, RAG=MedBioRAG2025.12 | 97.79 | |
| Fine-Tuned GPT-4oBackbone=GPT-4o, Fine-tuned=true2025.12 | 97.06 | |
| GPT-4o + MedBioRAGBackbone=GPT-4o, Fine-tuned=false, RAG=MedBioRAG2025.12 | 97.06 | |
| GPT-4o-mini + MedBioRAGBackbone=GPT-4o-mini, Fine-tuned=false, RAG=MedBioRAG2025.12 | 97.06 | |
| GPT-4o-miniBackbone=GPT-4o-mini, Fine-tuned=false2025.12 | 96.32 | |
| GPT-4Backbone=GPT-4, Fine-tuned=false2025.12 | 96.32 | |
| GPT-4oBackbone=GPT-4o, Fine-tuned=false2025.12 | 96.12 | |
| GalacticaModel Size=120B, Shots=0, Domain=in-domain2022.11 | 94.3 | |
| BLOOMShots=0, Domain=in-domain2022.11 | 91.4 | |
| GPT-3.5Backbone=GPT-3.5, Fine-tuned=false2025.12 | 88.24 | |
| OPTShots=0, Domain=in-domain2022.11 | 81.4 | |
| TTARAGBackbone=Llama-3.1-8b-it2026.01 | 75 | |
| TTARAGBackbone=Llama-2-7b-chat2026.01 | 71.8 | |
| GPT-3.5 + MedBioRAGBackbone=GPT-3.5, Fine-tuned=false, RAG=MedBioRAG2025.12 | 66.91 | |
| USCLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 61.5 | |
| USCLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 61.5 | |
| ASTUTE RAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 60.1 | |
| Self-RouteLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 59.1 | |
| USCLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 58.4 | |
| ASTUTE RAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 58.4 | |
| TTARAGBackbone=ChatGLM-3-6b2026.01 | 58.4 | |
| RAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 58 | |
| InstructRAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 58 | |
| Self-RouteLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 58 | |
| ASTUTE RAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 58 | |
| InstructRAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 57.3 | |
| Self-RouteLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 57.3 | |
| GenReadLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 57 | |
| RobustRAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 56.3 | |
| RAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 55.9 | |
| GenReadLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 55.6 | |
| naive-ragBackbone=Llama-3.1-8b-it2026.01 | 55.6 | |
| In-Context LearningBackbone=Llama-2-7b-chat2026.01 | 55.6 | |
| RAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 55.2 | |
| Chain-of-ThoughtBackbone=Llama-2-7b-chat2026.01 | 55.1 | |
| GenReadLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 54.9 | |
| InstructRAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 54.9 | |
| Chain-of-ThoughtBackbone=Llama-3.1-8b-it2026.01 | 54.6 | |
| naive-ragBackbone=Llama-2-7b-chat2026.01 | 54.1 | |
| ProvenceBackbone=Mistral-7B2025.03 | 54 | |
| RECOMPBackbone=Mistral-7B2025.03 | 53 | |
| No compressionBackbone=Qwen-7B2025.03 | 53 | |
| No compressionBackbone=Mistral-24B2025.03 | 53 | |
| OSCAR-llamaBackbone=Mistral-24B2025.03 | 53 | |
| OSCAR-llamaBackbone=Mistral-7B2025.03 | 52 | |
| OSCAR-8-LayersBackbone=Mistral-7B2025.03 | 52 | |
| OSCAR-llamaBackbone=Qwen-7B2025.03 | 52 | |
| Self-RouteLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 51.8 | |
| naive-ragBackbone=ChatGLM-3-6b2026.01 | 51.4 | |
| No compressionBackbone=Mistral-7B2025.03 | 51 | |
| OSCAR-8-LayersBackbone=Qwen-7B2025.03 | 51 | |
| In-Context LearningBackbone=ChatGLM-3-6b2026.01 | 50.8 | |
| No RAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 50.4 | |
| RobustRAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 50.4 | |
| InstructRAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 50.4 | |
| OSCAR-5-LayersBackbone=Mistral-7B2025.03 | 50 | |
| In-Context LearningBackbone=Llama-3.1-8b-it2026.01 | 49.8 | |
| ASTUTE RAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 49.3 | |
| RAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 49 | |
| PISCOBackbone=Mistral-7B2025.03 | 49 | |
| GenReadLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 48.3 | |
| OSCAR-5-LayersBackbone=Llama-1B2025.03 | 46 | |
| No RAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 45.8 | |
| Chain-of-ThoughtBackbone=ChatGLM-3-6b2026.01 | 44.3 | |
| RobustRAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 44.1 | |
| RobustRAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 44.1 | |
| No RAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 43.7 | |
| No RAGBackbone=Mistral-7B2025.03 | 40 | |
| No compressionBackbone=Llama-1B2025.03 | 40 | |
| USCLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 36 | |
| No RAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 34.3 |