Multimodal Medical Reasoning on ArogyaBodha 1.0 (test)
47.69Assamese ScoreArogyaSutra(Qwen 2.5 VL 7B)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| ArogyaSutra(Qwen 2.5 VL 7B)Base Model=Qwen 2.5 VL, Model Scale=7B2026.06 | 47.69 | 45.38 | 42.3 | 42.3 | 43.07 | 41.53 | 41.53 | 43.4 | |
| Qwen3-VL-8B-InstructBase Model=Qwen3-VL, Model Scale=8B, Instruction Tuned=true2026.06 | 41.3 | 40.5 | 41.5 | 41.5 | 41.6 | 38.7 | 39.9 | 40.71 | |
| Mistral-Small-3.2-24B-InstructBase Model=Mistral-Small-3.2, Model Scale=24B, Instruction Tuned=true2026.06 | 40.9 | 42.9 | 44.1 | 43.2 | 40.9 | 41.4 | 42.5 | 42.27 | |
| MedGemma-4B-itBase Model=MedGemma, Model Scale=4B, Instruction Tuned=true2026.06 | 38.5 | 35.1 | 37.4 | 34.5 | 35.7 | 36.7 | 34.9 | 36.11 | |
| GPT 4.0Model Scale=4.02026.06 | 38.4 | 44.6 | 40.1 | 37.8 | 39.2 | 36.9 | 39.1 | 39.3 | |
| GPT 4.0 miniModel Scale=mini2026.06 | 35.38 | 45.38 | 37.69 | 33.07 | 36.15 | 31.53 | 33.2 | 36.05 | |
| ArogyaSutra(Qwen 2.5 VL 3B)Base Model=Qwen 2.5 VL, Model Scale=3B2026.06 | 34.69 | 37.08 | 38.36 | 37.03 | 33.07 | 35.73 | 33.6 | 35.65 | |
| Qwen2.5-VL-7B-InstructBase Model=Qwen2.5-VL, Model Scale=7B, Instruction Tuned=true2026.06 | 32.3 | 36.5 | 33.84 | 36.12 | 32.3 | 35.38 | 33.07 | 34.21 | |
| LLaVA-v1.6-34BBase Model=LLaVA-v1.6, Model Scale=34B2026.06 | 32.09 | 31.45 | 34.44 | 33.66 | 31.7 | 31.31 | 35.42 | 32.87 | |
| Qwen2.5-VL-3B-InstructBase Model=Qwen2.5-VL, Model Scale=3B, Instruction Tuned=true2026.06 | 28.7 | 30.7 | 31.5 | 31.3 | 28.4 | 28.2 | 28.1 | 29.56 | |
| BioMistral-7BBase Model=BioMistral, Model Scale=7B2026.06 | 25.64 | 25.78 | 28.38 | 25.83 | 24.27 | 27.59 | 26.95 | 26.35 | |
| MedVLM-R1Base Model=MedVLM, Model Scale=R12026.06 | 23.5 | 22.8 | 25.6 | 22.8 | 24.2 | 25.4 | 22.2 | 23.79 | |
| Llama3-OpenBioLLM-8BBase Model=Llama3-OpenBioLLM, Model Scale=8B2026.06 | 16.1 | 15.6 | 22.2 | 20.5 | 11.7 | 12.1 | 7.7 | 15.13 |