Visual Question Answering on SurgAtlas Aexp Open partition
41Perception & ID AccuracySurgAtlas-VLM (SurgAtlas)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SurgAtlas-VLM (SurgAtlas)Backbone=Qwen3-VL-8B, Fine-tuning regime=SurgAtlas QA pairs (Aqa) alone2026.06 | 41 | 34.5 | 37.9 | 40 | 67.2 | 39 | |
| SurgAtlas-VLM (+ public)Backbone=Qwen3-VL-8B, Fine-tuning regime=SurgAtlas QA pairs (Aqa) + public dataset supervision (Dit)2026.06 | 39.4 | 36 | 38.5 | 41.3 | 68.9 | 39.3 | |
| GPT-5Frame sampling=4 frames per QA clip2026.06 | 36.5 | 32.5 | 51.2 | 34.8 | 67.2 | 37.6 | |
| GPT-5.1Frame sampling=4 frames per QA clip2026.06 | 35.2 | 34.3 | 47.6 | 32.9 | 59 | 37 | |
| Gemini 2.5 ProFrame sampling=4 frames per QA clip2026.06 | 27.2 | 29.2 | 32.9 | 31.2 | 56.5 | 30.3 | |
| Qwen3-VL-32BBackbone=Qwen3-VL-32B2026.06 | 21.9 | 24.6 | 31.3 | 27.1 | 62.3 | 26.3 | |
| Qwen3-VL-8BBackbone=Qwen3-VL-8B2026.06 | 16.3 | 19.4 | 18.1 | 22.6 | 50.8 | 19.9 |