Critical View of Safety assessment on Endoscapes (frame-level)
77.1Component 1 Score (C1)SurgAtlas-VLM
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| SurgAtlas-VLMProtocol=Open-vocabulary generation2026.06 | 77.1 | 82.9 | 73 | 77.7 | |
| SurgVLM-72BProtocol=Open-vocabulary generation2026.06 | 76.1 | 83.1 | 70.6 | 76.6 | |
| SurgVLM-7BProtocol=Open-vocabulary generation2026.06 | 75.3 | 82.4 | 72.9 | 76.9 | |
| Qwen2.5-VL-7BProtocol=MCQ2026.06 | 56.1 | 82.4 | 59.2 | 65.9 | |
| Gemini 2.0 FlashProtocol=MCQ2026.06 | 47.8 | 63.9 | 67.1 | 59.6 | |
| InternVL3-8BProtocol=MCQ2026.06 | 40 | 53.3 | 51.4 | 48.2 | |
| Sum-of-ChecksModel=GPT-4o mini2026.04 | 33.6 | 29.1 | 47.9 | 36.9 | |
| Sum-of-ChecksModel=Claude Opus 3.52026.04 | 31.1 | 32.9 | 46.9 | 37 | |
| Sum-of-ChecksModel=Claude Haiku 3.52026.04 | 29.2 | 24.2 | 48.5 | 34 | |
| Direct+FSModel=Claude Opus 3.52026.04 | 28 | 26.7 | 43.9 | 32.9 | |
| CoT+FSModel=Claude Opus 3.52026.04 | 27.6 | 26 | 43.4 | 32.3 | |
| Direct+FSModel=GPT-4o mini2026.04 | 26.9 | 26.8 | 42.9 | 32.2 | |
| CoT+FSModel=GPT-4o mini2026.04 | 26.9 | 28.2 | 44.3 | 33.1 | |
| CoTModel=Claude Opus 3.52026.04 | 26.4 | 26.6 | 42.5 | 31.9 | |
| CoTModel=GPT-4o mini2026.04 | 26.2 | 27.7 | 43.9 | 32.6 | |
| DirectModel=Claude Opus 3.52026.04 | 24.3 | 25.4 | 42.9 | 30.9 | |
| SubQModel=Claude Opus 3.52026.04 | 24.1 | 24.5 | 43 | 30.5 | |
| SubQModel=Claude Haiku 3.52026.04 | 23.4 | 22.5 | 43.3 | 29.8 | |
| DirectModel=GPT-4o mini2026.04 | 22.9 | 23.9 | 42.5 | 29.7 | |
| SubQModel=GPT-4o mini2026.04 | 22.5 | 25.9 | 43.4 | 30.6 | |
| CoT+FSModel=Claude Haiku 3.52026.04 | 22.5 | 23.7 | 43.1 | 29.7 | |
| SubQ+FSModel=Claude Haiku 3.52026.04 | 22.4 | 23.8 | 42.5 | 29.6 | |
| SubQ+FSModel=GPT-4o mini2026.04 | 22.1 | 21.5 | 43.8 | 29.1 | |
| Direct+FSModel=Claude Haiku 3.52026.04 | 21.1 | 21.9 | 46.5 | 29.8 | |
| DirectModel=Claude Haiku 3.52026.04 | 20.7 | 20.8 | 42.2 | 27.9 | |
| CoTModel=Claude Haiku 3.52026.04 | 19.7 | 21.7 | 43.8 | 28.4 | |
| SubQ+FSModel=Claude Opus 3.52026.04 | 19.2 | 19.7 | 43.9 | 27.6 | |
| GPT-4oProtocol=MCQ2026.06 | 6.7 | 5.9 | 7.5 | 6.7 |