Triplet Recognition on CholecT50
51.69Accuracy (Triplet)Surg-R1
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Surg-R1Thinking=true2026.03 | 51.69 | 86.5 | 72.15 | 57.49 | 87.77 | 57.16 | 29.4 | 16.17 | — | |
| Surg-R12026.03 | 51.69 | — | — | — | — | — | — | — | — | |
| Gemini-3.0-Pro-ThinkingThinking=true2026.03 | 8.2 | 24.32 | 52.86 | 12.76 | 39.29 | 28.99 | 16.29 | 6.86 | — | |
| Gemini-3.0-Pro2026.03 | 8.2 | — | — | — | — | — | — | — | — | |
| Qwen2.5-VL-7B-Surg-CholecT50Thinking=false2026.03 | 8.01 | 54.75 | 43.67 | 11.08 | 42.39 | 28.96 | 15.15 | 5.69 | — | |
| GPT-5.1-ThinkingThinking=true2026.03 | 6.77 | 49.31 | 45.61 | 12.91 | 31.73 | 22.65 | 12.54 | 4.59 | — | |
| GPT-5.12026.03 | 6.77 | — | — | — | — | — | — | — | — | |
| InternVL3-8BThinking=false2026.03 | 2.08 | 51.24 | 8.94 | 26.2 | 22.35 | 14.01 | 9.7 | 2.39 | — | |
| Gemini-2.0-FlashThinking=false2026.03 | 1.85 | 15.18 | 7.44 | 15.87 | 23.14 | 14.45 | 10.91 | 2.54 | — | |
| GPT-4o-2024-0806Thinking=false2026.03 | 1.5 | 13.33 | 5.89 | 5.94 | 22.8 | 14.34 | 10.42 | 2.6 | — | |
| Qwen2.5-VL-72B-InstructThinking=false2026.03 | 1.27 | 32.66 | 7.91 | 5.25 | 23.38 | 14.45 | 10.86 | 2.71 | — | |
| Qwen2.5-VL-32B-InstructThinking=false2026.03 | 0.98 | 27.81 | 7.91 | 3.12 | 22.79 | 14.68 | 10.09 | 2.53 | — | |
| Llama-4-Scout-17B-16E-InstructThinking=false2026.03 | 0.58 | 9.46 | 6.17 | 9.64 | 22.73 | 14.22 | 9.64 | 2.37 | — | |
| InternVL3-78BThinking=false2026.03 | 0.52 | 38.14 | 9.52 | 3.29 | 22.74 | 14.08 | 9.44 | 2.36 | — | |
| Mistral-Small-3.1-24B-Instruct-2503Thinking=false2026.03 | 0.46 | 9.81 | 8.02 | 5.48 | 22.43 | 14.05 | 9.41 | 2.35 | — | |
| Qwen2.5-VL-7B-InstructThinking=false2026.03 | 0.35 | 10.21 | 4.9 | 5.54 | 22.67 | 14.14 | 9.47 | 2.36 | — | |
| Qwen2.5-MaxThinking=false2026.03 | 0.35 | 7.21 | 4.85 | 5.94 | 22.4 | 14.12 | 9.59 | 2.39 | — | |
| Phi-4-MultimodalThinking=false2026.03 | 0.12 | 14.83 | 12.98 | 3.46 | 22.32 | 13.99 | 9.39 | 2.35 | — | |
| MiniCPM-O-2_6Thinking=false2026.03 | 0.06 | 24.99 | 7.79 | 5.71 | 22.49 | 14.02 | 9.4 | 2.35 | — | |
| Gemma3-27B-itThinking=false2026.03 | 0.06 | 8.89 | 7.39 | 1.56 | 22.44 | 14.01 | 9.44 | 2.35 | — | |
| LLaVA-1.5-7BThinking=false2026.03 | 0 | 3.69 | 9.41 | 0.23 | 22.48 | 14.12 | 9.41 | 2.35 | — | |
| MiniCPM-V-2_6Thinking=false2026.03 | 0 | 18.7 | 13.04 | 1.62 | 22.54 | 14.2 | 9.42 | 2.35 | — | |
| Skywork-R1V-38BThinking=false2026.03 | 0 | 19.85 | 9.69 | 1.67 | 22.42 | 14.02 | 9.41 | 2.35 | — | |
| Gemini 2.0 FlashEvaluation protocol=MCQ2026.06 | — | — | — | — | — | — | — | — | 2.5 | |
| GPT-4oEvaluation protocol=MCQ2026.06 | — | — | — | — | — | — | — | — | 2.6 | |
| InternVL3-8BEvaluation protocol=MCQ2026.06 | — | — | — | — | — | — | — | — | 2.4 | |
| Qwen2.5-VL-7BEvaluation protocol=MCQ2026.06 | — | — | — | — | — | — | — | — | 2.4 | |
| SurgAtlas-VLMEvaluation protocol=open-vocabulary generation2026.06 | — | — | — | — | — | — | — | — | 5.5 | |
| SurgVLM-72BEvaluation protocol=open-vocabulary generation2026.06 | — | — | — | — | — | — | — | — | 4.8 | |
| SurgVLM-7BEvaluation protocol=open-vocabulary generation2026.06 | — | — | — | — | — | — | — | — | 2.4 |