Medical GUI task automation on CareFlow 1.0 (test) benchmark leaderboard · SOTA2 Research

Medical GUI task automation on CareFlow 1.0 (test)

92.5Weasis Success With Action (SWA)CarePilot

Evaluation Results

Method	Links
CarePilot2026.03		92.5	48.76	88.9	54.8	91.8	56.67	87.52	46.25	90.18	51.45
CarePilot2026.03		90.38	40	82.09	54.75	93.8	55	90.18	56.7	88.05	48.9
GPT-52026.03		88.72	31.25	81.42	37.9	86.92	46.67	83.82	31.25	85.22	36.19
Llama 4 Maverick2026.03		88.21	18.69	71.55	3.4	84.99	27.99	77.97	25.68	80.53	19.2
Mistral3.2 VL 24B2026.03		88.15	5.13	64.81	0.67	68.44	0.79	61.43	0	70.65	1.67
Nemotron 12B VL2026.03		86.98	12.82	73.95	5.13	73.56	14.46	66.55	12.36	77.93	10.71
Llama 3.2 11B2026.03		86.03	10.26	76.47	4.6	69.58	13.58	62.56	11.47	75.65	9.5
GPT-4o2026.03		85.3	20	77.5	27.37	88.5	26.67	85.1	27.5	83.13	25.4
Llama 4 Scout2026.03		84.34	10.26	78.47	2.7	88.56	23.9	85.55	21.79	85.65	13.5
Qwen3 VL 235B2026.03		83.14	17.69	72.44	5.3	87.48	25.4	84.46	24.52	81.85	19.7
Qwen2.5 VL 32B2026.03		79.95	5.13	68	1.9	48.62	2.42	41.6	0.32	60.72	2.43
Gemini 2.5 Pro2026.03		68.9	3.75	59.7	5.26	71.3	6.66	61.7	6.75	65.15	5.39
Qwen 3 VL 8B2026.03		65.2	1.33	68.38	3.33	67.71	5	66.31	5	66.9	3.66
Qwen 2.5 VL 7B2026.03		58.64	1.33	61.4	1.66	65.5	2.5	63.2	1.66	57.18	1.78