Task Success on MemGUI-Bench 1.0 (test)
66.7Pass@1 L1Claude-Sonnet-4.6
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Claude-Sonnet-4.6Model Category=General-purpose Multimodal Model + Active Memory Instruction (AMI)2026.05 | 66.7 | 70.4 | 51.9 | 63 | 77.8 | 77.8 | 70.4 | 75.3 | |
| Gemini-3.1-ProModel Category=General-purpose Multimodal Model + Active Memory Instruction (AMI)2026.05 | 48.1 | 37 | 22.2 | 35.8 | 74.1 | 51.9 | 44.4 | 56.8 | |
| GPT-5.4Model Category=General-purpose Multimodal Model + Active Memory Instruction (AMI)2026.05 | 37 | 30.8 | 33.3 | 33.7 | 70.4 | 59.3 | 74.1 | 67.9 | |
| Stamp-GUITraining Initialization=GUI-Owl-1.5-8B-Think2026.05 | 25.9 | 22.2 | 22.2 | 23.4 | 37 | 33.3 | 40.7 | 37 | |
| MAI-UI-8BModel Category=GUI-specialized Model + History-Retention Protocol (HRP)2026.05 | 22.2 | 14.8 | 18.5 | 18.5 | 29.6 | 22.2 | 22.2 | 24.7 | |
| GUI-Owl-1.5-8B-ThinkModel Category=GUI-specialized Model + History-Retention Protocol (HRP)2026.05 | 22.2 | 18.5 | 18.5 | 19.7 | 25.9 | 22.2 | 25.9 | 24.7 | |
| Qwen3-VL-8B-ThinkModel Category=General-purpose Multimodal Model + Active Memory Instruction (AMI)2026.05 | 18.5 | 7.4 | 3.7 | 9.9 | 40.7 | 14.8 | 14.8 | 23.4 | |
| Qwen3-VL-8B-InstructModel Category=General-purpose Multimodal Model + Active Memory Instruction (AMI)2026.05 | 11.1 | 0 | 7.4 | 6.2 | 25.9 | 7.4 | 18.5 | 17.3 | |
| UI-Tars-1.5-7BModel Category=GUI-specialized Model + History-Retention Protocol (HRP)2026.05 | 7.4 | 7.4 | 0 | 4.9 | 14.8 | 14.8 | 7.4 | 12.3 | |
| UI-Venus-1.5-8BModel Category=GUI-specialized Model + History-Retention Protocol (HRP)2026.05 | 3.7 | 7.4 | 14.8 | 8.6 | 18.5 | 11.1 | 25.9 | 18.5 | |
| CogAgent-9BModel Category=GUI-specialized Model + History-Retention Protocol (HRP)2026.05 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |