GUI Agent Task Planning on AITW
36.34General Step AccuracyGemini-2.0-flash-exp + Qwen2-VL-7B (AutoGUI)
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini-2.0-flash-exp + Qwen2-VL-7B (AutoGUI)Planner=Gemini-2.0-flash-exp, Grounding Model=Qwen2-VL-7B SFT w/ AutoGUI2025.02 | 36.34 | 36.54 | 50.95 | 48.95 | 40.99 | 40.52 | 50.95 | 48.95 | 32.83 | 43.52 | 39.23 | |
| Gemini-2.0-flash-expPlanner=Gemini-2.0-flash-exp, Grounding Model=Default/Proprietary2025.02 | 26.37 | 18.16 | 28.49 | 26.91 | 30.3 | 22.88 | 41.94 | 28.95 | 20.22 | 22.65 | 29.5 | |
| GPT-4o-mini + Qwen2-VL-7B (AutoGUI)Planner=GPT-4o-mini, Grounding Model=Qwen2-VL-7B SFT w/ AutoGUI2025.02 | 20.43 | 20.56 | 25.59 | 22.49 | 15.25 | 12.33 | 25.59 | 22.49 | 16.15 | 20.53 | 18.37 | |
| GPT-4o-miniPlanner=GPT-4o-mini, Grounding Model=Default/Proprietary2025.02 | 14.85 | 9.58 | 11.17 | 5.76 | 12.08 | 6.85 | 21.09 | 11.24 | 10.89 | 11.22 | 14.01 |