Action Prediction on Mind2Web (Cross-Website)
57.2Element AccuracyMagma-8B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Magma-8BBackbone=LLaMA3 [92], DoM Tree=true2025.02 | 57.2 | 76.9 | 45.4 | — | |
| SKILL.nbZero-shot=true2026.06 | 46.8 | 54.3 | 38.1 | 4.1 | |
| ReasoningBankZero-shot=true2026.06 | 44.3 | 52.6 | 34.9 | 3.1 | |
| AWMonlineZero-shot=true2026.06 | 42.1 | 45.1 | 31.6 | 1.6 | |
| MindActw/o HTML=false2024.01 | 42 | 65.2 | 38.9 | — | |
| MindActrequired HTML text inputs=true2024.11 | 42 | 65.2 | 38.9 | — | |
| ShowUIFT?=true2024.11 | 41.6 | 83.5 | 35.1 | — | |
| OmniParser2024.11 | 41 | 84.8 | 36.5 | — | |
| ShowUI †FT?=true, Action history only=true2024.11 | 41 | 83.6 | 34.2 | — | |
| GPT-4V-OmniParserBackbone=GPT-4V [99], DoM Tree=true, Image=true2025.02 | 41 | 84.8 | 36.5 | — | |
| CodeActZero-shot=true2026.06 | 39.8 | 45.1 | 31.7 | 1.7 | |
| Qwen2-UIXBackbone=Qwen2 [124]2025.02 | 39.2 | — | 31 | — | |
| SeeAct (GPT-4V)Backbone=GPT-4V [99], Image=true2025.02 | 38 | 67.8 | 32.4 | — | |
| Qwen2-VL-2BFT?=true2024.11 | 36 | 79.2 | 27.6 | — | |
| GPT-4w/o HTML=false2024.01 | 35.8 | 51.1 | 30.1 | — | |
| GPT-4close-source=true2024.11 | 35.8 | 51.1 | 30.1 | — | |
| GPT-4-MindActBackbone=GPT-4 [98], Image=true2025.02 | 35.8 | 51.1 | 30.1 | — | |
| DUAL-VCRRanker=DUAL-VCR_VNEI-TXT+VIS, Action Predictor=DUAL-VCRPRED2024.02 | 32.7 | 72 | 32.5 | — | |
| DUAL-VCRRanker=DUAL-VCR_VNEI-TXT, Action Predictor=DUAL-VCRPRED2024.02 | 32 | 71.5 | 31.5 | — | |
| MindActRanker=MINDACTRANK, Action Predictor=MINDACTPRED2024.02 | 30.7 | 67 | 30 | — | |
| CogAgentBackbone=CogVLM [118]2025.02 | 27.3 | — | 23.4 | — | |
| DAVELanguage Model Backbone=Llama-3.2-3B-Instruct2025.12 | 24.2 | — | 18 | — | |
| ShowUI-ZSFT?=false2024.11 | 21.9 | 81.9 | 16.8 | — | |
| SeeAct (Gemini-Pro)Backbone=Gemini-Pro [36], Image=true2025.02 | 21.5 | 67.7 | 19.6 | — | |
| SeeClickw/o HTML=true2024.01 | 21.4 | 80.6 | 16.4 | — | |
| SeeClickFT?=true2024.11 | 21.4 | 80.6 | 16.4 | — | |
| SeeClickBackbone=Qwen-VL [3]2025.02 | 21.4 | 80.6 | 16.4 | — | |
| DolphinLanguage Model Backbone=Llama-3.2-3B-Instruct2025.12 | 20.9 | — | 13.6 | — | |
| MiniCPM-V-GUIBackbone=MiniCPM-V [128], DoM Tree=true2025.02 | 20.3 | 81.7 | 17.3 | — | |
| GPT-3.5-Turbow/o HTML=false2024.01 | 19.3 | 48.8 | 16.2 | — | |
| CogAgentFT?=true2024.11 | 18.4 | 42.4 | 13.4 | — | |
| MAELanguage Model Backbone=Llama-3.2-3B-Instruct2025.12 | 16.5 | — | 11.9 | — | |
| MindAct (gen)w/o HTML=false2024.01 | 13.9 | 44.7 | 11 | — | |
| Fuyu-8B-GUIBackbone=Fuyu-8B [4], DoM Tree=true2025.02 | 13.9 | 80.7 | 12.2 | — | |
| Qwen-VLw/o HTML=true2024.01 | 13.2 | 83.5 | 9.2 | — | |
| Qwen-VLFT?=true2024.11 | 13.2 | 83.5 | 9.2 | — | |
| Qwen-VLBackbone=Qwen-VL [3]2025.02 | 13.2 | 83.5 | 9.2 | — | |
| Pix2StructLanguage Model Backbone=Llama-3.2-3B-Instruct2025.12 | 12.1 | — | 8.2 | — | |
| SigLIP 2Language Model Backbone=Llama-3.2-3B-Instruct2025.12 | 11.8 | — | 8.7 | — | |
| DinoV2Language Model Backbone=Llama-3.2-3B-Instruct2025.12 | 10.2 | — | 6.6 | — | |
| AIMv2Language Model Backbone=Llama-3.2-3B-Instruct2025.12 | 9.2 | — | 5.9 | — | |
| MiniCPM-V+Backbone=MiniCPM-V [128], DoM Tree=true2025.02 | 8.2 | 78.2 | 6 | — | |
| Fuyu-8B+Backbone=Fuyu-8B [4], DoM Tree=true2025.02 | 4.8 | 81.3 | 4 | — | |
| DiTLanguage Model Backbone=Llama-3.2-3B-Instruct2025.12 | 2.7 | — | 1.2 | — | |
| GPT-4VBackbone=GPT-4V [99], Image=true2025.02 | — | 13.9 | — | — |