Mobile GUI Action Matching on AITW instruction-wise (test)
7.7Overall ErrorChatGPT-CoT
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| ChatGPT-CoTModality=Text2024.01 | 7.7 | 5.9 | 4.4 | 10.5 | 9.4 | 8.4 | — | |
| PaLM2-CoTModality=Text2024.01 | 39.6 | — | — | — | — | — | — | |
| GPT-4VModality=Image2024.01 | 50.5 | 41.7 | 42.6 | 49.8 | 72.8 | 45.7 | — | |
| Qwen-VLModality=Image2024.01 | 54.3 | 49.5 | 59.9 | 46.9 | 64.7 | 50.7 | 57.4 | |
| SeeClickModality=Image2024.01 | 59.3 | 54 | 66.4 | 54.9 | 63.5 | 57.6 | 66.4 |