Step success rate on AndroidControl Task-Unseen
62.3SSRT3A* (Qwen2.5-7B-Instruct-ASL)
Evaluation results
| Method | Links | |
|---|---|---|
| T3A* (Qwen2.5-7B-Instruct-ASL)Type=Fine-Tuned, Agent=T3A*, Model=Qwen2.5-7B-Instruct-ASL (Ours), Input Modality=Text2025.06 | 62.3 | |
| T3A* (Qwen2.5-7B-Instruct-ASL)Type=Fine-Tuned, Agent=T3A*, Model=Qwen2.5-7B-Instruct-ASL (Ours), Input Modality=Text2025.06 | 61.8 | |
| T3A* (Qwen2.5-7B-Instruct-SFT)Type=Fine-Tuned, Agent=T3A*, Model=Qwen2.5-7B-Instruct-SFT, Input Modality=Text2025.06 | 61.2 | |
| M3A (GPT-4o)Type=Prompt-Driven, Agent=M3A, Model=GPT-4o, Input Modality=Image + Text2025.06 | 60.4 | |
| T3A* (Qwen2.5-7B-Instruct-SFT)Type=Fine-Tuned, Agent=T3A*, Model=Qwen2.5-7B-Instruct-SFT, Input Modality=Text2025.06 | 59.7 | |
| M3A (GPT-4o)Type=Prompt-Driven, Agent=M3A, Model=GPT-4o, Input Modality=Image + Text2025.06 | 59.3 | |
| T3A (GPT-4o)Type=Prompt-Driven, Agent=T3A, Model=GPT-4o, Input Modality=Text2025.06 | 55.8 | |
| T3A (GPT-4o)Type=Prompt-Driven, Agent=T3A, Model=GPT-4o, Input Modality=Text2025.06 | 54.2 | |
| M3A (Gemini-2.5-Flash)Type=Prompt-Driven, Agent=M3A, Model=Gemini-2.5-Flash, Input Modality=Image + Text2025.06 | 47.9 | |
| M3A (Gemini-2.5-Flash)Type=Prompt-Driven, Agent=M3A, Model=Gemini-2.5-Flash, Input Modality=Image + Text2025.06 | 47.9 | |
| T3A (Gemini-2.5-Flash)Type=Prompt-Driven, Agent=T3A, Model=Gemini-2.5-Flash, Input Modality=Text2025.06 | 45.4 | |
| T3A* (Gemini-2.5-Flash)Type=Prompt-Driven, Agent=T3A*, Model=Gemini-2.5-Flash, Input Modality=Text2025.06 | 45.3 | |
| T3A (Gemini-2.5-Flash)Type=Prompt-Driven, Agent=T3A, Model=Gemini-2.5-Flash, Input Modality=Text2025.06 | 45 | |
| T3A* (Gemini-2.5-Flash)Type=Prompt-Driven, Agent=T3A*, Model=Gemini-2.5-Flash, Input Modality=Text2025.06 | 43.5 | |
| SeeAct (Gemini-2.5-Flash)Type=Prompt-Driven, Agent=SeeAct, Model=Gemini-2.5-Flash, Input Modality=Image + Text2025.06 | 35.9 | |
| SeeAct (Gemini-2.5-Flash)Type=Prompt-Driven, Agent=SeeAct, Model=Gemini-2.5-Flash, Input Modality=Image + Text2025.06 | 34.4 | |
| SeeAct (GPT-4o)Type=Prompt-Driven, Agent=SeeAct, Model=GPT-4o, Input Modality=Image + Text2025.06 | 30.9 | |
| SeeAct (GPT-4o)Type=Prompt-Driven, Agent=SeeAct, Model=GPT-4o, Input Modality=Image + Text2025.06 | 30.7 | |
| T3A* (Qwen2.5-7B-Instruct)Type=Prompt-Driven, Agent=T3A*, Model=Qwen2.5-7B-Instruct, Input Modality=Text2025.06 | 27.2 | |
| T3A* (Qwen2.5-7B-Instruct)Type=Prompt-Driven, Agent=T3A*, Model=Qwen2.5-7B-Instruct, Input Modality=Text2025.06 | 26.3 |