Step success rate on AndroidControl (Cat-Unseen)
61.2Step Success Rate (SSR)T3A* (Qwen2.5-7B-Instruct-ASL)
Evaluation results
| Method | Links | |
|---|---|---|
| T3A* (Qwen2.5-7B-Instruct-ASL)Type=Fine-Tuned, Agent=T3A*, Model=Qwen2.5-7B-Instruct-ASL (Ours), Input Modality=Text2025.06 | 61.2 | |
| M3A (GPT-4o)Type=Prompt-Driven, Agent=M3A, Model=GPT-4o, Input Modality=Image + Text2025.06 | 60.8 | |
| T3A* (Qwen2.5-7B-Instruct-SFT)Type=Fine-Tuned, Agent=T3A*, Model=Qwen2.5-7B-Instruct-SFT, Input Modality=Text2025.06 | 59.7 | |
| T3A (GPT-4o)Type=Prompt-Driven, Agent=T3A, Model=GPT-4o, Input Modality=Text2025.06 | 56.5 | |
| M3A (Gemini-2.5-Flash)Type=Prompt-Driven, Agent=M3A, Model=Gemini-2.5-Flash, Input Modality=Image + Text2025.06 | 47.9 | |
| T3A* (Gemini-2.5-Flash)Type=Prompt-Driven, Agent=T3A*, Model=Gemini-2.5-Flash, Input Modality=Text2025.06 | 44.7 | |
| T3A (Gemini-2.5-Flash)Type=Prompt-Driven, Agent=T3A, Model=Gemini-2.5-Flash, Input Modality=Text2025.06 | 44.4 | |
| SeeAct (Gemini-2.5-Flash)Type=Prompt-Driven, Agent=SeeAct, Model=Gemini-2.5-Flash, Input Modality=Image + Text2025.06 | 33.7 | |
| SeeAct (GPT-4o)Type=Prompt-Driven, Agent=SeeAct, Model=GPT-4o, Input Modality=Image + Text2025.06 | 30.6 | |
| T3A* (Qwen2.5-7B-Instruct)Type=Prompt-Driven, Agent=T3A*, Model=Qwen2.5-7B-Instruct, Input Modality=Text2025.06 | 26.7 |