Web Navigation and Automation on WorkArena Held-out Goals (test)
53.8Success Rateo1-Mini
Evaluation Results
| Method | Links | |
|---|---|---|
| o1-MiniBackbone Model=o1-Mini2025.07 | 53.8 | |
| Claude-3.5-SonnetBackbone Model=Claude-3.5-Sonnet2025.07 | 52.5 | |
| GPT-4oBackbone Model=GPT-4o2025.07 | 42.1 | |
| Llama-3.1-405B-InstructBackbone Model=Llama-3.1-405B2025.07 | 39.2 | |
| Llama-3.3-70B Instruct (Teacher)Backbone Model=Llama-3.3-70B, Model Role=Teacher2025.07 | 36 | |
| Llama-3.1-8B SFT+RL (Ours)Backbone Model=Llama-3.1-8B, Training Protocol=SFT+RL2025.07 | 35.4 | |
| Qwen-2.5-72B Instruct (Teacher)Backbone Model=Qwen-2.5-72B, Model Role=Teacher2025.07 | 33.3 | |
| Qwen-2.5-7B SFT+RL (Ours)Backbone Model=Qwen-2.5-7B, Training Protocol=SFT+RL2025.07 | 32.5 | |
| Llama-3.1-8B SFT (Ours)Backbone Model=Llama-3.1-8B, Training Protocol=SFT2025.07 | 28.4 | |
| GPT-4o-MiniBackbone Model=GPT-4o-Mini2025.07 | 27.1 | |
| Llama-3.1-70B-InstructBackbone Model=Llama-3.1-70B2025.07 | 25 | |
| Qwen-2.5-7B SFT (Ours)Backbone Model=Qwen-2.5-7B, Training Protocol=SFT2025.07 | 25 | |
| Llama-3.1-8B Instruct (Student)Backbone Model=Llama-3.1-8B, Model Role=Student2025.07 | 8.3 | |
| Llama-3.1-8B RL (Ours)Backbone Model=Llama-3.1-8B, Training Protocol=RL2025.07 | 8 | |
| Qwen-2.5-7B Instruct (Student)Backbone Model=Qwen-2.5-7B, Model Role=Student2025.07 | 5.2 | |
| Qwen-2.5-7B RL (Ours)Backbone Model=Qwen-2.5-7B, Training Protocol=RL2025.07 | 4.5 |