Web-based Interaction on MiniWoB
74.7Success RateGemini 3 Pro
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini 3 ProCategory=Proprietary, Teacher for A3-SYNTH=true2026.04 | 74.7 | |
| Gemini 3.1 Flash L.Category=Proprietary2026.04 | 74.1 | |
| GPT-5Harness=GenericAgent, Params=-2026.04 | 71.5 | |
| Qwen3.5-27BCategory=Open-weight (base)2026.04 | 70.9 | |
| Claude 4 SonnetHarness=GenericAgent, Params=-2026.04 | 70.7 | |
| A3-Qwen3.5-9BCategory=A3 fine-tuned, Teacher=Gemini 3 Pro2026.04 | 69 | |
| A3-Qwen3.5-9BHarness=GenericAgent, Params=9B2026.04 | 69 | |
| A3-Qwen3.5-4BCategory=A3 fine-tuned, Teacher=Gemini 3 Pro2026.04 | 66.9 | |
| GPT-oss-120BHarness=GenericAgent, Params=120B2026.04 | 66.4 | |
| Qwen3.5-9BCategory=Open-weight (base)2026.04 | 63.2 | |
| Qwen3.5-9B (base)Harness=GenericAgent, Params=9B2026.04 | 63.2 | |
| Pruning + LLM-Judge (10K steps)Model=Qwen3-8B, # Data=10K, # Epochs=2, Time=7.0 hr, Speed-up=12.6×2026.05 | 61.9 | |
| WEASEL (10K steps)Model=Qwen3-8B, # Data=10K, # Epochs=2, Time=8.3 hr, Speed-up=10.7×2026.05 | 61.9 | |
| Pruning + Sampling (10K steps)Model=Qwen3-8B, # Data=10K, # Epochs=2, Time=7.0 hr, Speed-up=12.6×2026.05 | 61.4 | |
| Qwen3.5-4BCategory=Open-weight (base)2026.04 | 61.1 | |
| Qwen3-8BModel=Qwen3-8B2026.05 | 61.1 | |
| Full (52K steps)Model=Qwen3-8B, # Data=52K, # Epochs=2, Time=88.5 hr, Speed-up=1.0×2026.05 | 59.4 | |
| WEASEL (10K steps)Model=Qwen2.5-7B-Instruct, # Data=10K, # Epochs=4, Time=12.0 hr, Speed-up=11.3×2026.05 | 48 | |
| Pruning (52K steps)Model=Qwen2.5-7B-Instruct, # Data=52K, # Epochs=4, Time=62.0 hr, Speed-up=2.2×2026.05 | 47.7 | |
| Pruning + Sampling (10K steps)Model=Qwen2.5-7B-Instruct, # Data=10K, # Epochs=4, Time=12.0 hr, Speed-up=11.3×2026.05 | 46.7 | |
| Pruning + LLM-Judge (10K steps)Model=Qwen2.5-7B-Instruct, # Data=10K, # Epochs=4, Time=12.0 hr, Speed-up=11.3×2026.05 | 45.4 | |
| Full (52K steps)Model=Qwen2.5-7B-Instruct, # Data=52K, # Epochs=4, Time=136.0 hr, Speed-up=1.0×2026.05 | 44.6 | |
| Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct2026.05 | 41.8 | |
| Pruning (52K steps)Model=Qwen3-8B, # Data=52K, # Epochs=2, Time=44.0 hr, Speed-up=2.0×2026.05 | 40.3 | |
| A3-Qwen3.5-2BCategory=A3 fine-tuned, Teacher=Gemini 3 Pro2026.04 | 38.6 | |
| WEASEL (10K steps)Model=Gemma3-4B-IT, # Data=10K, # Epochs=2, Time=6.4 hr, Speed-up=12.5×2026.05 | 30.6 | |
| Pruning + Sampling (10K steps)Model=Gemma3-4B-IT, # Data=10K, # Epochs=2, Time=6.4 hr, Speed-up=12.5×2026.05 | 29.4 | |
| Pruning + LLM-Judge (10K steps)Model=Gemma3-4B-IT, # Data=10K, # Epochs=2, Time=6.4 hr, Speed-up=12.5×2026.05 | 29.3 | |
| Full (52K steps)Model=Gemma3-4B-IT, # Data=52K, # Epochs=2, Time=80.0 hr, Speed-up=1.0×2026.05 | 28.6 | |
| Pruning (52K steps)Model=Gemma3-4B-IT, # Data=52K, # Epochs=2, Time=30.0 hr, Speed-up=2.7×2026.05 | 28.2 | |
| Gemma3-4B-ITModel=Gemma3-4B-IT2026.05 | 27.4 | |
| Qwen3.5-2BCategory=Open-weight (base)2026.04 | 11.8 |