Web Agent Task Success on τ-bench
76.52Retail Success RateAsym-AC-SFT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Asym-AC-SFTActor model=Claude-4 Sonnet, Critic model=Qwen3-8B, Task-specific fine-tuning=true2026.03 | 76.52 | — | — | 54.4 | 65.46 | |
| Asym-ACActor model=Claude-4 Sonnet, Critic=Off-the-shelf, Task-specific fine-tuning=false2026.03 | 73.56 | — | — | 52.8 | 63.18 | |
| ReActActor model=Claude-4 Sonnet, Implementation=native ReAct2026.03 | 71.3 | — | — | 50.4 | 60.85 | |
| CRITICActor model=Claude-4 Sonnet, Critique mode=in-context critique2026.03 | 70.61 | — | — | 52 | 61.31 | |
| xLAM-2-70b-fc-rActor model=Claude-4 Sonnet, Source=APIGen-MT2026.03 | 67.1 | — | — | 45.2 | 56.2 | |
| IRMAActor model=Claude-4 Sonnet2026.03 | 58.3 | — | — | 47.2 | 52.75 | |
| Qwen 2.5 7BDefense=None, Stage=Baseline, Poison %=02025.10 | — | 22.61 | — | — | — | |
| Qwen 2.5 7B (Environment Poisoning)Defense=None, Stage=SFT, Poison %=502025.10 | — | 39.13 | 100 | — | — | |
| Qwen 2.5 7B (Environment Poisoning)Defense=Llama-firewall (LF), Stage=SFT, Poison %=502025.10 | — | 41.16 | 100 | — | — | |
| Qwen 2.5 7B (Environment Poisoning)Defense=Granite Guardian 3.3-8B (GG), Stage=SFT, Poison %=502025.10 | — | 37.68 | 100 | — | — |