Web-based Task Automation on WebShop-AC 1.0 (test)
92.6TSRFragFuse
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| FragFuseAgent=RAP, Access Control (AC)=GuardAgent, Core LLM=GPT-4o2026.06 | 92.6 | 81 | 75 | |
| FragFuseAgent=RAP, Access Control (AC)=LLM-AC, Core LLM=GPT-4o2026.06 | 92.5 | 93 | 86 | |
| direct queryingAgent=RAP, Access Control (AC)=None, Core LLM=GPT-4o2026.06 | 88 | — | 88 | |
| FragFuseAgent=RAP, Access Control (AC)=GuardAgent, Core LLM=Average2026.06 | 82.2 | 79.3 | 65.3 | |
| FragFuseAgent=OSAgent, Access Control (AC)=LLM-AC, Core LLM=GPT-4o2026.06 | 80.5 | 82 | 66 | |
| direct queryingAgent=OSAgent, Access Control (AC)=None, Core LLM=GPT-4o2026.06 | 80 | — | 80 | |
| FragFuseAgent=OSAgent, Access Control (AC)=Agrail, Core LLM=Average2026.06 | 77.6 | 92.7 | 72 | |
| baselineAgent=RAP, Access Control (AC)=LLM-AC, Core LLM=GPT-4o2026.06 | 77.5 | 40 | 31 | |
| baselineAgent=RAP, Access Control (AC)=GuardAgent, Core LLM=GPT-4o2026.06 | 76.2 | 42 | 32 | |
| FragFuseAgent=OSAgent, Access Control (AC)=Agrail, Core LLM=GPT-4o2026.06 | 70.5 | 88 | 62 | |
| FragFuseAgent=RAP, Access Control (AC)=LLM-AC, Core LLM=GPT-5.12026.06 | 70.4 | 98 | 69 | |
| baselineAgent=RAP, Access Control (AC)=GuardAgent, Core LLM=Average2026.06 | 51.8 | 35.3 | 17.3 | |
| baselineAgent=InspAgent, Access Control (AC)=LLM-AC, Core LLM=GPT-4o2026.06 | 50 | 3.4 | 1.7 | |
| baselineAgent=OSAgent, Access Control (AC)=Agrail, Core LLM=GPT-4o2026.06 | 40 | 10 | 4 | |
| direct queryingAgent=InspAgent, Access Control (AC)=None, Core LLM=GPT-4o2026.06 | 38.6 | — | 38.6 | |
| baselineAgent=SeeAct, Access Control (AC)=LLM-AC, Core LLM=GPT-4o2026.06 | 33.3 | 3 | 1 | |
| FragFuseAgent=InspAgent, Access Control (AC)=LLM-AC, Core LLM=GPT-4o2026.06 | 25 | 45.5 | 11.4 | |
| FragFuseAgent=SeeAct, Access Control (AC)=Agrail, Core LLM=GPT-4o2026.06 | 23.6 | 72 | 17 | |
| direct queryingAgent=SeeAct, Access Control (AC)=None, Core LLM=GPT-4o2026.06 | 22 | — | 22 | |
| FragFuseAgent=SeeAct, Access Control (AC)=LLM-AC, Core LLM=GPT-4o2026.06 | 20.5 | 88 | 18 | |
| baselineAgent=SeeAct, Access Control (AC)=Agrail, Core LLM=GPT-4o2026.06 | 20 | 5 | 1 | |
| FragFuseAgent=SeeAct, Access Control (AC)=LLM-AC, Core LLM=Average2026.06 | 19.4 | 93 | 18 | |
| baselineAgent=RAP, Access Control (AC)=LLM-AC, Core LLM=GPT-5.12026.06 | 17.8 | 45 | 8 | |
| FragFuseAgent=InspAgent, Access Control (AC)=ShieldAgent, Core LLM=GPT-4o2026.06 | 17.2 | 82.4 | 14.2 | |
| FragFuseAgent=InspAgent, Access Control (AC)=ShieldAgent, Core LLM=Average2026.06 | 13.1 | 88.3 | 11.1 | |
| baselineAgent=OSAgent, Access Control (AC)=LLM-AC, Core LLM=GPT-4o2026.06 | 0 | 0 | 0 | |
| baselineAgent=InspAgent, Access Control (AC)=ShieldAgent, Core LLM=GPT-4o2026.06 | 0 | 1.1 | 0 |