Scenario-level policy synthesis on AppWorld normal (test)
98.2Task Goal Completion (TGC)HCL-GP
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| HCL-GPModel=Claude Sonnet 4.62026.05 | 98.2 | 98.2 | |
| GPModel=Claude Sonnet 4.6, Note=our baseline2026.05 | 96.4 | 96.4 | |
| Alibaba AgentRLModel=Qwen3-14B2026.05 | 86.9 | 80.4 | |
| IBM CUGAModel=GPT-4.12026.05 | 73.2 | 62.5 | |
| LOOPModel=Qwen2.5-32B2026.05 | 72.6 | 53.6 | |
| Smolagents CodeModel=Claude Opus 4.52026.05 | 70 | 60.4 | |
| OpenAI SoloModel=Claude Opus 4.52026.05 | 68 | 56.6 | |
| Claude CodeModel=Claude Opus 4.52026.05 | 66 | 56.6 | |
| ReAct ShortModel=Claude Opus 4.52026.05 | 64 | 54.7 | |
| HCL-GPModel=GPT-OSS 120B2026.05 | 62.5 | 62.5 | |
| ReActModel=Claude Opus 4.52026.05 | 61 | 52.8 | |
| GPModel=GPT-OSS 120B, Note=our baseline2026.05 | 0 | 0 |