Task Goal Completion on AppWorld
18.46Average Completion Score @4HINT-SD-Multi
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HINT-SD-Multibackbone=Qwen3-4B-Instruct-2507, distill_strategy=multiple selected failure-relevant steps, rollouts=42026.05 | 18.46 | 31.11 | |
| HINT-SD-Singlebackbone=Qwen3-4B-Instruct-2507, distill_strategy=first failure-relevant step, rollouts=42026.05 | 16.54 | 29.4 | |
| SDPObackbone=Qwen3-4B-Instruct-2507, rollouts=42026.05 | 9.74 | 19.32 | |
| OpenClaw-RLbackbone=Qwen3-4B-Instruct-2507, rollouts=42026.05 | 7.65 | 12.31 | |
| GRPObackbone=Qwen3-4B-Instruct-2507, rollouts=42026.05 | 7.49 | 15.21 | |
| SFTmode=supervised fine-tuning, backbone=Qwen3-4B-Instruct-25072026.05 | 6.82 | 13.16 | |
| Initialmode=zero-shot, backbone=Qwen3-4B-Instruct-25072026.05 | 5.98 | 13.85 |