Agent Planning and Execution on TaskCraft
0.7533pass@1Agent KB
Evaluation results
| Method | Links | |
|---|---|---|
| Agent KBModel Family=GPT-4.1, pass@k protocol=pass@32026.02 | 0.7533 | |
| Agent KBModel Family=GPT-4.1, pass@k protocol=pass@22026.02 | 0.7267 | |
| TodoEvolve + SmolagentsModel Family=GPT-5-Mini, pass@k protocol=pass@12026.02 | 0.7267 | |
| TodoEvolve + SmolagentsModel Family=DeepSeek V3.2, pass@k protocol=pass@12026.02 | 0.7133 | |
| Flash-SearcherModel Family=GPT-5-mini, pass@k protocol=pass@12026.02 | 0.6967 | |
| Flash-SearcherModel Family=DeepSeek V3.2, pass@k protocol=pass@12026.02 | 0.6933 | |
| TodoEvolve + SmolagentsModel Family=Kimi K2, pass@k protocol=pass@12026.02 | 0.6933 | |
| Cognitive Kernel-ProModel Family=Claude-3.7 etc., pass@k protocol=pass@12026.02 | 0.66 | |
| SmolagentsModel Family=GPT-5-mini, pass@k protocol=pass@12026.02 | 0.64 | |
| Agent KBModel Family=GPT-4.1, pass@k protocol=pass@12026.02 | 0.6167 | |
| OWL WorkforceModel Family=GPT-4o+o3-mini, pass@k protocol=pass@32026.02 | 0.5833 | |
| Flash-SearcherModel Family=Kimi K2, pass@k protocol=pass@12026.02 | 0.58 |