Agentic Instruction Following on TaskCraft
79.33Performance ScoreMemEvolve + Flash-Searcher
Evaluation Results
| Method | Links | |
|---|---|---|
| MemEvolve + Flash-SearcherModel Family=GPT-5-mini, pass@k=32025.12 | 79.33 | |
| MemEvolve + SmolAgentModel Family=GPT-5-mini, pass@k=32025.12 | 77 | |
| Agent KBModel Family=GPT-4.1, pass@k=32025.12 | 75.33 | |
| MemEvolve + SmolAgentModel Family=GPT-5-mini, pass@k=22025.12 | 75 | |
| MemEvolve + Flash-SearcherModel Family=GPT-5-mini, pass@k=22025.12 | 75 | |
| Agent KBModel Family=GPT-4.1, pass@k=22025.12 | 72.67 | |
| MemEvolve + Flash-SearcherModel Family=DeepSeek V3.2, pass@k=12025.12 | 72.67 | |
| MemEvolve + Flash-SearcherModel Family=GPT-5-mini, pass@k=12025.12 | 72 | |
| Flash-SearcherModel Family=GPT-5-mini, pass@k=12025.12 | 69.67 | |
| Flash-SearcherModel Family=DeepSeek V3.2, pass@k=12025.12 | 69.33 | |
| MemEvolve + Flash-SearcherModel Family=Kimi K2, pass@k=12025.12 | 68 | |
| MemEvolve + SmolAgentModel Family=GPT-5-mini, pass@k=12025.12 | 67.67 | |
| Cognitive Kernel-ProModel Family=Claude-3.7 etc., pass@k=12025.12 | 66 | |
| SmolagentsModel Family=GPT-5-mini2025.12 | 64 | |
| Agent KBModel Family=GPT-4.1, pass@k=12025.12 | 61.67 | |
| OWL WorkforceModel Family=GPT-4o+o3-mini, pass@k=32025.12 | 58.33 | |
| Flash-SearcherModel Family=Kimi K2, pass@k=12025.12 | 58 |