Data Science Task Automation on xBench-DS
78ScoreMemEvolve + Flash-Searcher
Evaluation Results
| Method | Links | |
|---|---|---|
| MemEvolve + Flash-SearcherModel Family=GPT-5-mini, pass@k=32025.12 | 78 | |
| MemEvolve + Flash-SearcherModel Family=GPT-5-mini, pass@k=22025.12 | 77 | |
| MemEvolve + Flash-SearcherModel Family=GPT-5-mini, pass@k=12025.12 | 74 | |
| MemEvolve + Flash-SearcherModel Family=DeepSeek V3.2, pass@k=12025.12 | 70 | |
| Flash-SearcherModel Family=GPT-5-mini, pass@k=12025.12 | 69 | |
| Agent KBModel Family=GPT-4.1, pass@k=32025.12 | 68 | |
| Flash-SearcherModel Family=DeepSeek V3.2, pass@k=12025.12 | 68 | |
| MemEvolve + SmolAgentModel Family=GPT-5-mini, pass@k=32025.12 | 68 | |
| MemEvolve + Flash-SearcherModel Family=Kimi K2, pass@k=12025.12 | 68 | |
| Flash-SearcherModel Family=Kimi K2, pass@k=12025.12 | 66 | |
| MemEvolve + SmolAgentModel Family=GPT-5-mini, pass@k=22025.12 | 63 | |
| Agent KBModel Family=GPT-4.1, pass@k=22025.12 | 58 | |
| MemEvolve + SmolAgentModel Family=GPT-5-mini, pass@k=12025.12 | 57 | |
| Cognitive Kernel-ProModel Family=Claude-3.7 etc., pass@k=12025.12 | 56 | |
| OWL WorkforceModel Family=GPT-4o+o3-mini, pass@k=32025.12 | 55 | |
| SmolagentsModel Family=GPT-5-mini2025.12 | 51 | |
| Agent KBModel Family=GPT-4.1, pass@k=12025.12 | 48 | |
| OAgentsModel Family=Claude-3.7 etc.2025.12 | 47 |