Task Completion on SOP-Bench
100AccuracyGA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GAModel=Claude Sonnet 4.62026.06 | 100 | 2.02 | 53 | 2.08 | 0.48 | |
| OpenClawModel=Claude Sonnet 4.62026.06 | 100 | 2.6 | 40 | 2.64 | 0.38 | |
| GAModel=deepseek-v4-pro2026.06 | 100 | 857 | 45 | 902 | 22.18 | |
| BA-FullModel=deepseek-v4-pro2026.06 | 100 | 914 | 51 | 965 | 20.73 | |
| BA-IncModel=deepseek-v4-pro2026.06 | 100 | 0 | 0 | 0 | — | |
| BA-FullModel=deepseek-v4-flash2026.06 | 95 | 1.2 | 60 | 1.26 | 15.13 | |
| BA-IncModel=deepseek-v4-flash2026.06 | 95 | 145 | 8 | 153 | 19.63 | |
| Claude CodeModel=Claude Sonnet 4.62026.06 | 85 | 1.23 | 23 | 1.25 | 0.68 | |
| GAModel=deepseek-v4-flash2026.06 | 80 | 1.34 | 57 | 1.39 | 11.47 |