Standard Operating Procedure execution on SOPBench bank 40-task (val)
76.87Success Rateo4-mini-high*
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| o4-mini-high*Topology=single, Search cost=—2026.03 | 76.87 | 17.91 | |
| ABSTRAL full (val)Backbone=GPT-4o, Topology=ensemble, 6 agents, Search cost=154K tokens, Evaluation protocol=identical to published baselines2026.03 | 70 | 11.04 | |
| GPT-4.1*Topology=single, Search cost=—2026.03 | 69.4 | 10.44 | |
| Claude-3.7-Sonnet*Topology=single, Search cost=—2026.03 | 65.67 | 6.71 | |
| GPT-4o*Topology=single, Search cost=—2026.03 | 58.96 | — | |
| ABSTRAL inner-only (O1)Backbone=GPT-4o, Topology=hierarchical, 5 agents, Search cost=54K tokens, Evaluation protocol=identical to published baselines2026.03 | 55 | 3.96 | |
| GPT-4o single-agent (ours)Backbone=GPT-4o, Topology=single, 1 agent, Search cost=18K tokens, Evaluation protocol=identical to published baselines2026.03 | 50 | 8.96 | |
| GPT-4o-mini*Topology=single, Search cost=—2026.03 | 33.58 | 25.38 |