Agentic Tool-use on AppWorld Normal
89.3Task Goal Completion (TGC)RCL (all primitives)
Evaluation results
| Method | Links | |
|---|---|---|
| RCL (all primitives)Agent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 89.3 | |
| ACE + Auxiliary LossesAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 89.2 | |
| RCL (all primitives)Agent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 89.1 | |
| ACE + BatchingAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 88.7 | |
| ACE + Optimizer StateAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 88.1 | |
| ACE + BatchingAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 87.2 | |
| ACEAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 86.9 | |
| ACE + Grouped RolloutsAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 86.8 | |
| ACE + Grouped RolloutsAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 86.3 | |
| ACE + Auxiliary LossesAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 86.3 | |
| ACE + Failure ReplayAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 84.5 | |
| ACE + Optimizer StateAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 84.5 | |
| ACE + Credit AssignmentAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 84.5 | |
| ACE + Credit AssignmentAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 83.9 | |
| ACEAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 83.3 | |
| GEPAAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 82.7 | |
| ACE + Failure ReplayAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 82.7 | |
| SeedAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 81.5 | |
| SeedAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 78 | |
| GEPAAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 76.2 |