Reward Modeling Evaluation on RewardBench2 (test)
82.9AccuracyACE + Batching
Evaluation results
| Method | Links | |
|---|---|---|
| ACE + BatchingAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 82.9 | |
| ACE + Optimizer StateAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 82.8 | |
| ACE + Auxiliary LossesAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 82.5 | |
| RCL (all primitives)Agent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 82.4 | |
| ACE + Grouped RolloutsAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 81.5 | |
| ACE + Failure ReplayAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 80.4 | |
| GEPAAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 79.2 | |
| ACEAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 79.2 | |
| ACE + Grouped RolloutsAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 77.8 | |
| ACE + Credit AssignmentAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 77.2 | |
| GEPAAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 76.5 | |
| SeedAgent Model=Gemini 3.1 Flash Lite, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 75.9 | |
| ACE + Auxiliary LossesAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 71.2 | |
| ACE + Optimizer StateAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 69.9 | |
| RCL (all primitives)Agent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 69.8 | |
| SeedAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 67.2 | |
| ACE + Failure ReplayAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 65.4 | |
| ACE + BatchingAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 63.6 | |
| ACE + Credit AssignmentAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 63.5 | |
| ACEAgent Model=GPT-5.4 Nano, Optimizer Model=Claude Opus, Optimization Iterations=302026.04 | 62.7 |