Reward Modeling on OSWorld Verified Class-Balanced Human Evaluation 1.0 (test)
94PrecisionVAGEN
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| VAGENVerified Agent Backbone=Claude-Sonnet-4.52026.01 | 94 | 95.2 | 94.6 | 92.9 | |
| AndroidGenVerified Agent Backbone=Claude-Sonnet-4.52026.01 | 92.6 | 77 | 84.1 | 80.2 | |
| WebRLVerified Agent Backbone=Claude-Sonnet-4.52026.01 | 91.6 | 77.2 | 83.8 | 79.8 | |
| DistRLVerified Agent Backbone=Claude-Sonnet-4.52026.01 | 87.7 | 74.2 | 80.4 | 75.6 | |
| FullTrajEvalVerified Agent Backbone=Claude-Sonnet-4.52026.01 | 87.7 | 89.5 | 88.6 | 84.7 | |
| ZeroGUIVerified Agent Backbone=Claude-Sonnet-4.52026.01 | 87.1 | 88.2 | 87.7 | 83.7 | |
| DigiRLVerified Agent Backbone=Claude-Sonnet-4.52026.01 | 85.7 | 72.8 | 78.7 | 73.8 |