Agent Utility Evaluation on PinchBench v2.0.0
89.6Utility ScoreOutput guardrail
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Output guardrailModel=Claude Sonnet 4.62026.07 | 89.6 | 13.2 | |
| Input guardrailModel=Claude Sonnet 4.62026.07 | 88.8 | 5.3 | |
| BaselineModel=Claude Sonnet 4.62026.07 | 88.5 | 0 | |
| Dual LLMModel=Claude Sonnet 4.62026.07 | 86.3 | 61 | |
| Input guardrailModel=Claude Haiku 4.52026.07 | 85.3 | 18.3 | |
| Output guardrailModel=Claude Haiku 4.52026.07 | 84.7 | 14.4 | |
| BaselineModel=Claude Haiku 4.52026.07 | 83.4 | 0 | |
| DUALVIEWModel=Claude Sonnet 4.62026.07 | 82.1 | 93.4 | |
| Dual LLMModel=Claude Haiku 4.52026.07 | 81.8 | 40.8 | |
| DUALVIEWModel=Claude Haiku 4.52026.07 | 81.6 | 48.2 | |
| SandboxingModel=Claude Sonnet 4.62026.07 | 66.5 | 115.6 | |
| SandboxingModel=Claude Haiku 4.52026.07 | 62.5 | 57.6 |