Incident response evaluation on 200 incident response cases
0Violation RateHuman baseline
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Human baselineRuns=2002026.05 | 0 | 0 | 0 | 0 | 100 | 0 | |
| ClaudeArm=zero, Runs=2002026.05 | 36 | 29.67 | 36.5 | 36 | 75.67 | 6.78 | |
| GPTArm=policy, Runs=2002026.05 | 47 | 40.2 | 49 | 47 | 74 | 9.17 | |
| GPTArm=zero, Runs=2002026.05 | 54 | 47.08 | 56.5 | 54 | 81 | 10.92 | |
| ClaudeArm=policy, Runs=2002026.05 | 87 | 81.63 | 91 | 87 | 76.34 | 15.44 |