Autonomous Incident Management on AIOpsLab full benchmark (86 tasks)
100Detection Rate (best@k)AOI
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| AOIModel=Qwen3-14B, Sampling strategy=5-round sampling2026.03 | 100 | 66.9 | 53.6 | 27.9 | 30.8 | 7.7 | 46.2 | 23.1 | 66.3 | 38.6 | |
| AOIModel=GPT-4o-mini, Sampling strategy=single-run2026.03 | 90.6 | — | 32.1 | — | 38.5 | — | 53.8 | — | 58.1 | — | |
| STRATUSModel=GPT-4o-mini, Sampling strategy=single-run2026.03 | 78.1 | — | 25 | — | 15.4 | — | 23.1 | — | 43 | — | |
| STRATUSModel=Qwen3-14B, Sampling strategy=5-round sampling2026.03 | 75 | 41.3 | 32.1 | 11.4 | 7.7 | 4.6 | 15.4 | 15.4 | 41.9 | 22.1 | |
| AOL-agentModel=Claude Sonnet 4.5, Sampling strategy=single-run2026.03 | 68.8 | — | 53.6 | — | 15.4 | — | 76.9 | — | 57 | — | |
| AOL-agentModel=GPT-4o-mini, Sampling strategy=single-run2026.03 | 25 | — | 9.5 | — | 7.7 | — | 7.7 | — | 14.7 | — |