Penetration Testing on AutoPenBench
52.5Pass@1Seed Agent
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Seed AgentBackbone=Kimi-K2.52026.05 | 52.5 | 60.3 | 60.6 | — | — | — | — | — | — | |
| Seed AgentBackbone=DeepSeek-V3.12026.05 | 34.4 | 50.3 | 51.5 | — | — | — | — | — | — | |
| Seed AgentBackbone=MiniMax-M2.52026.05 | 29.4 | 41.2 | 42.4 | — | — | — | — | — | — | |
| Seed AgentBackbone=Qwen3-235B2026.05 | 13.4 | 22.9 | 24.3 | — | — | — | — | — | — | |
| ACEBackbone=DeepSeek-V3.12026.05 | — | — | — | 36.4 | -15.1 | — | — | — | — | |
| ACEBackbone=Kimi-K2.52026.05 | — | — | — | 45.5 | -15.1 | — | — | — | — | |
| ACEBackbone=MiniMax-M2.52026.05 | — | — | — | 36.4 | -6 | — | — | — | — | |
| ACEBackbone=Qwen3-235B2026.05 | — | — | — | 36.4 | 12.1 | — | — | — | — | |
| AutoPenBench-AgentBackbone=DeepSeek-V3.12026.05 | — | — | — | — | — | — | — | 36.4 | — | |
| AutoPenBench-AgentBackbone=Kimi-K2.52026.05 | — | — | — | — | — | — | — | 42.4 | — | |
| AutoPenBench-AgentBackbone=MiniMax-M2.52026.05 | — | — | — | — | — | — | — | 27.3 | — | |
| AutoPenBench-AgentBackbone=Qwen3-235B2026.05 | — | — | — | — | — | — | — | 27.3 | — | |
| CyberEvolverBackbone=DeepSeek-V3.12026.05 | — | — | — | — | — | 60.6 | 9.1 | — | — | |
| CyberEvolverBackbone=Kimi-K2.52026.05 | — | — | — | — | — | 72.7 | 12.1 | — | — | |
| CyberEvolverBackbone=MiniMax-M2.52026.05 | — | — | — | — | — | 66.7 | 24.3 | — | — | |
| CyberEvolverBackbone=Qwen3-235B2026.05 | — | — | — | — | — | 63.6 | 39.3 | — | — | |
| VulnBotBackbone=DeepSeek-V3.12026.05 | — | — | — | — | — | — | — | — | 33.3 | |
| VulnBotBackbone=Kimi-K2.52026.05 | — | — | — | — | — | — | — | — | 39.4 | |
| VulnBotBackbone=MiniMax-M2.52026.05 | — | — | — | — | — | — | — | — | 33.3 | |
| VulnBotBackbone=Qwen3-235B2026.05 | — | — | — | — | — | — | — | — | 12.1 |