CTF Challenges on NYU-CTF
30.4Pass@1Seed Agent
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Seed AgentBackbone=Kimi-K2.52026.05 | 30.4 | 38.6 | 39.6 | — | — | — | — | — | — | |
| Seed AgentBackbone=MiniMax-M2.52026.05 | 17.7 | 22 | 22.4 | — | — | — | — | — | — | |
| Seed AgentBackbone=DeepSeek-V3.12026.05 | 14.5 | 19.4 | 20.3 | — | — | — | — | — | — | |
| Seed AgentBackbone=Qwen3-235B2026.05 | 14.5 | 19.5 | 20.3 | — | — | — | — | — | — | |
| ACEBackbone=DeepSeek-V3.12026.05 | — | — | — | 24.5 | 4.2 | — | — | — | — | |
| ACEBackbone=Kimi-K2.52026.05 | — | — | — | 34.4 | -5.2 | — | — | — | — | |
| ACEBackbone=MiniMax-M2.52026.05 | — | — | — | 24.5 | 2.1 | — | — | — | — | |
| ACEBackbone=Qwen3-235B2026.05 | — | — | — | 17.2 | -3.1 | — | — | — | — | |
| CyberEvolverBackbone=DeepSeek-V3.12026.05 | — | — | — | — | — | 35.5 | 15.2 | — | — | |
| CyberEvolverBackbone=Kimi-K2.52026.05 | — | — | — | — | — | 54.7 | 15.1 | — | — | |
| CyberEvolverBackbone=MiniMax-M2.52026.05 | — | — | — | — | — | 32.9 | 10.5 | — | — | |
| CyberEvolverBackbone=Qwen3-235B2026.05 | — | — | — | — | — | 29.2 | 8.9 | — | — | |
| DCipherBackbone=DeepSeek-V3.12026.05 | — | — | — | — | — | — | — | — | 26.6 | |
| DCipherBackbone=Kimi-K2.52026.05 | — | — | — | — | — | — | — | — | 42.2 | |
| DCipherBackbone=MiniMax-M2.52026.05 | — | — | — | — | — | — | — | — | 27.6 | |
| DCipherBackbone=Qwen3-235B2026.05 | — | — | — | — | — | — | — | — | 20.8 | |
| NYUCTFAgentBackbone=DeepSeek-V3.12026.05 | — | — | — | — | — | — | — | 18.2 | — | |
| NYUCTFAgentBackbone=Kimi-K2.52026.05 | — | — | — | — | — | — | — | 35.4 | — | |
| NYUCTFAgentBackbone=MiniMax-M2.52026.05 | — | — | — | — | — | — | — | 22.9 | — | |
| NYUCTFAgentBackbone=Qwen3-235B2026.05 | — | — | — | — | — | — | — | 17.7 | — |