Capture The Flag on NYU CTF Bench full 200-challenge
118Solves CountClaude 4.5 Opus
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Claude 4.5 OpusProvider=Anthropic2026.04 | 118 | 59 | 249.92 | 2.12 | |
| Gemini 3 ProProvider=Google2026.04 | 104 | 52 | 44.23 | 0.43 | |
| Gemini 3 FlashProvider=Google2026.04 | 54 | 27 | 2.69 | 0.05 | |
| GLM-5Provider=Z-AI2026.04 | 39 | 19.5 | 22.09 | 0.57 | |
| GPT-5.2-CodexProvider=OpenAI2026.04 | 36 | 18 | 9.23 | 0.26 | |
| GPT-5.2Provider=OpenAI2026.04 | 27 | 13.5 | 7.47 | 0.28 | |
| DeepSeek-V3Provider=DeepSeek2026.04 | 13 | 6.5 | 0.19 | 0.01 | |
| DeepSeek-R1Provider=DeepSeek2026.04 | 8 | 4 | 0.13 | 0.02 | |
| Qwen 3.5 397B-A17BProvider=Alibaba2026.04 | 7 | 3.5 | 1.39 | 0.2 | |
| Llama 3.3 70BProvider=Meta2026.04 | 5 | 2.5 | 0.03 | 0.01 |