Cybersecurity Capture The Flag on CTFTiny
76Total Success RateClaude 4 Sonnet
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Claude 4 SonnetVariant=Sonnet2025.08 | 76 | 1.16 | 75 | 50 | 63.6 | 81.3 | 100 | 83.3 | |
| Gemini 2.5 FlashVariant=Flash2025.08 | 64 | 0.26 | 50 | 100 | 72.7 | 68.8 | 33.3 | 66.7 | |
| Gemini 2.5 ProVariant=Pro2025.08 | 48 | 0.33 | 50 | 100 | 45.5 | 37.5 | 33.3 | 66.7 | |
| GPT 4.12025.08 | 40 | 0.77 | 33.3 | 0 | 36.4 | 37.5 | 66.7 | 66.7 | |
| Qwen 32025.08 | 28 | 0.04 | 25 | 0 | 18.2 | 18.8 | 100 | 50 | |
| DeepSeek V32025.08 | 22 | 0.02 | 16.7 | 0 | 18.2 | 18.8 | 66.7 | 33.3 | |
| LLaMa 4 MaverickParameters=17B2025.08 | 8 | 0.14 | 0 | 0 | 0 | 12.5 | 33.3 | 16.7 |