CAPTCHA Solving on CaptchaBench
82.9Average Success RateCaptchaMind
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| CaptchaMindModel Category=Training-based Methods2026.05 | 82.9 | 71 | 91 | 93 | 72.5 | 86 | 71 | 87.5 | 91.5 | |
| SFT-onlyModel Category=Training-based Methods2026.05 | 68.1 | 49 | 79.5 | 77 | 76 | 88 | 14 | 70.5 | 91 | |
| HalliganModel Category=Agentic Methods2026.05 | 54.7 | 34 | 76.5 | 69.5 | 43 | 36 | 66 | 71 | 41.5 | |
| OedipusModel Category=Agentic Methods2026.05 | 51.9 | 35 | 68 | 76 | 44.5 | 31.5 | 53 | 68.5 | 38.5 | |
| Claude-4-SonnetModel Category=Closed-source VLMs2026.05 | 47.4 | 7.5 | 93.5 | 68.5 | 51 | 20 | 43 | 71.5 | 24.5 | |
| OpenAI-o3Model Category=Closed-source VLMs2026.05 | 46.9 | 37 | 69 | 51 | 72 | 4 | 46 | 62 | 34 | |
| Gemini-3-ProModel Category=Closed-source VLMs2026.05 | 45.8 | 47.5 | 62 | 88.5 | 63 | 50 | 52 | 0 | 3 | |
| Claude-3.7-SonnetModel Category=Closed-source VLMs2026.05 | 43.9 | 19 | 83.5 | 51 | 75.5 | 10 | 38.5 | 30 | 44 | |
| Qwen-VL-MaxModel Category=Closed-source VLMs2026.05 | 38.6 | 14.5 | 55.5 | 40 | 42.5 | 5.5 | 39.5 | 72.5 | 39 | |
| GPT-5Model Category=Closed-source VLMs2026.05 | 33.6 | 15 | 35.5 | 37 | 42 | 3.5 | 32.5 | 67 | 36 | |
| GUI_R1_7bModel Category=GUI Agent2026.05 | 6.5 | 0 | 13.5 | 14 | 0.5 | 3.5 | 18 | 2 | 0 |