CAPTCHA Solving on Dynamic CAPTCHA system (held-out test)
62.11SR TextReCAP-32B
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ReCAP-32B2026.03 | 62.11 | 55.83 | 99.42 | 82.88 | 85.4 | 100 | 78.1 | 81 | 1.31 | 1.34 | 1.03 | 1.12 | 2.87 | 1 | 2.21 | 1.54 | |
| ReCAP-8B2026.03 | 60.25 | 47.5 | 95.32 | 80.14 | 72.99 | 88.28 | 52.55 | 71.9 | 1.24 | 1.25 | 1.14 | 1.38 | 2.59 | 1.42 | 2.11 | 1.54 | |
| Halligan Frameworktype=Specialized Pipeline2026.03 | 42 | 44 | 2 | 14 | 8 | 2 | 64 | 25.14 | 3 | 3 | 5 | 3 | 5 | 3 | 3 | — | |
| Qwen3-VL-Thinking-32Btype=Open-source GUI Agent2026.03 | 37.89 | 29.17 | 25.15 | 38.36 | 63.5 | 8.59 | 2.92 | 29.7 | 2.51 | 2.57 | 2.74 | 1.58 | 2.99 | 3 | 3.25 | 2.55 | |
| OpenAI CUAtype=Closed-source GUI Agent2026.03 | 36.02 | 33.33 | 0 | 81.5 | 65.69 | 0 | 8.03 | 31.8 | 3 | 2.95 | — | 1.57 | 2.61 | — | 3.82 | 2.33 | |
| Qwen3-VL-Thinking-8Btype=Open-source GUI Agent2026.03 | 33.54 | 25.83 | 27.49 | 41.1 | 59.85 | 5.47 | 4.38 | 28.7 | 2.44 | 2.55 | 2.66 | 1.83 | 2.84 | 2.71 | 4 | 2.52 | |
| UI-TARS-1.5-7Btype=Open-source GUI Agent2026.03 | 28.57 | 17.36 | 51.3 | 66.01 | 44.68 | 18.88 | 0.75 | 33.6 | 2.31 | 2.39 | 2.24 | 1.78 | 2.98 | 2.56 | 3.44 | 2.47 |