Agentic Task Execution on CAPTRACEBENCH Hard 1.0
44Success Rate (SR)Claude-Opus-4.6
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Claude-Opus-4.6Setting=No Skills2026.06 | 44 | 58.4 | |
| Average.Setting=w/ Oracle Skills2026.06 | 42.1 | 63.5 | |
| Claude-Opus-4.6Setting=REDACT Protected (w/ Extracted Skills)2026.06 | 41.1 | 57.5 | |
| GPT-5.2-CodexSetting=No Skills2026.06 | 38.8 | 60.7 | |
| Gemini-3-ProSetting=No Skills2026.06 | 37.6 | 50.8 | |
| Average.Setting=REDACT Protected (w/ Extracted Skills)2026.06 | 34.7 | 56.1 | |
| Average.Setting=No Skills2026.06 | 33.2 | 51.3 | |
| Average.Setting=REDACT Protected (w/ Evolved Skills)2026.06 | 32.3 | 49.9 | |
| Average.Setting=REDACT Protected (w/ Retrieval Reuse)2026.06 | 31.8 | 49.4 | |
| Gemini-3-FlashSetting=No Skills2026.06 | 29.4 | 49.5 | |
| Claude-Sonnet-4.6Setting=No Skills2026.06 | 29.1 | 48.9 | |
| Claude-Haiku-4.5Setting=No Skills2026.06 | 20.5 | 39.3 |