Counting Task on Spatial and Temporal Simulation Tasks Episode-level
4.203Subtask AccuracyCortex (full harness)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Cortex (full harness)Level=Episode-level2026.07 | 4.203 | 4.262 | 8.464 | |
| Cortex (harness on skills)Level=Episode-level2026.07 | 3.95 | 4.142 | 8.091 | |
| Cortex (w/o harness)Level=Episode-level2026.07 | 3.741 | 3.995 | 7.736 | |
| Cortex (baseline)Level=Episode-level2026.07 | 3.525 | 4.29 | 7.815 | |
| GPT-5Level=Episode-level2026.07 | 3.32 | 4.056 | 7.376 | |
| GeminiLevel=Episode-level2026.07 | 3.06 | 3.945 | 7.006 | |
| Qwen3-VL-8B-InstructLevel=Episode-level2026.07 | 2.614 | 3.775 | 6.388 |