Synthesis Correctness on IDS suite Monotonic Writes
3Pass RateClaude Code
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Claude Code2026.05 | 3 | 4 | 70 | |
| IDS2026.05 | 3 | 3 | 58 | |
| Codex2026.05 | 2 | 5 | 65 |
Loading the SOTA2 catalog…
| Method | Links | |||
|---|---|---|---|---|
| Claude Code2026.05 | 3 | 4 | 70 | |
| IDS2026.05 | 3 | 3 | 58 | |
| Codex2026.05 | 2 | 5 | 65 |