Synthesis Correctness on IDS suite Read-Your-Writes
3Pass RateCodex
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Codex2026.05 | 3 | 4 | 60 | |
| Claude Code2026.05 | 3 | 3 | 70 | |
| IDS2026.05 | 3 | 2 | 52 |
Loading the SOTA2 catalog…
| Method | Links | |||
|---|---|---|---|---|
| Codex2026.05 | 3 | 4 | 60 | |
| Claude Code2026.05 | 3 | 3 | 70 | |
| IDS2026.05 | 3 | 2 | 52 |