Compositional String Transformations on CodeIO (200 unseen compositions)
76AccuracyORACLE
Evaluation results
| Method | Links | |
|---|---|---|
| ORACLE2026.05 | 76 | |
| RTE2026.05 | 45.3 | |
| COT + MAJORITY VOTEN=16, Reasoning Strategy=Chain of Thought (CoT)2026.05 | 30.2 | |
| FEW-SHOT BASELINE2026.05 | 19.8 |