Coding Tasks on Claude Code Evaluation Set
26Number of SamplesClaude Code (Original)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Claude Code (Original)Comparison Baseline=Vanilla2026.05 | 26 | 0.002 | 3 | 21 | 2 | 0.031 | 0.9756 | 0.006 | |
| Claude Code (Compiled)Comparison Baseline=Vanilla2026.05 | 23 | 0.265 | 7 | 16 | 0 | 2.837 | 0.0096 | 0.592 | |
| Claude Code (Compiled)Comparison Baseline=Original2026.05 | 22 | 0.274 | 7 | 15 | 0 | 2.82 | 0.0103 | 0.601 |