Code Generation on APPS stdin-style Plus
83.4Syntax ValidityDeepSeek
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DeepSeekParameters=6.7B2026.01 | 83.4 | 14.3 | 96.3 | |
| CodeLlamaParameters=7B2026.01 | 48.9 | 7.6 | 99.5 | |
| StarCoder2Parameters=7B2026.01 | 20.2 | 1.3 | 99.7 |
Loading the SOTA2 catalog…