Python reference-model generation on CVDP cid03
43.3Pass@1ChipMate-Python-9B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ChipMate-Python-9BType=ChipMate-P, Size=9B2026.05 | 43.3 | 46.7 | |
| GPT-5.5Type=Foundation Models, Size=-2026.05 | 41.6 | 47.4 | |
| ChipMate-Python-4BType=ChipMate-P, Size=4B2026.05 | 34.2 | 45.3 | |
| DeepSeek R1Type=Foundation Models, Size=671B2026.05 | 26.2 | 37.2 | |
| DeepSeek V4Type=Foundation Models, Size=1.6T2026.05 | 24.7 | 36.5 | |
| CodeV-R1 (distill)Type=Specialized Models, Size=7B2026.05 | 24.7 | 37.2 | |
| CodeV-R1Type=Specialized Models, Size=7B2026.05 | 22.3 | 34.6 | |
| DeepSeek CoderType=Foundation Models, Size=236B2026.05 | 21.5 | 35.1 | |
| Qwen3.5-9BType=Base Models, Size=9B2026.05 | 11.4 | 19.9 | |
| Qwen3.5-4BType=Base Models, Size=4B2026.05 | 10.2 | 12.7 |