HLS Code Generation on ForgeHLS (test)
84.3Syntax Correctness (pass@1)GPT-5.1
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GPT-5.1Type=Foundational Models2026.05 | 84.3 | 89.8 | 91.7 | 73.1 | 80.6 | 82.4 | |
| HLS-SeekTraining Protocol=GRPO, Size=7B2026.05 | 81.5 | 91.7 | 93.5 | 71.3 | 84.3 | 87 | |
| HLS-SeekTraining Protocol=SFT, Size=7B2026.05 | 78.7 | 89.8 | 90.7 | 66.7 | 79.6 | 82.4 | |
| Qwen2.5-CoderType=Code Models, Size=14B2026.05 | 77.8 | 86.1 | 87 | 63.9 | 76.9 | 77.8 | |
| SAGE-HLSBackbone=Qwen2.5-Coder, Size=7B2026.05 | 77.8 | 88 | 88 | 63.9 | 76.9 | 77.8 | |
| Qwen2.5-CoderType=Code Models, Size=32B2026.05 | 75 | 86.1 | 87 | 61.1 | 75.9 | 75.9 | |
| DeepSeek-V3.2Type=Foundational Models2026.05 | 73.1 | 88 | 89.8 | 62 | 77.8 | 80.6 | |
| DeepSeek-CoderType=Code Models, Size=6.7B2026.05 | 63.9 | 86.1 | 88 | 46.3 | 74.1 | 78.7 | |
| Gemini-3-ProType=Foundational Models2026.05 | 62 | 88 | 90.7 | 55.6 | 78.7 | 80.6 | |
| Qwen2.5-CoderType=Code Models, Size=7B2026.05 | 41.7 | 76.9 | 82.4 | 24.1 | 63 | 69.4 | |
| Qwen3-235BType=Foundational Models, Size=235B2026.05 | 33.3 | 42.6 | 43.5 | 24.1 | 37 | 39.8 | |
| Gai et al.Backbone=CodeLlama, Size=7B2026.05 | 22.2 | 28.7 | 45.4 | 15.7 | 17.6 | 27.8 | |
| Starcoder2Type=Code Models, Size=15B2026.05 | 19.4 | 31.5 | 49.1 | 8.1 | 13.9 | 24.1 | |
| CodeLlamaType=Code Models, Size=7B2026.05 | 17.6 | 57.4 | 75 | 10.2 | 38 | 52.8 |