HLS Code Generation on HLS-eval
91.7Syntax Correctness pass@1GPT-5.1
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-5.1Type=Foundational Models2026.05 | 91.7 | 97.7 | 71.7 | 79.1 | |
| HLS-SeekTraining Protocol=GRPO, Size=7B2026.05 | 84.7 | 95.3 | 68.9 | 81.4 | |
| HLS-SeekTraining Protocol=SFT, Size=7B2026.05 | 77.5 | 88.4 | 50.7 | 60.5 | |
| DeepSeek-V3.2Type=Foundational Models2026.05 | 77.3 | 93 | 51.9 | 65.1 | |
| Qwen2.5-CoderType=Code Models, Size=32B2026.05 | 77 | 88.4 | 46.8 | 58.1 | |
| Qwen2.5-CoderType=Code Models, Size=14B2026.05 | 73.6 | 81.4 | 36.7 | 44.2 | |
| DeepSeek-CoderType=Code Models, Size=6.7B2026.05 | 69 | 93 | 29.1 | 46.5 | |
| SAGE-HLSBackbone=Qwen2.5-Coder, Size=7B2026.05 | 67.8 | 76.7 | 34.8 | 41.9 | |
| Gemini-3-ProType=Foundational Models2026.05 | 67.1 | 95.3 | 55.9 | 79.1 | |
| Qwen3-235BType=Foundational Models, Size=235B2026.05 | 61.8 | 79.1 | 43.9 | 67.4 | |
| Qwen2.5-CoderType=Code Models, Size=7B2026.05 | 45.4 | 83.7 | 16 | 41.9 | |
| CodeLlamaType=Code Models, Size=7B2026.05 | 19.3 | 62.8 | 10.6 | 39.5 | |
| Starcoder2Type=Code Models, Size=15B2026.05 | 8.6 | 41.9 | 4.8 | 34.9 | |
| Gai et al.Backbone=CodeLlama, Size=7B2026.05 | 6.3 | 48.8 | 1.4 | 27.9 |