Reduction on CUDA-LLM task suite
5.75TimeOptiML
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| OptiMLOptimization=OptiML (G+X)2026.02 | 5.75 | 1.67 | 25.14 | 63.27 | 25.73 | 722 | 1,115 | 942 | 72.18 | 36 | |
| HPC-Coder-V2 + OptiML-XOptimization=OptiML-X2026.02 | 6.01 | 1.6 | 24.18 | 65.12 | 24.81 | 742 | 1,179 | 968 | 71.08 | 36 | |
| GPT 5.1 + OptiML-XOptimization=OptiML-X2026.02 | 6.23 | 1.54 | 23.62 | 66.41 | 24.27 | 759 | 1,207 | 982 | 70.38 | 36 | |
| StarCoder2 + OptiML-XOptimization=OptiML-X2026.02 | 6.49 | 1.48 | 22.94 | 66.96 | 23.72 | 775 | 1,231 | 999 | 69.44 | 36 | |
| Qwen2.5-Coder + OptiML-XOptimization=OptiML-X2026.02 | 6.71 | 1.43 | 22.53 | 67.88 | 23.31 | 793 | 1,254 | 1,014 | 68.66 | 36 | |
| GPT 5.1Optimization=None (LLM-only baseline)2026.02 | 9.6 | — | 18.41 | 74.32 | 20.58 | 917 | 1,457 | 1,046 | 62.21 | 34 | |
| Qwen2.5-CoderOptimization=None (LLM-only baseline)2026.02 | 9.9 | 0.97 | 17.92 | 75.04 | 20.12 | 952 | 1,522 | 1,088 | 61.63 | 36 | |
| HPC-Coder-V2Optimization=None (LLM-only baseline)2026.02 | 10.18 | 0.94 | 17.51 | 75.46 | 19.84 | 973 | 1,564 | 1,122 | 61.14 | 36 | |
| StarCoder2Optimization=None (LLM-only baseline)2026.02 | 10.46 | 0.92 | 17.14 | 75.83 | 19.59 | 992 | 1,602 | 1,154 | 60.71 | 38 |