Vectorized code generation on SimdBench AVX
76.76Correlation (Corr)AUTOVECCODER-8B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| AUTOVECCODER-8BVECRL=true, Label=Ours2026.05 | 76.76 | 47.35 | 0.99 | — | |
| DeepSeek-R1-2505282026.05 | 73.53 | 44.12 | 0.97 | 2.83 | |
| Gemini-2.5-Pro2026.05 | 63.97 | 39.71 | 0.88 | 2.84 | |
| AUTOVECCODER-8BVECRL=false, Protocol=VECPROMPT SFT only2026.05 | 62.79 | 35.59 | 0.81 | 1.85 | |
| GPT-52026.05 | 62.5 | 36.76 | 0.82 | 1.18 | |
| Claude-4-Sonnet-202505142026.05 | 58.09 | 28.68 | 0.72 | 1.03 | |
| Qwen3-Coder-480B-A35B2026.05 | 55.15 | 33.09 | 0.67 | 1.25 | |
| DeepSeek-V3.2-Thinking2026.05 | 53.68 | 30.88 | 0.47 | 1.3 | |
| DeepSeek-V3-2503242026.05 | 43.38 | 24.26 | — | 1 | |
| Qwen3-Coder-Plus2026.05 | 34.56 | 19.85 | — | 0.89 | |
| Grok4-Fast2026.05 | 18.38 | 9.56 | — | — | |
| Qwen3-8BTraining=w/o Training2026.05 | 9.41 | 2.79 | — | — |