Vectorized code generation on SimdBench SSE
77.35CorrelationAUTOVECCODER-8B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| AUTOVECCODER-8BVECRL=true, Label=Ours2026.05 | 77.35 | 53.53 | 1.02 | 2.22 | |
| DeepSeek-R1-2505282026.05 | 69.85 | 46.32 | 0.99 | 1.95 | |
| Claude-4-Sonnet-202505142026.05 | 66.91 | 40.44 | 0.93 | 1.51 | |
| AUTOVECCODER-8BVECRL=false, Protocol=VECPROMPT SFT only2026.05 | 62.94 | 43.53 | 0.95 | 1.7 | |
| Gemini-2.5-Pro2026.05 | 61.76 | 47.06 | 0.93 | 2.35 | |
| GPT-52026.05 | 55.88 | 33.82 | 0.6 | 1.15 | |
| Qwen3-Coder-480B-A35B2026.05 | 45.59 | 29.41 | — | 1.08 | |
| Qwen3-Coder-Plus2026.05 | 44.85 | 25 | — | 1 | |
| DeepSeek-V3.2-Thinking2026.05 | 42.65 | 27.94 | — | 1.01 | |
| DeepSeek-V3-2503242026.05 | 34.56 | 22.79 | — | 0.97 | |
| Grok4-Fast2026.05 | 19.85 | 11.03 | — | — | |
| Qwen3-8BTraining=w/o Training2026.05 | 10.88 | 5 | — | — |