Kernel Code Generation on KernelBench v0.2.0.dev0 (full 250-task suite)
98.4Correctness RateHTAM
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| HTAMTTS=true, Evo=true, Mem=true, HieDec=true2026.05 | 98.4 | 84 | 1.978 | 1.532 | 2.598 | 1.909 | |
| CudaForgeTTS=true, Evo=true, Mem=false, HieDec=false2026.05 | 97.6 | 70.8 | 1.677 | 1.448 | 2.104 | 1.283 | |
| Flat Memory RetrievalTTS=true, Evo=true, Mem=true, HieDec=false2026.05 | 89.6 | 58.8 | 1.464 | 1.194 | 1.917 | 1.284 | |
| Gemini-2.5-Flash-ThinkingTTS=false, Evo=false, Mem=false, HieDec=false2026.05 | 86 | 22.8 | 0.355 | 0.099 | 0.904 | 0.699 | |
| Feedback-only RefinementTTS=true, Evo=true, Mem=false, HieDec=false2026.05 | 84.8 | 49.2 | 1.065 | 0.955 | 1.18 | 1.08 | |
| DeepSeek-V3TTS=false, Evo=false, Mem=false, HieDec=false2026.05 | 82 | 7.2 | 0.194 | 0.06 | 0.314 | 0.78 | |
| Best-of-N SamplingTTS=true, Evo=false, Mem=false, HieDec=false2026.05 | 80.4 | 24.8 | 0.909 | 0.78 | 1.03 | 0.96 | |
| KernelBlasterTTS=true, Evo=true, Mem=true, HieDec=false2026.05 | 80.2 | 62.8 | 1.756 | 1.497 | 2.592 | 1.11 | |
| DeepSeek-R1TTS=false, Evo=false, Mem=false, HieDec=false2026.05 | 74.4 | 14.8 | 0.827 | 0.705 | 0.926 | 0.907 | |
| DeepSeek-V4-FlashTTS=false, Evo=false, Mem=false, HieDec=false2026.05 | 64 | 10 | 0.33 | 0.171 | 0.375 | 0.952 | |
| OpenAI-o3TTS=false, Evo=false, Mem=false, HieDec=false2026.05 | 57.6 | 31.6 | 0.68 | — | — | — |