GPU Kernel Generation and Optimization on KernelBench Level 1
99CorrectnessKernel Foundry
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Kernel FoundryOptimizer=Claude-sonnet-4.5, Expert=AutoTriton2026.05 | 99 | 21 | 1.22 | 0.39 | |
| Kernel FoundryOptimizer=ChatGPT-5.4, Expert=AutoTriton2026.05 | 96 | 31 | 1.14 | 0.7 | |
| Kernel FoundryOptimizer=Qwen-3.5, Expert=AutoTriton2026.05 | 91 | 18 | 0.87 | 0.31 | |
| Kernel FoundryOptimizer=Deepseek-V3, Expert=AutoTriton2026.05 | 86 | 17 | 1.3 | 0.37 | |
| AutoTriton2026.05 | 84 | 21 | 1.09 | 0.66 | |
| AutoTritonevaluation protocol=w/o cheating2026.05 | 76 | 11 | 0.98 | 0.56 | |
| Kernel FoundryOptimizer=Deepseek-V3, Expert=KernelLLM2026.05 | 60 | 16 | 0.91 | 0.72 | |
| KernelLLM2026.05 | 55 | 21 | 0.49 | 0.8 | |
| Kernel FoundryOptimizer=Deepseek-V3, Expert=No Expert2026.05 | 55 | 10 | 0.81 | 0.54 | |
| DeepSeek-V32026.05 | 52 | 6 | 0.71 | 0.56 | |
| KernelLLMevaluation protocol=w/o cheating2026.05 | 19 | 3 | 0.13 | 0.51 |