CUDA Kernel Generation on KernelBench Level 2
48Execution CountClaude-Sonnet-4
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Claude-Sonnet-4Type=Code, #Params=-2026.02 | 48 | 24 | 18 | |
| Mercury-Coder-SmallType=Code, #Params=-2026.02 | 44 | 22 | 9 | |
| DICEType=Kernel (CUDA), #Params=8B2026.02 | 39 | 14 | 0 | |
| cudaLLMType=Reasoning (CUDA), #Params=8B2026.02 | 34 | 12 | 6 | |
| Qwen3Type=Reasoning, #Params=4B2026.02 | 29 | 14 | 4 | |
| Qwen3Type=Reasoning (General), #Params=8B2026.02 | 25 | 11 | 6 | |
| DICEType=Kernel (CUDA), #Params=4B2026.02 | 24 | 15 | 9 | |
| Gemini-3-ProType=General, #Params=-2026.02 | 14 | 7 | 4 | |
| Dream-Coder-v0-InstructType=Code, #Params=7B2026.02 | 8 | 8 | 5 | |
| Deepseek-Coder-Instruct-v1.5Type=Code, #Params=7B2026.02 | 7 | 2 | 1 | |
| DICEType=Kernel (CUDA), #Params=1.7B2026.02 | 6 | 3 | 2 | |
| Stable-DiffCoderType=Code, #Params=8B2026.02 | 5 | 4 | 1 | |
| Seed-Coder-ReasoningType=Code, #Params=8B2026.02 | 2 | 0 | 0 | |
| SDAR-ChatType=General, #Params=8B2026.02 | 2 | 0 | 0 | |
| Trado-InstructType=Code & Math, #Params=8B2026.02 | 2 | 0 | 0 | |
| WeDLMType=General, #Params=8B2026.02 | 1 | 0 | 0 | |
| Qwen3Type=Reasoning, #Params=1.7B2026.02 | 1 | 0 | 0 | |
| SDAR-ChatType=General, #Params=4B2026.02 | 1 | 0 | 0 | |
| Qwen2.5-Coder-InstructType=Code, #Params=3B2026.02 | 1 | 1 | 1 | |
| Qwen2.5-Coder-InstructType=Code, #Params=7B2026.02 | 0 | 0 | 0 | |
| LLaDA-InstructType=General, #Params=8B2026.02 | 0 | 0 | 0 | |
| LLaDA-1.5Type=General, #Params=8B2026.02 | 0 | 0 | 0 | |
| Dream-v0-InstructType=General, #Params=7B2026.02 | 0 | 0 | 0 | |
| DiffuCoder-InstructType=Code, #Params=7B2026.02 | 0 | 0 | 0 | |
| Gemma-2-ItType=General, #Params=2B2026.02 | 0 | 0 | 0 | |
| SDAR-ChatType=General, #Params=1.7B2026.02 | 0 | 0 | 0 | |
| Qwen2.5-Coder-InstructType=Code, #Params=1.5B2026.02 | 0 | 0 | 0 | |
| Deepseek-Coder-InstructType=Code, #Params=1.3B2026.02 | 0 | 0 | 0 | |
| Llama-3.2-InstructType=General, #Params=3B2026.02 | 0 | 0 | 0 | |
| Trado-InstructType=Code & Math, #Params=4B2026.02 | 0 | 0 | 0 | |
| Phi-4-Mini-ReasoningType=Reasoning, #Params=4B2026.02 | 0 | 0 | 0 |