CUDA Kernel Generation on KernelBench Level 3
24Executions CountMercury-Coder-Small
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Mercury-Coder-SmallType=Code, #Params=-2026.02 | 24 | 12 | 10 | |
| DICEType=Kernel (CUDA), #Params=8B2026.02 | 16 | 8 | 4 | |
| Qwen3Type=Reasoning, #Params=4B2026.02 | 14 | 2 | 0 | |
| DICEType=Kernel (CUDA), #Params=4B2026.02 | 12 | 6 | 4 | |
| Stable-DiffCoderType=Code, #Params=8B2026.02 | 10 | 6 | 2 | |
| DICEType=Kernel (CUDA), #Params=1.7B2026.02 | 10 | 6 | 4 | |
| Gemini-3-ProType=General, #Params=-2026.02 | 8 | 4 | 2 | |
| cudaLLMType=Reasoning (CUDA), #Params=8B2026.02 | 6 | 4 | 0 | |
| Dream-Coder-v0-InstructType=Code, #Params=7B2026.02 | 6 | 0 | 0 | |
| Trado-InstructType=Code & Math, #Params=8B2026.02 | 6 | 4 | 0 | |
| Claude-Sonnet-4Type=Code, #Params=-2026.02 | 4 | 0 | 0 | |
| Qwen2.5-Coder-InstructType=Code, #Params=7B2026.02 | 4 | 0 | 0 | |
| Qwen3Type=Reasoning (General), #Params=8B2026.02 | 4 | 2 | 2 | |
| SDAR-ChatType=General, #Params=4B2026.02 | 4 | 0 | 0 | |
| Dream-v0-InstructType=General, #Params=7B2026.02 | 2 | 2 | 2 | |
| Phi-4-Mini-ReasoningType=Reasoning, #Params=4B2026.02 | 2 | 0 | 0 | |
| Deepseek-Coder-Instruct-v1.5Type=Code, #Params=7B2026.02 | 0 | 0 | 0 | |
| Seed-Coder-ReasoningType=Code, #Params=8B2026.02 | 0 | 0 | 0 | |
| LLaDA-InstructType=General, #Params=8B2026.02 | 0 | 0 | 0 | |
| LLaDA-1.5Type=General, #Params=8B2026.02 | 0 | 0 | 0 | |
| SDAR-ChatType=General, #Params=8B2026.02 | 0 | 0 | 0 | |
| WeDLMType=General, #Params=8B2026.02 | 0 | 0 | 0 | |
| DiffuCoder-InstructType=Code, #Params=7B2026.02 | 0 | 0 | 0 | |
| Gemma-2-ItType=General, #Params=2B2026.02 | 0 | 0 | 0 | |
| SDAR-ChatType=General, #Params=1.7B2026.02 | 0 | 0 | 0 | |
| Qwen2.5-Coder-InstructType=Code, #Params=1.5B2026.02 | 0 | 0 | 0 | |
| Deepseek-Coder-InstructType=Code, #Params=1.3B2026.02 | 0 | 0 | 0 | |
| Qwen3Type=Reasoning, #Params=1.7B2026.02 | 0 | 0 | 0 | |
| Llama-3.2-InstructType=General, #Params=3B2026.02 | 0 | 0 | 0 | |
| Trado-InstructType=Code & Math, #Params=4B2026.02 | 0 | 0 | 0 | |
| Qwen2.5-Coder-InstructType=Code, #Params=3B2026.02 | 0 | 0 | 0 |