Mobile kernel generation on MobileKernelBench (test)
93.7CSRMoKA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MoKAMethod Strategy=Agentic Workflow2026.03 | 93.7 | 75.3 | 62.6 | 46.8 | 27.4 | |
| Claude-Sonnet-4.5 (@10)Method Strategy=Agentic Workflow, Protocol=@102026.03 | 62.1 | 47.9 | 41.6 | 20.5 | 5.3 | |
| Claude-Sonnet-4.5 (Base)Method Strategy=Agentic Workflow, Variant=Base2026.03 | 46.3 | 34.2 | 31.1 | 16.3 | 4.7 | |
| Qwen3-32B-LoRAMethod Strategy=Supervised Fine-Tuning (SFT), Technique=LoRA2026.03 | 25 | 18.5 | 11.1 | 5.6 | 0 | |
| Qwen3-4B-Instruct-2507-GRPOMethod Strategy=Reinforcement Learning (RL), Algorithm=GRPO2026.03 | 25 | 7.5 | 7.5 | 5 | 0 | |
| Qwen3-32BMethod Strategy=Supervised Fine-Tuning (SFT)2026.03 | 16.7 | 13.9 | 8.3 | 2.8 | 0 | |
| Qwen3-4B-Instruct-2507Method Strategy=Reinforcement Learning (RL)2026.03 | 10 | 5 | 5 | 5 | 0 |