Embedded Systems Code
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Embedded Systems Code Zephyr RTOS (test)
10Generative Token Accuracy
4
Embedded Systems Code Crypto (test)
13.3Generative Token Accuracy
4
Embedded Systems Code Wireless/BLE/WiFi (test)
15.3Generative Token Accuracy
4
Embedded Systems Code Linux Kernel (test)
12Generative Token Accuracy
4
Embedded Systems Code USB Stack (test)
19.2Generative Token Accuracy
4
Embedded Systems Code AMD GPU Registers (test)
31.3Generative Token Accuracy
4
Embedded Systems Code NXP i.MX (test)
33Generative Token Accuracy
4
Embedded Systems Code Device Tree (test)
35.2Token Accuracy
4
Embedded Systems Code (STM32 HAL) (test)
39.9Generative Token Accuracy
4
Embedded Systems Code Infineon AURIX (test)
54.3Generative Token Accuracy
4
Embedded Systems Code ARM Cortex ASM (test)
55.8Generative Token Accuracy
4
Embedded Systems Code Register Defines (test)
59.6Generative Token Accuracy
4
Embedded Systems Code General (test)
85.1Generative Token Accuracy
4