Code Generation on LiveCodeBench (Accuracy and Absolute Accuracy Change)
83.33AccuracySKILLGEN
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SKILLGENModel Name=Gemma-4-26B, Skill Augmentation Protocol=SKILL2026.05 | 83.33 | 0 | |
| SKILLGENModel Name=GPT-5.4-Mini, Skill Augmentation Protocol=SKILL2026.05 | 65.33 | 6 | |
| ReQATTQSQuantization Format=MXFP4, Precision=W4A162026.06 | 54.52 | — | |
| ReQATTQQuantization Format=MXFP4, Precision=W4A162026.06 | 53.82 | — | |
| BF16 FTQuantization Format=MXFP4, Precision=W4A162026.06 | 53.68 | — | |
| BF16 FTQuantization Format=NVFP4, Precision=W4A4KV42026.06 | 53.68 | — | |
| ReQATTQSQuantization Format=NVFP4, Precision=W4A4KV42026.06 | 53.59 | — | |
| ReQATTQQuantization Format=NVFP4, Precision=W4A4KV42026.06 | 53.08 | — | |
| QATQuantization Format=MXFP4, Precision=W4A162026.06 | 52.01 | — | |
| BF16 BaselineQuantization Format=MXFP4, Precision=W4A162026.06 | 51.68 | — | |
| BF16 BaselineQuantization Format=NVFP4, Precision=W4A4KV42026.06 | 51.68 | — | |
| QATQuantization Format=NVFP4, Precision=W4A4KV42026.06 | 50.68 | — | |
| PTQQuantization Format=MXFP4, Precision=W4A162026.06 | 48.88 | — | |
| PTQQuantization Format=NVFP4, Precision=W4A4KV42026.06 | 47.99 | — | |
| Dynamic-MAS + PRMBackbone=Qwen3-14B2026.02 | 33 | — | |
| Dynamic-MAS + Self-RefineBackbone=Qwen3-14B2026.02 | 32.5 | — | |
| Dynamic-MAS + ADv2Backbone=Qwen3-14B2026.02 | 32 | — | |
| Single AgentBackbone=Qwen3-14B2026.02 | 29.5 | — | |
| Dynamic-MAS + Multi-TAGBackbone=Qwen3-14B2026.02 | 29.5 | — | |
| Dynamic-MASBackbone=Qwen3-14B2026.02 | 29 | — | |
| OPDTeacher-Student Setting=DeepSeek-R1 -> DeepSeek-R1-Distill-Qwen-7B2026.06 | 25.1 | — | |
| OPDTeacher-Student Setting=Qwen3-8B -> Llama3.1-8B-SFT2026.06 | 24.8 | — | |
| CDMTeacher-Student Setting=DeepSeek-R1 -> DeepSeek-R1-Distill-Qwen-7B2026.06 | 24.7 | — | |
| SFTTeacher-Student Setting=DeepSeek-R1 -> DeepSeek-R1-Distill-Qwen-7B2026.06 | 24.3 | — | |
| ALMTeacher-Student Setting=DeepSeek-R1 -> DeepSeek-R1-Distill-Qwen-7B2026.06 | 24 | — | |
| CDMTeacher-Student Setting=Qwen3-8B -> Llama3.1-8B-SFT2026.06 | 22.9 | — | |
| ALMTeacher-Student Setting=Qwen3-8B -> Llama3.1-8B-SFT2026.06 | 22.4 | — | |
| SFTTeacher-Student Setting=Qwen3-8B -> Llama3.1-8B-SFT2026.06 | 21.8 | — |