Code Generation on Ag-LiveCodeBench-X
25Lua Pass@1Llama 3.3 70B Ins
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Llama 3.3 70B InsModel=Llama 3.3 70B Ins2025.08 | 25 | 22 | 13 | |
| Qwen3-8B-CF-XModel=Qwen3-8B-CF-X2025.08 | 25 | 25 | 19 | |
| Qwen3-4B-CF-XModel=Qwen3-4B-CF-X2025.08 | 23 | 22 | 15 | |
| Qwen 3 32BModel=Qwen 3 32B2025.08 | 22 | 26 | 17 | |
| Qwen3-4B-MBPP-XModel=Qwen3-4B-MBPP-X2025.08 | 15 | 15 | 9 | |
| DSC v2 Lite Ins 16BModel=DSC v2 Lite Ins 16B2025.08 | 13 | 12 | 9 | |
| Phi4-mini-ins-CF-XTraining=Agnostics (CF-X)2025.08 | 12 | 8 | 12 | |
| Qwen 3 4BModel=Qwen 3 4B2025.08 | 11 | 10 | 10 | |
| Qwen 3 8BModel=Qwen 3 8B2025.08 | 11 | 9 | 9 | |
| DSC-6.7B-Ins-CF-XModel Size=6.7B, Training=Agnostics (CF-X)2025.08 | 9 | 9 | 10 | |
| Phi 4 mini insTraining=Instruction-tuned baseline2025.08 | 8 | 8 | 5 | |
| DSC 6.7B InsModel Size=6.7B, Training=Instruction-tuned baseline2025.08 | 8 | 5 | 8 | |
| SmolLM3-3B-CF-XModel Size=3B, Training=Agnostics (CF-X)2025.08 | 8 | 8 | 6 | |
| SmolLM3 3BModel Size=3B, Training=Baseline2025.08 | 1 | 2 | 2 |