Code Generation on DataGovbench Operator-Level
38.74ATSQwen3-coder
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Qwen3-coderAccess=Open-Source2025.12 | 38.74 | 48 | 67 | 51.25 | 732.5 | 185.07 | 122.6 | |
| DeepSeek-V3Access=Open-Source2025.12 | 35.68 | 47 | 74 | 52.23 | 680.51 | 1,663.45 | 572.13 | |
| Qwen3-235b-a22bAccess=Open-Source2025.12 | 34.73 | 46 | 69 | 49.91 | 950.68 | 1,335.47 | 519.87 | |
| Gemma-3-27BAccess=Open-Source2025.12 | 29.62 | 43 | 76 | 49.54 | 1,425.84 | 4,042.13 | 60.92 | |
| Qwen2.5-coderAccess=Open-Source2025.12 | 27.99 | 38 | 58 | 41.33 | 589.57 | 1,039.39 | 81.26 | |
| Llama-3-70BAccess=Open-Source2025.12 | 26.87 | 35 | 49 | 36.96 | 536.03 | 140.12 | 72.48 | |
| Phi4Access=Open-Source2025.12 | 23.24 | 32 | 42 | 32.41 | 982.37 | 1,642.61 | 98.73 | |
| Llama-4-scoutAccess=Open-Source2025.12 | 14.88 | 23 | 37 | 24.96 | 702.5 | 618.06 | 151.65 | |
| Mistral-7BAccess=Open-Source2025.12 | 10.41 | 15 | 27 | 17.47 | 715.78 | 525.99 | 87.74 |