SQL Generation on BIRD Original (dev)
65.51Execution Accuracy (Simple)Qwen3-Max
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen3-MaxReasoning=false2026.01 | 65.51 | 52.59 | 52.41 | 60.37 | |
| Qwen2.5-Coder (32B)Reasoning=false2026.01 | 64.11 | 47.84 | 39.31 | 56.84 | |
| Qwen3-Coder (30B)Reasoning=false2026.01 | 64.11 | 51.29 | 43.54 | 58.28 | |
| Qwen2.5-Coder (14B)Reasoning=false2026.01 | 63.24 | 46.55 | 44.14 | 56.39 | |
| Qwen-MaxReasoning=false2026.01 | 63.14 | 45.47 | 34.48 | 55.08 | |
| Qwen3-Coder-PlusReasoning=false2026.01 | 62.92 | 53.45 | 50.34 | 58.87 | |
| DeepSeek-R1Reasoning=true2026.01 | 62.81 | 44.4 | 40 | 55.08 | |
| Qwen3 (7B)Reasoning=true2026.01 | 62.15 | 46 | 32.64 | 54.48 | |
| Qwen3 (32B)Reasoning=true2026.01 | 61.84 | 47.84 | 35.17 | 55.08 | |
| Qwen2.5-Coder (7B)Reasoning=false2026.01 | 60.65 | 41.16 | 31.03 | 51.96 | |
| Qwen3 (14B)Reasoning=true2026.01 | 59.46 | 44.18 | 39.31 | 52.93 | |
| TableGPT2 (7B)Reasoning=false2026.01 | 39.68 | 22.84 | 18.62 | 32.59 | |
| AutoCoder (7B)Reasoning=false2026.01 | 12.54 | 3.23 | 6.9 | 9.19 | |
| StarCoder2 (7B)Reasoning=false2026.01 | 0.65 | 0.86 | 0.69 | 0.72 |