SQL Generation on BIRD Verified
69.41Execution Accuracy (Simple)Qwen3-Max
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen3-MaxReasoning=false2026.01 | 69.41 | 55.82 | 46.21 | 63.1 | |
| Qwen3 (32B)Reasoning=true2026.01 | 69.08 | 54.09 | 40 | 61.8 | |
| Qwen3 (7B)Reasoning=true2026.01 | 68.98 | 52.92 | 38.19 | 61.22 | |
| Qwen-MaxReasoning=false2026.01 | 68.32 | 51.72 | 40 | 59.2 | |
| DeepSeek-R1Reasoning=true2026.01 | 67.54 | 53.48 | 57.93 | 62.32 | |
| Qwen3-Coder (30B)Reasoning=false2026.01 | 67.35 | 55.82 | 43.45 | 61.6 | |
| Qwen2.5-Coder (32B)Reasoning=false2026.01 | 66.7 | 50.65 | 41.38 | 59.45 | |
| Qwen3 (14B)Reasoning=true2026.01 | 65.84 | 48.28 | 42.76 | 58.34 | |
| Qwen3-Coder-PlusReasoning=false2026.01 | 65.3 | 56.25 | 50.34 | 61.15 | |
| Qwen2.5-Coder (14B)Reasoning=false2026.01 | 63.78 | 51.29 | 50.34 | 58.74 | |
| Qwen2.5-Coder (7B)Reasoning=false2026.01 | 61.41 | 42.67 | 31.72 | 52.93 | |
| TableGPT2 (7B)Reasoning=false2026.01 | 40.97 | 25 | 18.62 | 34.03 | |
| AutoCoder (7B)Reasoning=false2026.01 | 13.84 | 3.88 | 7.59 | 10.23 | |
| StarCoder2 (7B)Reasoning=false2026.01 | 0.97 | 1.08 | 0 | 0.91 |