Text-to-SQL on Synthesized Business Logic-Driven Dataset
90.24Single Execution AccuracyGPT-5
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-5Model Type=Proprietary, Agent architecture=ReAct-style2026.01 | 90.24 | 70.97 | 80.95 | 76.19 | 80.7 | |
| Gemini-2.5-ProModel Type=Proprietary, Agent architecture=ReAct-style2026.01 | 75.61 | 64.52 | 71.43 | 42.86 | 65.79 | |
| Claude-4.5-SonnetModel Type=Proprietary, Agent architecture=ReAct-style2026.01 | 73.17 | 48.39 | 57.14 | 28.57 | 55.26 | |
| Qwen3-235BModel Type=Open-Source, Agent architecture=ReAct-style2026.01 | 70.73 | 29.03 | 47.62 | 28.57 | 47.37 | |
| Qwen3-30BModel Type=Open-Source, Agent architecture=ReAct-style2026.01 | 68.29 | 29.03 | 38.1 | 19.05 | 42.98 | |
| Gemini-3-ProModel Type=Proprietary, Agent architecture=ReAct-style2026.01 | 65.85 | 67.74 | 76.19 | 33.33 | 62.28 | |
| DeepSeek-R1Model Type=Open-Source, Agent architecture=ReAct-style2026.01 | 63.41 | 41.94 | 42.86 | 33.33 | 48.25 | |
| DeepSeek-v3.2Model Type=Open-Source, Agent architecture=ReAct-style2026.01 | 58.54 | 25.81 | 38.1 | 28.57 | 40.35 | |
| Qwen3-Coder-30bModel Type=Open-Source, Agent architecture=ReAct-style2026.01 | 51.22 | 25.81 | 42.86 | 23.81 | 37.72 |