Text-to-SQL on Spider (test)
91.2Execution AccuracyMiniSeek
Evaluation results
| Method | Links | |||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MiniSeek2025.05 | 91.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MARS-SQL2025.11 | 89.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MARS-SQLSize=7B each, # agents=3, Thinking?=Yes, Training set=Bird2025.11 | 89.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| XiYan-SQL2024.11 | 89.65 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| XiYan-SQL# agents=3, Thinking?=No2025.11 | 89.65 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MCS-SQL + GPT-4Training with Spider=false2024.10 | 89.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MCS-SQLbackbone=GPT-42024.11 | 89.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MCS-SQL + GPT-4# Samples=-, Method Type=Prompting with Proprietary LLMs2026.01 | 89.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MCS-SQLBase Model=GPT-42026.06 | 89.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MCS-SQLBase Model=GPT-4, Method Category=Prompt-based2026.04 | 89.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemini3-ProModel Accessibility=Closed-Source2025.04 | 89.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EXPO-SQLBase Model=Qwen2.5-Coder-14B2026.04 | 89.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Arctic-SQL-R1Base Model=Qwen2.5-Coder-14B, Method Category=RL-based2026.04 | 89.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EXPO-SQLBase Model=Qwen2.5-Coder-7B2026.04 | 89.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OmniSQL-7BBase Model=Qwen2.5-Coder-7B, Method Category=SFT-based2026.04 | 88.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Arctic-Text2SQL-R1Size=32B, # agents=1, Thinking?=Yes, Training set=Bird+Spider2025.11 | 88.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Arctic-SQL-R1Base Model=Qwen2.5-Coder-7B, Method Category=RL-based2026.04 | 88.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQL-R1Base Model=Qwen2.5-Coder-7B2026.06 | 88.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQL-R1Base Model=Qwen2.5-Coder-7B, Method Category=RL-based2026.04 | 88.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KE-ICLLLM=Gemini-Pro-1.52026.05 | 88.68 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeSLLM=Gemini-Pro-1.5, Setting=ICL2026.05 | 88.31 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQL-R1Base Model=Qwen2.5-Coder-14B, Method Category=RL-based2026.04 | 88.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KE-RLLLM=Qwen2.5-Coder-7B2026.05 | 87.97 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OmniSQL-7B# Samples=2.5M, Method Type=Fine-Tuning with Open-Source LLMs2026.01 | 87.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BASE-SQLBase Model=Qwen2.5-Coder-14B2026.06 | 87.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OmniSQLBase Model=Qwen2.5-Coder-7B2026.06 | 87.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CHASE-SQLBase=Gemini-1.5, Size=-, Multiple calls=true2025.05 | 87.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PET-SQLvoting_strategy=Difficulty-aware voting2024.03 | 87.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CHASE-SQL + Gemini 1.5Training with Spider=false2024.10 | 87.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CHASE-SQLbackbone=Gemini 1.52024.11 | 87.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PET-SQL2024.11 | 87.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CHASE-SQL + Gemini# agents=4, Thinking?=Yes2025.11 | 87.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OmniSQLSize=32B, # agents=1, Thinking?=No, Training set=OmniSQL2025.11 | 87.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CHASE-SQLBase Model=Gemini-1.5-Pro, Method Category=Prompt-based2026.04 | 87.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQLForge-LMBase=CodeLlama, Size=13B, Multiple calls=false2025.05 | 87.4 | — | 77.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CHESSTraining with Spider=false2024.10 | 87.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CHESS2025.05 | 87.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CHESS# agents=4, Thinking?=No2025.11 | 87.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLLLM=Gemini-Pro-1.5, Setting=ICL2026.05 | 87.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ACE-SQLBase Model=Qwen3-8B2026.06 | 87.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CHESS-SQLBase Model=Deepseek-33B, Method Category=Prompt-based2026.04 | 87.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OpenSearch-SQL+ GPT-4o# agents=4, Thinking?=No2025.11 | 87.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SuperSQL2024.11 | 87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| N-Rep2025.05 | 87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| JOLT-SQLBase Model=Qwen2.5-Coder-7B2026.06 | 86.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PET-SQLvoting_strategy=Naive voting2024.03 | 86.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLBase=GPT-4, Size=-, Multiple calls=true2025.05 | 86.6 | — | 75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SENSEBase=CodeLlama, Size=13B, Multiple calls=false2025.05 | 86.6 | — | 75.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLbackbone=GPT-4, strategy=Self-consistency2024.03 | 86.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQL + GPT-4Training with Spider=true2024.10 | 86.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLbackbone=GPT-42024.11 | 86.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQL + GPT-4# Samples=-, Method Type=Prompting with Proprietary LLMs2026.01 | 86.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQL2025.05 | 86.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLBase Model=GPT-42026.06 | 86.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLBase Model=GPT-4, Method Category=Prompt-based2026.04 | 86.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SquRL-7Bworkflow=dynamic, number of workflows=5, aggregation=majority voting, parameter scale=7B2026.02 | 86.58 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KE-ICLLLM=GPT-4o2026.05 | 86.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLCategory=Prompting-based Methods, Base Model=GPT-42025.05 | 86.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeSLLM=GPT-4o, Setting=ICL2026.05 | 86.16 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EVOLSQL-Qwen-7B# Samples=140K, Method Type=Fine-Tuning with Open-Source LLMs2026.01 | 86.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KE-ICLLLM=Qwen2.5-Coder-7B2026.05 | 85.98 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ShareSize=8B, # agents=1, Thinking?=No, Training set=Bird2025.11 | 85.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQLForge-LMBase=CodeLlama, Size=7B, Multiple calls=false2025.05 | 85.8 | — | 75.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeSLLM=Qwen2.5-Coder-7B, Setting=ICL2026.05 | 85.61 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DPG-SQLbackbone=GPT-42024.11 | 85.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Tool-SQLbackbone=GPT-42024.11 | 85.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Think2SQL-14BModel Scale Category=10-50B, Model Accessibility=Open-source2025.04 | 85.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Reward-SQLCategory=Our Approach, Base Model=Qwen3-8B, Decoding Strategy=PRM@32 (Best-of-N)2025.05 | 85.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQL-GENLLM=Qwen2.5-Coder-7B, Setting=RL2026.05 | 85.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIN-SQLBase=GPT-4, Size=-, Multiple calls=true2025.05 | 85.3 | — | 73 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIN-SQL + GPT-4Backbone=GPT-42023.04 | 85.3 | — | — | — | — | 60 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIN-SQLbackbone=GPT-42024.03 | 85.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIN-SQL + GPT-4Training with Spider=true2024.10 | 85.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIN-SQLbackbone=GPT-42024.11 | 85.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIN-SQL + GPT-4# Samples=-, Method Type=Prompting with Proprietary LLMs2026.01 | 85.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIN-SQLBase Model=GPT-42026.06 | 85.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIN-SQLCategory=Prompting-based Methods, Base Model=GPT-42025.05 | 85.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OmniLLM=Qwen2.5-Coder-7B, Setting=RL2026.05 | 85.14 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQL-o1Category=Supervised Fine-tuning Methods, Base Model=Qwen2.5-7B, Decoding Strategy=Vote@N2025.05 | 85.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ROUTEBase Model=Qwen2.5-Coder-7B, Method Category=SFT-based2026.04 | 85.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ExCoTBase Model=Qwen2.5-Coder-32B, Method Category=RL-based2026.04 | 85.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLLLM=GPT-4o, Setting=ICL2026.05 | 85.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EVOLSQL-Llama-8B# Samples=140K, Method Type=Fine-Tuning with Open-Source LLMs2026.01 | 84.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SquRL-1.5Bworkflow=dynamic, number of workflows=5, aggregation=majority voting, parameter scale=1.5B2026.02 | 84.83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQLFLOW# Samples=90K, Method Type=Fine-Tuning with Open-Source LLMs2026.01 | 84.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KE-RLLLM=Deepseek-Coder-7B2026.05 | 84.76 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLLLM=Qwen2.5-Coder-7B, Setting=ICL2026.05 | 84.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DTS-SQLBase Model=DeepSeek-Coder-7B2026.06 | 84.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DTS-SQLBase Model=DeepSeek-7B, Method Category=SFT-based2026.04 | 84.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SquRL-3Bworkflow=dynamic, number of workflows=5, aggregation=majority voting, parameter scale=3B2026.02 | 84.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Alpha-SQLCategory=Prompting-based Methods, Base Model=Qwen2.5-7B, Decoding Strategy=Vote@N2025.05 | 84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ROUTE + Llama3-8B# Samples=46K, Method Type=Fine-Tuning with Open-Source LLMs2026.01 | 83.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RSL-SQLworkflow=static, number of workflows=5, aggregation=majority voting2026.02 | 83.74 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ROUTE + Qwen2.5-7B# Samples=46K, Method Type=Fine-Tuning with Open-Source LLMs2026.01 | 83.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Think2SQL-4BModel Scale Category=< 5B, Model Accessibility=Open-source2025.04 | 83.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Think2SQL-8BModel Scale Category=5-10B, Model Accessibility=Open-source2025.04 | 83.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4o2024.11 | 83.54 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SENSE-7B# Samples=25K, Method Type=Fine-Tuning with Open-Source LLMs2026.01 | 83.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeSBase Model=StarCoderBase-7B2026.06 | 83.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SENSE-7BBase Model=CodeLLaMA-7B, Method Category=SFT-based2026.04 | 83.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |