Text-to-SQL on Spider (EX, VES)
88.4Execution Accuracy (EX)LearNAT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| LearNATLLMs=Qwen2.5-Coder-32B, Level=Ours2025.04 | 88.4 | — | |
| SuperSQLLLMs=GPT-4, Level=System-Level2025.04 | 87 | — | |
| LearNATLLMs=Qwen2.5-Coder-14B, Level=Ours2025.04 | 86.9 | — | |
| MAC-SQLLLMs=GPT-4, Level=System-Level2025.04 | 86.7 | — | |
| LearNATLLMs=Qwen2.5-Coder-7B, Level=Ours2025.04 | 86.4 | — | |
| CodeSLLMs=CodeS-7B, Level=Model-Level2025.04 | 85.4 | — | |
| MAG-SQLLLMs=GPT-4, Level=System-Level2025.04 | 85.3 | — | |
| CodeSLLMs=CodeS-15B, Level=Model-Level2025.04 | 84.9 | — | |
| C3-SQLLLMs=GPT-4, Level=System-Level2025.04 | 82 | — | |
| PURPLECategory=Shallow prompt-based methods, Model=Qwen2.5-7B2026.06 | 81.9 | 80.97 | |
| CoTE-SQLCategory=Fine-tuned-based methods, Model=Llama-3.1-8B2026.06 | 79.6 | 77.19 | |
| CoTE-SQLCategory=Fine-tuned-based methods, Model=Qwen2.5-7B2026.06 | 77.8 | 74.68 | |
| DACCategory=Shallow prompt-based methods, Model=Qwen2.5-7B2026.06 | 76.8 | 72.95 | |
| MAC-SQLCategory=Fine-tuned-based methods, Model=SQL-Llama (7B)2026.06 | 76.25 | — | |
| DAIL-SQLLLMs=GPT-4, Level=Model-Level2025.04 | 76.2 | — | |
| DACCategory=Shallow prompt-based methods, Model=Llama-3.1-8B2026.06 | 75.7 | 71.93 | |
| Qwen2.5-7BCategory=Direct, Model=Qwen2.5-7B2026.06 | 75.3 | 71.67 | |
| DAIL-SQLCategory=Shallow prompt-based methods, Model=Llama-3.1-8B2026.06 | 74.8 | 35.05 | |
| ACT-SQLLLMs=GPT-4, Level=Model-Level2025.04 | 74.5 | — | |
| DIN-SQLLLMs=GPT-4, Level=System-Level2025.04 | 74.2 | — | |
| DTS-SQLCategory=Fine-tuned-based methods, Model=Qwen2.5-7B2026.06 | 73.5 | 68.96 | |
| GRABBackbone=Qwen3-4B-Base2026.06 | 72.46 | — | |
| DAIL-SQLCategory=Shallow prompt-based methods, Model=Qwen2.5-7B2026.06 | 72.4 | 67.71 | |
| GPT-4Category=Direct, Model=GPT-42026.06 | 72.3 | — | |
| DTS-SQLCategory=Fine-tuned-based methods, Model=Llama-3.1-8B2026.06 | 71 | 66.93 | |
| PURPLECategory=Shallow prompt-based methods, Model=Llama-3.1-8B2026.06 | 70.6 | 73.03 | |
| MetaSQLLLMs=GPT-4, Level=System-Level2025.04 | 69.6 | — | |
| Soft PromptBackbone=Qwen3-4B-Base2026.06 | 68.84 | — | |
| MetaSQLCategory=Shallow prompt-based methods, Model=Qwen2.5-7B2026.06 | 68.6 | 64.98 | |
| MetaSQLCategory=Shallow prompt-based methods, Model=Llama-3.1-8B2026.06 | 65.4 | 62.18 | |
| Llama-3.1-8BCategory=Direct, Model=Llama-3.1-8B2026.06 | 56 | 53.43 | |
| DIN-SQLCategory=CoT-based methods, Model=Llama-3.1-8B2026.06 | 55.3 | 55.48 | |
| DEA-SQLCategory=CoT-based methods, Model=Qwen2.5-7B2026.06 | 52.1 | 49.52 | |
| DeepSeek-R1-Llama-8BCategory=Direct, Model=DeepSeek-R1-Llama-8B2026.06 | 41.5 | 18.93 | |
| DIN-SQLCategory=CoT-based methods, Model=Qwen2.5-7B2026.06 | 41 | 40.01 | |
| DeepSeek-R1-Qwen-7BCategory=Direct, Model=DeepSeek-R1-Qwen-7B2026.06 | 39.1 | 12.18 | |
| BaseBackbone=Qwen3-4B-Base2026.06 | 23.78 | — | |
| DEA-SQLCategory=CoT-based methods, Model=Llama-3.1-8B2026.06 | 19 | 16.76 |