List your company
ResearchFormerly WizWand

Text-to-SQL on Spider (EX, VES)

88.4Execution Accuracy (EX)LearNAT
16.22434.96253.772.438Apr 3, 2025Jun 17, 2025Aug 31, 2025Nov 14, 2025Jan 28, 2026Apr 13, 2026Jun 27, 2026
Updated 4mo ago

Evaluation results

MethodLinks
LearNATLLMs=Qwen2.5-Coder-32B, Level=Ours2025.0488.4—
SuperSQLLLMs=GPT-4, Level=System-Level2025.0487—
LearNATLLMs=Qwen2.5-Coder-14B, Level=Ours2025.0486.9—
MAC-SQLLLMs=GPT-4, Level=System-Level2025.0486.7—
LearNATLLMs=Qwen2.5-Coder-7B, Level=Ours2025.0486.4—
CodeSLLMs=CodeS-7B, Level=Model-Level2025.0485.4—
MAG-SQLLLMs=GPT-4, Level=System-Level2025.0485.3—
CodeSLLMs=CodeS-15B, Level=Model-Level2025.0484.9—
C3-SQLLLMs=GPT-4, Level=System-Level2025.0482—
PURPLECategory=Shallow prompt-based methods, Model=Qwen2.5-7B2026.0681.980.97
CoTE-SQLCategory=Fine-tuned-based methods, Model=Llama-3.1-8B2026.0679.677.19
CoTE-SQLCategory=Fine-tuned-based methods, Model=Qwen2.5-7B2026.0677.874.68
DACCategory=Shallow prompt-based methods, Model=Qwen2.5-7B2026.0676.872.95
MAC-SQLCategory=Fine-tuned-based methods, Model=SQL-Llama (7B)2026.0676.25—
DAIL-SQLLLMs=GPT-4, Level=Model-Level2025.0476.2—
DACCategory=Shallow prompt-based methods, Model=Llama-3.1-8B2026.0675.771.93
Qwen2.5-7BCategory=Direct, Model=Qwen2.5-7B2026.0675.371.67
DAIL-SQLCategory=Shallow prompt-based methods, Model=Llama-3.1-8B2026.0674.835.05
ACT-SQLLLMs=GPT-4, Level=Model-Level2025.0474.5—
DIN-SQLLLMs=GPT-4, Level=System-Level2025.0474.2—
DTS-SQLCategory=Fine-tuned-based methods, Model=Qwen2.5-7B2026.0673.568.96
GRABBackbone=Qwen3-4B-Base2026.0672.46—
DAIL-SQLCategory=Shallow prompt-based methods, Model=Qwen2.5-7B2026.0672.467.71
GPT-4Category=Direct, Model=GPT-42026.0672.3—
DTS-SQLCategory=Fine-tuned-based methods, Model=Llama-3.1-8B2026.067166.93
PURPLECategory=Shallow prompt-based methods, Model=Llama-3.1-8B2026.0670.673.03
MetaSQLLLMs=GPT-4, Level=System-Level2025.0469.6—
Soft PromptBackbone=Qwen3-4B-Base2026.0668.84—
MetaSQLCategory=Shallow prompt-based methods, Model=Qwen2.5-7B2026.0668.664.98
MetaSQLCategory=Shallow prompt-based methods, Model=Llama-3.1-8B2026.0665.462.18
Llama-3.1-8BCategory=Direct, Model=Llama-3.1-8B2026.065653.43
DIN-SQLCategory=CoT-based methods, Model=Llama-3.1-8B2026.0655.355.48
DEA-SQLCategory=CoT-based methods, Model=Qwen2.5-7B2026.0652.149.52
DeepSeek-R1-Llama-8BCategory=Direct, Model=DeepSeek-R1-Llama-8B2026.0641.518.93
DIN-SQLCategory=CoT-based methods, Model=Qwen2.5-7B2026.064140.01
DeepSeek-R1-Qwen-7BCategory=Direct, Model=DeepSeek-R1-Qwen-7B2026.0639.112.18
BaseBackbone=Qwen3-4B-Base2026.0623.78—
DEA-SQLCategory=CoT-based methods, Model=Llama-3.1-8B2026.061916.76