List your company
ResearchFormerly WizWand

Text-to-SQL on BIRD (EX, VES)

65.12Execution Accuracy (EX)PV-SQL
0.515217.287634.0650.8324Apr 19, 2026Apr 28, 2026May 7, 2026May 17, 2026May 26, 2026Jun 4, 2026Jun 14, 2026
Updated 4mo ago

Evaluation results

MethodLinks
PV-SQLBase LLM=GPT-4o2026.0465.1275.55
PV-SQLBase LLM=GPT-4o2026.0465.1275.55
PV-SQLBase LLM=GPT-4.1-mini2026.0463.6286.9
TA-SQLBase LLM=GPT-4o2026.0460.4362.99
TA-SQLBase LLM=GPT-4o2026.0460.4362.99
TA-SQLBase LLM=GPT-4.1-mini2026.0460.266.11
PV-SQLBase LLM=GPT-OSS-20B2026.0459.4561.99
E-SQLBase LLM=GPT-4o2026.0459.164.65
E-SQLBase LLM=GPT-4o2026.0459.164.65
MAC-SQLBase LLM=GPT-4o2026.0458.762.77
MAC-SQLBase LLM=GPT-4o2026.0458.762.77
XiYan-SQLBase LLM=GPT-4o2026.0457.663.96
XiYan-SQLBase LLM=GPT-4o2026.0457.663.96
TS-SQLBase LLM=GPT-4o2026.0457.3763.43
TS-SQLBase LLM=GPT-4o2026.0457.3763.43
TA-SQLBase LLM=GPT-OSS-20B2026.0455.7455.51
MAC-SQLBase LLM=GPT-4.1-mini2026.0455.660.62
DIN-SQLBase LLM=GPT-4o2026.0454.856.75
DIN-SQLBase LLM=GPT-4o2026.0454.856.75
E-SQLBase LLM=GPT-4.1-mini2026.0454.860.95
DAIL-SQLBase LLM=GPT-4o2026.0454.158.77
DAIL-SQLBase LLM=GPT-4o2026.0454.158.77
E-SQLBase LLM=GPT-OSS-20B2026.0453.9553.05
DIN-SQLBase LLM=GPT-4.1-mini2026.0453.858.22
MAC-SQLBase LLM=GPT-OSS-20B2026.0453.5951.5
TS-SQLBase LLM=GPT-4.1-mini2026.0453.5258.81
CoTE-SQLCategory=Fine-tuned-based methods, Model=Qwen2.5-7B2026.0653.3959.02
GPT-4o2026.0452.2256.99
ZS-CoTBase LLM=GPT-4o2026.0452.2256.99
XiYan-SQLBase LLM=GPT-4.1-mini2026.0451.658.99
TS-SQLBase LLM=GPT-OSS-20B2026.0450.9855.2
ZS-CoTBase LLM=GPT-OSS-20B2026.0450.8552.84
DAIL-SQLBase LLM=GPT-4.1-mini2026.0450.455.05
ZS-CoTBase LLM=GPT-4.1-mini2026.0450.2653.75
DIN-SQLBase LLM=GPT-OSS-20B2026.0450.0346.56
PV-SQLBase LLM=Qwen3-4B2026.0449.6151.47
CoTE-SQLCategory=Fine-tuned-based methods, Model=Llama-3.1-8B2026.0649.5455.25
DAIL-SQLBase LLM=GPT-OSS-20B2026.0449.3948.22
DACCategory=Shallow prompt-based methods, Model=Qwen2.5-7B2026.0646.5448.25
GPT-4Category=Direct, Model=GPT-42026.0646.3549.77
DACCategory=Shallow prompt-based methods, Model=Llama-3.1-8B2026.0645.5748.32
Qwen2.5-7BCategory=Direct, Model=Qwen2.5-7B2026.0645.4448.37
MAC-SQLCategory=Fine-tuned-based methods, Model=SQL-Llama (7B)2026.0643.9457.36
DTS-SQLCategory=Fine-tuned-based methods, Model=Qwen2.5-7B2026.0642.1844.43
DTS-SQLCategory=Fine-tuned-based methods, Model=Llama-3.1-8B2026.0641.9243.59
E-SQLBase LLM=Qwen3-4B2026.0441.6646.38
Llama-3.1-8BCategory=Direct, Model=Llama-3.1-8B2026.0641.246.58
DAIL-SQLCategory=Shallow prompt-based methods, Model=Llama-3.1-8B2026.064143.82
DIN-SQLCategory=CoT-based methods, Model=Qwen2.5-7B2026.0639.9641.41
DAIL-SQLCategory=Shallow prompt-based methods, Model=Qwen2.5-7B2026.0639.941.59
MAC-SQLBase LLM=Qwen3-4B2026.0439.7640.88
PV-SQLBase LLM=Gemma3-4B2026.0439.542.39
DIN-SQLCategory=CoT-based methods, Model=Llama-3.1-8B2026.0638.7240.65
MAC-SQLBase LLM=Gemma3-4B2026.0437.737.36
ZS-CoTBase LLM=Qwen3-4B2026.0436.4439.01
XiYan-SQLBase LLM=Qwen3-4B2026.0436.438.31
PURPLECategory=Shallow prompt-based methods, Model=Qwen2.5-7B2026.0636.3142.33
XiYan-SQLBase LLM=GPT-OSS-20B2026.0435.9838.03
ZS-CoTBase LLM=Gemma3-4B2026.0434.5535.65
TA-SQLBase LLM=Qwen3-4B2026.0430.232.48
DAIL-SQLBase LLM=Qwen3-4B2026.0428.8129.99
PURPLECategory=Shallow prompt-based methods, Model=Llama-3.1-8B2026.0628.49—
TS-SQLBase LLM=Qwen3-4B2026.0427.431.47
DAIL-SQLBase LLM=Gemma3-4B2026.0427.3227.41
E-SQLBase LLM=Gemma3-4B2026.0426.229.14
TS-SQLBase LLM=Gemma3-4B2026.0425.0225.29
DIN-SQLBase LLM=Qwen3-4B2026.0423.4422.51
TA-SQLBase LLM=Gemma3-4B2026.0423.0122.71
DIN-SQLBase LLM=Gemma3-4B2026.0422.2320.57
XiYan-SQLBase LLM=Gemma3-4B2026.0420.7930.69
DeepSeek-R1-Llama-8BCategory=Direct, Model=DeepSeek-R1-Llama-8B2026.0615.1215.98
DEA-SQLCategory=CoT-based methods, Model=Qwen2.5-7B2026.0613.1714.93
ZS-CoTBase LLM=Qwen3-0.6B2026.0413.115.52
E-SQLBase LLM=Qwen3-0.6B2026.0412.8413.06
MAC-SQLBase LLM=Qwen3-0.6B2026.0411.612.95
PV-SQLBase LLM=Qwen3-0.6B2026.0411.0815.39
DEA-SQLCategory=CoT-based methods, Model=Llama-3.1-8B2026.068.418.35
DAIL-SQLBase LLM=Qwen3-0.6B2026.048.158.5
DIN-SQLBase LLM=Qwen3-0.6B2026.046.847.13
XiYan-SQLBase LLM=Qwen3-0.6B2026.046.66.93
DeepSeek-R1-Qwen-7BCategory=Direct, Model=DeepSeek-R1-Qwen-7B2026.066.177.05
TS-SQLBase LLM=Qwen3-0.6B2026.045.68.48
TA-SQLBase LLM=Qwen3-0.6B2026.0433.16