List your company
ResearchFormerly WizWand

Accuracy Estimation on Text2SQL Transfer Suite (Spider, BIRD, WikiSQL, SParC, CoSQL, SynSQL-2.5M)

3.41MAEMetaEvaluator
2.8246.779510.73514.6905May 22, 2026
Updated 5mo ago

Evaluation results

MethodLinks
MetaEvaluatorUnseen model architecture=Meta-Llama-3-70B2026.053.41
MetaEvaluatorUnseen model architecture=XiYanSQL-14B2026.053.55
MetaEvaluatorUnseen model architecture=Avg.2026.053.66
MetaEvaluatorUnseen model architecture=Ministral-3-14B2026.053.69
MetaEvaluatorUnseen model architecture=Qwen2.5-32B2026.053.76
MetaEvaluatorUnseen model architecture=gemma-2-2b2026.053.88
NL2SQL-BUGSUnseen model architecture=Meta-Llama-3-70B2026.059.31
NL2SQL-BUGSUnseen model architecture=Ministral-3-14B2026.059.47
NL2SQL-BUGSUnseen model architecture=Avg.2026.059.61
NL2SQL-BUGSUnseen model architecture=Qwen2.5-32B2026.059.68
NL2SQL-BUGSUnseen model architecture=gemma-2-2b2026.059.76
NL2SQL-BUGSUnseen model architecture=XiYanSQL-14B2026.059.84
AutoEvalUnseen model architecture=Meta-Llama-3-70B2026.0511.62
AutoEvalUnseen model architecture=Ministral-3-14B2026.0511.89
AutoEvalUnseen model architecture=Avg.2026.0512.02
AutoEvalUnseen model architecture=Qwen2.5-32B2026.0512.05
AutoEvalUnseen model architecture=gemma-2-2b2026.0512.21
AutoEvalUnseen model architecture=XiYanSQL-14B2026.0512.33
PseudoAutoEvalUnseen model architecture=Meta-Llama-3-70B2026.0513.84
PseudoAutoEvalUnseen model architecture=Ministral-3-14B2026.0514.07
PseudoAutoEvalUnseen model architecture=Avg.2026.0514.24
PseudoAutoEvalUnseen model architecture=Qwen2.5-32B2026.0514.31
PseudoAutoEvalUnseen model architecture=gemma-2-2b2026.0514.42
PseudoAutoEvalUnseen model architecture=XiYanSQL-14B2026.0514.58
AGDUnseen model architecture=Meta-Llama-3-70B2026.0514.77
AGDUnseen model architecture=XiYanSQL-14B2026.0514.96
AGDUnseen model architecture=Ministral-3-14B2026.0515.04
AGDUnseen model architecture=Avg.2026.0515.04
AGDUnseen model architecture=Qwen2.5-32B2026.0515.18
AGDUnseen model architecture=gemma-2-2b2026.0515.25
DoCUnseen model architecture=Meta-Llama-3-70B2026.0515.42
DoCUnseen model architecture=Ministral-3-14B2026.0515.66
DoCUnseen model architecture=Avg.2026.0515.86
DoCUnseen model architecture=Qwen2.5-32B2026.0515.97
DoCUnseen model architecture=gemma-2-2b2026.0516.05
DoCUnseen model architecture=XiYanSQL-14B2026.0516.18
ATCUnseen model architecture=Meta-Llama-3-70B2026.0517.21
ATCUnseen model architecture=Ministral-3-14B2026.0517.52
ATCUnseen model architecture=Avg.2026.0517.72
ATCUnseen model architecture=Qwen2.5-32B2026.0517.88
ATCUnseen model architecture=gemma-2-2b2026.0517.95
ATCUnseen model architecture=XiYanSQL-14B2026.0518.06