List your company
ResearchFormerly WizWand

Text-to-SQL on Spider Lite 2.0

69.3Execution Accuracy (EX)SOMA-SQL
-1.253617.063235.3853.6968Feb 17, 2026Mar 7, 2026Mar 26, 2026Apr 14, 2026May 2, 2026May 21, 2026Jun 9, 2026
Updated 4mo ago

Evaluation results

MethodLinks
SOMA-SQLBackbone LLM=GPT-5.3 Codex2026.0669.3
SOMA-SQLBackbone LLM=GPT-52026.0665.8
ProSpyModel=Claude-Opus-4.5, Single Trial=true2026.0660.15
SOMA-SQLBackbone LLM=Gemma-4-31B2026.0653.7
SquRL-7BBackbone=Deepseek-R12026.0249.18
ProSpyModel=Claude + GLM, Single Trial=true2026.0648.63
AutoLinkBackbone LLM=GPT-5.3 Codex2026.0646.3
AutoLinkBackbone LLM=GPT-52026.0645.2
ProSpyModel=Claude + DeepSeek, Single Trial=true2026.0643.14
ProSpyModel=DeepSeek V3.2, Single Trial=true2026.0641.32
ProSpyModel=GLM-5, Single Trial=true2026.0638.21
AutoLinkBackbone LLM=Gemma-4-31B2026.0637.7
ReForceModel=GPT-o3, Single Trial=false2026.0636.56
AutoLinkModel=DeepSeek-R1, Single Trial=true2026.0634.92
DeepEye-SQLBackbone LLM=GPT-5.3 Codex2026.0633.6
LinkAlignBackbone=Deepseek-R12026.0233.09
RSL-SQLBackbone=Deepseek-R12026.0233.09
LinkAlignModel=DeepSeek-R1, Single Trial=false2026.0633.09
RSL-SQLModel=GPT-o3, Single Trial=false2026.0633.09
ReForceModel=o4-mini, Single Trial=false2026.0631.99
DeepEye-SQLBackbone LLM=GPT-52026.0631.4
RSL-SQLModel=DeepSeek-R1, Single Trial=false2026.0630.53
Spider-AgentModel=Claude-4-Sonnet, Single Trial=true2026.0627.79
DeepEye-SQLBackbone LLM=Gemma-4-31B2026.0624
Agentic SQL + CSMR + ATRRL Data=BIRD + Spider, Base Model=OmniSQL-7B, Config=Cold-Start Model, Decoding Strategy=8-time sampling (majority voting)2026.0317.7
GPT-4oDecoding Strategy=8-time sampling (majority voting)2026.0315.6
Deepseek-V3Decoding Strategy=8-time sampling (majority voting)2026.0315.6
Arctic-Text2SQL-R1-7BRL Data=BIRD + SPIDER + Gretel*, Base Model=OmniSQL-7B, Decoding Strategy=8-time sampling (majority voting)2026.0315.6
SQL-R1 + OmniSQL-7BRL Data=SynSQL-Complex-5K, Base Model=OmniSQL-7B, Decoding Strategy=8-time sampling (majority voting)2026.0314.8
Single-Turn GRPORL Data=BIRD + Spider, Base Model=OmniSQL-7B, Config=Cold-Start Model, Decoding Strategy=8-time sampling (majority voting)2026.0314.8
Spider-AgentModel=DeepSeek-R1, Single Trial=true2026.0613.71
OmniSQL-7BBase Model=Qwen2.5-Coder-7B, Decoding Strategy=8-time sampling (majority voting)2026.0312.6
Agentic SQL + CSMR + ATRRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Reward=CSMR, Decoding Strategy=8-time sampling (majority voting)2026.038.8
ATRRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Config=Ablation Study, Decoding Strategy=8-time sampling (majority voting)2026.038.8
Agentic SQL + CSMRRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Reward=CSMR, Decoding Strategy=8-time sampling (majority voting)2026.038.1
ATR w/ Step-wise UpdateRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Config=Ablation Study, Decoding Strategy=8-time sampling (majority voting)2026.038.1
Agentic SQL + ATRRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Reward=Binary, Decoding Strategy=8-time sampling (majority voting)2026.037.4
ATR w/ Symmetric MatrixRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Config=Ablation Study, Decoding Strategy=8-time sampling (majority voting)2026.037.4
Single-Turn GRPORL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Reward=Binary, Decoding Strategy=8-time sampling (majority voting)2026.036.6
Single-Turn GRPO + CSMRRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Reward=CSMR, Decoding Strategy=8-time sampling (majority voting)2026.036.6
Agentic SQLRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Reward=Binary, Decoding Strategy=8-time sampling (majority voting)2026.035.9
DailSQLBackbone=GPT-4o2026.025.68
Qwen2.5-7B-InstructDecoding Strategy=8-time sampling (majority voting)2026.035.1
CHESSBackbone=GPT-4o2026.023.83
Qwen2.5-Coder-7BDecoding Strategy=8-time sampling (majority voting)2026.032.2
DIN-SQLBackbone=GPT-4o2026.021.46