List your company
ResearchFormerly WizWand

Text-to-SQL on BIRD (Accuracy)

69.1AccuracyAgentic SQL + CSMR + ATR
20.63633.21845.858.382Oct 16, 2024Jan 10, 2025Apr 6, 2025Jul 1, 2025Sep 25, 2025Dec 20, 2025Mar 17, 2026
Updated 7mo ago

Evaluation results

MethodLinks
Agentic SQL + CSMR + ATRRL Data=BIRD + Spider, Base Model=OmniSQL-7B, Config=Cold-Start Model, Decoding Strategy=Greedy2026.0369.1
Arctic-Text2SQL-R1-7BRL Data=BIRD + SPIDER + Gretel*, Base Model=OmniSQL-7B, Decoding Strategy=Greedy2026.0367.6
Single-Turn GRPORL Data=BIRD + Spider, Base Model=OmniSQL-7B, Config=Cold-Start Model, Decoding Strategy=Greedy2026.0367.4
SQL-R1 + OmniSQL-7BRL Data=SynSQL-Complex-5K, Base Model=OmniSQL-7B, Decoding Strategy=Greedy2026.0366.6
GPT-4oDecoding Strategy=Greedy2026.0364.4
Agentic SQL + CSMR + ATRRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Reward=CSMR, Decoding Strategy=Greedy2026.0364.2
ATRRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Config=Ablation Study, Decoding Strategy=Greedy2026.0364.2
OmniSQL-7BBase Model=Qwen2.5-Coder-7B, Decoding Strategy=Greedy2026.0364.1
Reasoning-SQL-7BRL Data=BIRD, Base Model=Qwen2.5-Coder-7B, Decoding Strategy=Greedy2026.0364
Agentic SQL + ATRRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Reward=Binary, Decoding Strategy=Greedy2026.0363.6
SQL-R1 + Qwen2.5-Coder-7BRL Data=SynSQL-Complex-5K, Base Model=Qwen2.5-Coder-7B, Decoding Strategy=Greedy2026.0363.1
MTIR-SQL-4BRL Data=BIRD + SPIDER, Base Model=Qwen3-4B, Decoding Strategy=Greedy2026.0363.1
Deepseek-V3Decoding Strategy=Greedy2026.0362.5
Agentic SQL + CSMRRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Reward=CSMR, Decoding Strategy=Greedy2026.0362.5
ATR w/ Step-wise UpdateRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Config=Ablation Study, Decoding Strategy=Greedy2026.0361.3
Agentic SQLRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Reward=Binary, Decoding Strategy=Greedy2026.0361.1
ATR w/ Symmetric MatrixRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Config=Ablation Study, Decoding Strategy=Greedy2026.0360.1
Single-Turn GRPO + CSMRRL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Reward=CSMR, Decoding Strategy=Greedy2026.0359.4
Single-Turn GRPORL Data=BIRD, Base Model=Qwen2.5-7B-Instruct, Reward=Binary, Decoding Strategy=Greedy2026.0358.5
Qwen2.5-Coder-7BDecoding Strategy=Greedy2026.0358.2
Qwen2.5-7B-InstructDecoding Strategy=Greedy2026.0347.5
Table-SpecialistBase Model=GPT-4, Training Strategy=Table-Specialist2024.1047.5
GPT-4Base Model=GPT-4, Training Strategy=Vanilla2024.1043
Table-SpecialistBase Model=GPT-3.5, Training Strategy=Table-Specialist2024.1040.4
GPT-3.5Base Model=GPT-3.5, Training Strategy=Vanilla2024.1031.7
Table-SpecialistBase Model=Llama3.1-8B, Training Strategy=Table-Specialist2024.1026.1
Llama3.1-8BBase Model=Llama3.1-8B, Training Strategy=Vanilla2024.1022.5