# SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs

Source: [SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs](<https://www.sota2.com/research/paper/68fa7b801e209131cea88b5d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T04:27:42.240Z

Authors: Yu Guo, Dong Jin, Shenghao Ye, Shuangwu Chen, Jian Yang, Xiaobin Tan

Published: 2025-05-19T20:51:29.000Z

## Abstract

Large Language models (LLMs) have demonstrated significant potential in text-to-SQL reasoning tasks, yet a substantial performance gap persists between existing open-source models and their closed-source counterparts. In this paper, we introduce SQLForge, a novel approach for synthesizing reliable and diverse data to enhance text-to-SQL reasoning in LLMs. We improve data reliability through SQL syntax constraints and SQL-to-question reverse translation, ensuring data logic at both structural and semantic levels. We also propose an SQL template enrichment and iterative data domain exploration mechanism to boost data diversity. Building on the augmented data, we fine-tune a variety of open-source models with different architectures and parameter sizes, resulting in a family of models termed SQLForge-LM. SQLForge-LM achieves the state-of-the-art performance on the widely recognized Spider and BIRD benchmarks among the open-source models. Specifically, SQLForge-LM achieves EX accuracy of 85.7% on Spider Dev and 59.8% on BIRD Dev, significantly narrowing the performance gap with closed-source methods.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Text-to-SQL on Spider Robustness Suite SYN REALISTIC DK (dev)](<https://www.sota2.com/research/sota/text-to-sql-on-spider-robustness-suite-syn-realistic-dk-dev.md>): Text-to-SQL; Spider Robustness Suite SYN REALISTIC DK (dev); Execution Accuracy (SYN): 78.9.

- [Text-to-SQL on Spider (test)](<https://www.sota2.com/research/sota/text-to-sql-on-spider-test.md>): Text-to-SQL; Spider (test); Execution Accuracy: 87.4.

- [Text-to-SQL on BIRD (dev)](<https://www.sota2.com/research/sota/text-to-sql-on-bird-dev.md>): Text-to-SQL; BIRD (dev); Execution Accuracy (EA): 59.8.

- [Text-to-SQL on Spider (dev)](<https://www.sota2.com/research/sota/text-to-sql-on-spider-dev.md>): Text-to-SQL; Spider (dev); EX: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2505.13725>)

[PDF](<https://arxiv.org/pdf/2505.13725.pdf>)
