Multi-turn Text-to-SQL on CoSQL In-domain
65.2Exact Match[Long-Horizon] Warm-Start SFT + RL (Outcome + Process)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| [Long-Horizon] Warm-Start SFT + RL (Outcome + Process)Model Size=4B, Horizon Setting=Long-Horizon2025.10 | 65.2 | 79.9 | |
| [Short-Horizon] SFTModel Size=4B, Horizon Setting=Short-Horizon2025.10 | 64.8 | 73.1 | |
| [Short-Horizon] Direct RLModel Size=4B, Horizon Setting=Short-Horizon2025.10 | 64.8 | 75.2 | |
| [Long-Horizon] Warm-Start SFT + RL (Outcome Only)Model Size=4B, Horizon Setting=Long-Horizon2025.10 | 64.5 | 79.1 | |
| [Long-Horizon] Warm-Start SFT + RL (Outcome + Process)Model Size=1.7B, Horizon Setting=Long-Horizon2025.10 | 63.5 | 77.3 | |
| [Long-Horizon] + Warm-Start SFT Only (Round 3)Model Size=4B, Horizon Setting=Long-Horizon, Learning Round=Round 32025.10 | 63 | 75.2 | |
| [Long-Horizon] + Warm-Start SFT Only (Round 2)Model Size=4B, Horizon Setting=Long-Horizon, Learning Round=Round 22025.10 | 62.8 | 74.7 | |
| [Long-Horizon] Warm-Start SFT + RL (Outcome Only)Model Size=1.7B, Horizon Setting=Long-Horizon2025.10 | 62.7 | 76.6 | |
| [Long-Horizon] + Warm-Start SFT Only (Round 3)Model Size=1.7B, Horizon Setting=Long-Horizon, Learning Round=Round 32025.10 | 62.1 | 73 | |
| [Long-Horizon] Warm-Start SFT Only (Round 1)Model Size=4B, Horizon Setting=Long-Horizon, Learning Round=Round 12025.10 | 62.1 | 73.9 | |
| [Long-Horizon] + Warm-Start SFT Only (Round 2)Model Size=1.7B, Horizon Setting=Long-Horizon, Learning Round=Round 22025.10 | 60.5 | 72.2 | |
| [Short-Horizon] SFTModel Size=1.7B, Horizon Setting=Short-Horizon2025.10 | 59.3 | 68.1 | |
| [Short-Horizon] Direct RLModel Size=1.7B, Horizon Setting=Short-Horizon2025.10 | 59 | 72.8 | |
| RASAT+PICARDModel Size=3B2025.10 | 58.8 | 67 | |
| [Long-Horizon] Warm-Start SFT Only (Round 1)Model Size=1.7B, Horizon Setting=Long-Horizon, Learning Round=Round 12025.10 | 57.6 | 69.9 | |
| HIE-SQL+GraPPModel Size=~125M2025.10 | 56.4 | — | |
| Qwen-3-32BModel Size=32B2025.10 | 54.4 | 66.8 | |
| Qwen-3-14BModel Size=14B2025.10 | 54.3 | 66.5 | |
| CoE-SQLModel Size=Closed-Source, Prompting Strategy=Few-shot, 16-shot2025.10 | 52.4 | 69.6 | |
| Qwen-3-4BModel Size=4B2025.10 | 50.7 | 64 | |
| Qwen-3-1.7BModel Size=1.7B2025.10 | 49.3 | 59.9 | |
| ACT-SQLModel Size=Closed-Source2025.10 | 46 | 63.7 | |
| [Long-Horizon] Base Agent without TrainingModel Size=4B, Horizon Setting=Long-Horizon2025.10 | 45.6 | 60.3 | |
| GAZP+BERTModel Size=~215M2025.10 | 42 | 38.8 | |
| DeepSeek-R1Model Size=671B2025.10 | 36 | 58.5 | |
| Planning/Tool-using Agent built by LangGraphModel Size=Closed-Source2025.10 | 32.5 | 69.9 | |
| GPT-4.1Model Size=Closed-Source2025.10 | 32.1 | 60.9 | |
| OpenAI-O3Model Size=Closed-Source2025.10 | 29.1 | 59.8 | |
| [Long-Horizon] Base Agent without TrainingModel Size=1.7B, Horizon Setting=Long-Horizon2025.10 | 16.3 | 22.6 |