Multi-turn Text-to-SQL on SParC Out-of-domain
77.4Execution Accuracy (EX)[Long-Horizon] Warm-Start SFT + RL (Outcome + Process)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| [Long-Horizon] Warm-Start SFT + RL (Outcome + Process)Model Size=4B, Horizon Setting=Long-Horizon2025.10 | 77.4 | 69.1 | |
| [Long-Horizon] Warm-Start SFT + RL (Outcome Only)Model Size=4B, Horizon Setting=Long-Horizon2025.10 | 76 | 69 | |
| [Short-Horizon] SFTModel Size=4B, Horizon Setting=Short-Horizon2025.10 | 75.1 | 68.9 | |
| [Long-Horizon] Warm-Start SFT + RL (Outcome + Process)Model Size=1.7B, Horizon Setting=Long-Horizon2025.10 | 74.5 | 68 | |
| [Long-Horizon] Warm-Start SFT Only (Round 1)Model Size=4B, Horizon Setting=Long-Horizon, Learning Round=Round 12025.10 | 74 | 64 | |
| [Long-Horizon] + Warm-Start SFT Only (Round 3)Model Size=4B, Horizon Setting=Long-Horizon, Learning Round=Round 32025.10 | 74 | 64.4 | |
| [Long-Horizon] + Warm-Start SFT Only (Round 2)Model Size=4B, Horizon Setting=Long-Horizon, Learning Round=Round 22025.10 | 73.7 | 62.4 | |
| [Short-Horizon] Direct RLModel Size=4B, Horizon Setting=Short-Horizon2025.10 | 73.4 | 64 | |
| [Long-Horizon] Warm-Start SFT + RL (Outcome Only)Model Size=1.7B, Horizon Setting=Long-Horizon2025.10 | 73 | 66.2 | |
| [Short-Horizon] SFTModel Size=1.7B, Horizon Setting=Short-Horizon2025.10 | 71.7 | 65.1 | |
| [Long-Horizon] + Warm-Start SFT Only (Round 3)Model Size=1.7B, Horizon Setting=Long-Horizon, Learning Round=Round 32025.10 | 71.3 | 62.7 | |
| [Short-Horizon] Direct RLModel Size=1.7B, Horizon Setting=Short-Horizon2025.10 | 70.6 | 62.3 | |
| [Long-Horizon] + Warm-Start SFT Only (Round 2)Model Size=1.7B, Horizon Setting=Long-Horizon, Learning Round=Round 22025.10 | 70 | 61.5 | |
| [Long-Horizon] Warm-Start SFT Only (Round 1)Model Size=1.7B, Horizon Setting=Long-Horizon, Learning Round=Round 12025.10 | 67.3 | 58.1 | |
| RASAT+PICARDModel Size=3B2025.10 | 61.9 | 56.1 | |
| CoE-SQLModel Size=Closed-Source, Prompting Strategy=Few-shot, 16-shot2025.10 | 57.9 | 48.5 |