Text-to-SQL on Spider (Accuracy & Eqv@2pp)
82.5AccuracySGT (DPO Iteration 4)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SGT (DPO Iteration 4)Actor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_4 → πA2026.04 | 82.5 | — | — | — | |
| SGT (DPO Iteration 5)Actor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_5 → πA2026.04 | 82.3 | — | — | — | |
| SGT (DPO Iteration 3)Actor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_3 → πA2026.04 | 81 | — | — | — | |
| SGT (DPO Iteration 2)Actor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_2 → πA2026.04 | 79.8 | — | — | — | |
| SGT (DPO Iteration 1)Actor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_1 → πA2026.04 | 76.1 | — | — | — | |
| SGT (DPO Iteration 3)Actor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_3 → πA2026.04 | 76.1 | — | — | — | |
| SGT (DPO Iteration 1)Actor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_1 → πA2026.04 | 75.6 | — | — | — | |
| SGT (DPO Iteration 4)Actor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_4 → πA2026.04 | 74.7 | — | — | — | |
| SGT (DPO Iteration 2)Actor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_2 → πA2026.04 | 74.6 | — | — | — | |
| SGT (DPO Iteration 5)Actor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_5 → πA2026.04 | 74.4 | — | — | — | |
| SFT SupplementActor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_SFT → πA2026.04 | 72.1 | — | — | — | |
| DSPyActor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_DSPy → πA2026.04 | 71.9 | — | — | — | |
| TextGradActor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_TG → πA2026.04 | 71.6 | — | — | — | |
| SFT SupplementActor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_SFT → πA2026.04 | 71.5 | — | — | — | |
| Baseline ActorActor Model=gpt-oss-120b, Supplement Model=None, Supplement Generation Method=∅ → πA2026.04 | 70.7 | — | — | — | |
| Inference-time scaling (ITS)Actor Model=gpt-oss-120b, Supplement Model=None, Supplement Generation Method=∅ → πA_ITS2026.04 | 70.4 | — | — | — | |
| Prompted SupplementActor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_prompt → πA2026.04 | 69.9 | — | — | — | |
| DSPyActor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_DSPy → πA2026.04 | 69.5 | — | — | — | |
| Prompted SupplementActor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_prompt → πA2026.04 | 68.2 | — | — | — | |
| TextGradActor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_TG → πA2026.04 | 65.8 | — | — | — | |
| Inference-time scaling (ITS)Actor Model=v3.5-sonnet-v2, Supplement Model=None, Supplement Generation Method=∅ → πA_ITS2026.04 | 64.7 | — | — | — | |
| Baseline ActorActor Model=v3.5-sonnet-v2, Supplement Model=None, Supplement Generation Method=∅ → πA2026.04 | 64.1 | — | — | — | |
| Qwen1.5-MoE-A2.7BCondition=Full LoRA2026.03 | 52 | — | — | — | |
| MoE-Sieve (Qwen1.5-MoE-A2.7B)Condition=Hot (25%)2026.03 | 51.1 | 0.93 | 1.88 | — | |
| Supplement Solver OnlyActor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=∅ → πS_solve2026.04 | 49.5 | — | — | — | |
| Supplement Solver OnlyActor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=∅ → πS_solve2026.04 | 49.5 | — | — | — | |
| MoE-Sieve (OLMoE-1B-7B)Condition=Hot (25%)2026.03 | 39.9 | 0.3 | 2.04 | — | |
| OLMoE-1B-7BCondition=Full LoRA2026.03 | 39.6 | — | — | — |