Text-to-SQL on Spider (dev)
90.3EXFineStep
Evaluation results
| Method | Links | ||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FineStepModel Size=4B2026.05 | 90.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 84.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FineStep-4BParameters=4B2026.05 | 90.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 84.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAPOModel Size=4B2026.05 | 89.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 84.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GIGPOModel Size=4B2026.05 | 89.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GSPOModel Size=4B2026.05 | 89.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MCS-SQLBase Model=GPT-42026.05 | 89.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MCS-SQLBase Model=GPT-42026.06 | 89.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ReEx-SQL-7BParameters=7B2026.05 | 89.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FineStepModel Size=8B2026.05 | 88.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 82.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FineStep-8BParameters=8B2026.05 | 88.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 82.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MCTS-SQLBase Model=GPT-42026.05 | 88.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAPOModel Size=8B2026.05 | 88.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GIGPOModel Size=8B2026.05 | 88.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOModel Size=4B2026.05 | 88.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GSPOModel Size=8B2026.05 | 88.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DecoSearchLearning Approach=Open-source, Model Avail.=–2026.06 | 88.31 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOModel Size=30B-A3B2026.05 | 88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PAS-SQLBase Model=GPT-4o2026.05 | 87.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FineStepModel Size=30B-A3B2026.05 | 87.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQL-R1Base Model=Qwen2.5-Coder-7B2026.06 | 87.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQL-o1-8BParameters=8B2026.05 | 87.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 79.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CHESSLearning Approach=Training-free, Model Avail.=×2026.06 | 87.33 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Progress-SQL-14B(Ours)Reward=Progressive2026.06 | 87.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DataGpt-sql-7BAdditional Mechanism=Invalid-SQL Feedback2024.09 | 87.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOModel Size=8B2026.05 | 87.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 79.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Progress-SQL-7B(Ours)Reward=Progressive2026.06 | 87.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| JOLT-SQLBase Model=Qwen2.5-Coder-7B2026.06 | 87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MAC-SQLBase Model=GPT-42026.06 | 86.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BASE-SQLBase Model=Qwen2.5-Coder-14B2026.06 | 86.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MTIR-SQLBase Model=Qwen3-14B2026.05 | 86.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| STaR-SQL ORM@16Classification=Fine Tuning with Open Source LLMs, Model=Llama-3.1-8B-Instruct2025.02 | 86.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | 72.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SquRL-7Bworkflow=dynamic, number of workflows=5, aggregation=majority voting, parameter scale=7B2026.02 | 86.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OmniSQL-7BReward=-2026.06 | 85.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| No-TrainModel Size=4B2026.05 | 85.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 75.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQL-Trail-7B†Reward=One-shot2026.06 | 85.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 77.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeekLearning Approach=Open-source, Model Avail.=–2026.06 | 85.56 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SquRL-3Bworkflow=dynamic, number of workflows=5, aggregation=majority voting, parameter scale=3B2026.02 | 85.55 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DTS-SQLBase Model=DeepSeek-Coder-7B2026.06 | 85.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama-4-Maverick-17BLearning Approach=Open-source, Model Avail.=✓2026.06 | 85.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SFT CodeS-7BParameters=7B2026.05 | 85.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeSBase Model=StarCoderBase-7B2026.06 | 85.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SFT CodeS-7BReward=-2026.06 | 85.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SquRL-1.5Bworkflow=dynamic, number of workflows=5, aggregation=majority voting, parameter scale=1.5B2026.02 | 85.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DataGpt-sql-7BAdditional Mechanism=None2024.09 | 84.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ORM-based Best-of-NN=322026.06 | 84.53 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.13 | |
| SQl-R1-7BParameters=7B2026.05 | 84.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepEye-SQLLearning Approach=Training-free, Model Avail.=×2026.06 | 84.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLBackbone=GPT-4, Strategy=Self-Consistency2024.09 | 84.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| No-TrainModel Size=8B2026.05 | 84.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 72.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RSL-SQLworkflow=static, number of workflows=5, aggregation=majority voting2026.02 | 84.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemini 3 Prosampling=10 queries per question2026.04 | 84.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 84.3 | 0.843 | 0.721 | 53.7 | 13.9 | 33.7 | — | — | — | — | — | — | — | — | — | |
| GPT-OSS-120bLearning Approach=Open-source, Model Avail.=✓2026.06 | 84.16 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SENCEParameters=13B2024.09 | 84.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| REDSQL-3BFormat=NatSQL2024.09 | 84.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 73.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SkyRL-SQL-7BReward=One-shot2026.06 | 83.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Majority VotingN=322026.06 | 83.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 1.35 | |
| ROUTEClassification=Fine Tuning with Open Source LLMs, Model=Qwen2.5-7B2025.02 | 83.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLBase Model=GPT-42026.06 | 83.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MTIR-SQLBase Model=Qwen3-8B2026.06 | 83.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLLearning Approach=Training-free, Model Avail.=×2026.06 | 83.56 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLBackbone=GPT-42024.09 | 83.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 76.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ACE-SQLBase Model=Qwen3-8B2026.06 | 83.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-Coder-14B-InstructReward=-2026.06 | 83.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 77 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SENCEParameters=7B2024.09 | 83.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SENSE-7BClassification=Fine Tuning with Open Source LLMs, Model=CodeLlama-7B2025.02 | 83.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIN-SQLworkflow=static, number of workflows=5, aggregation=majority voting2026.02 | 83.14 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-OSS-20bLearning Approach=Open-source, Model Avail.=✓2026.06 | 83.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQL-PaLMMode=Fine-tuned2024.09 | 82.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIN-SQLBackbone=GPT-42024.09 | 82.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 74.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DIN-SQLBase Model=GPT-42026.06 | 82.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Execution-based Best-of-NN=322026.06 | 82.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.39 | |
| No-TrainModel Size=30B-A3B2026.05 | 82.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 69.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ExSLBase Model=DeepSeek-Coder-7B2026.06 | 82.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaselineN=12026.06 | 82.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KE-RLLLM=Qwen2.5-Coder-7B2026.05 | 82.34 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MARLOLLM=Llama 3 70B Instruct, k=82024.10 | 82.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SSLLM=Llama 3 70B Instruct, k=82024.10 | 82.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 82.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MAC-SQLworkflow=static, number of workflows=5, aggregation=majority voting2026.02 | 82.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LinkAlignworkflow=static, number of workflows=5, aggregation=majority voting2026.02 | 81.94 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GenaSQLLearning Approach=Training-free, Model Avail.=×2026.06 | 81.93 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mistral-nemotron-12BLearning Approach=Open-source, Model Avail.=✓2026.06 | 81.93 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MQSLLM=Llama 3 70B Instruct, k=82024.10 | 81.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 82.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SQL-R1-7B†Reward=One-shot2026.06 | 81.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 67.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLClassification=Prompting with LLMs, Model=GPT-42025.02 | 81.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | 69.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Claude Opus 4.5sampling=10 queries per question2026.04 | 81.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.6 | 0.779 | 0.65 | 60 | 3.8 | 22.4 | — | — | — | — | — | — | — | — | — | |
| OmniSQL-7BParameters=7B2026.05 | 81.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Graph-Reward-SQL-7BReward=One-shot2026.06 | 81.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| QSLLM=Llama 3 70B Instruct, k=82024.10 | 81.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OmniSQLBase Model=Qwen2.5-Coder-7B2026.06 | 81.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MARLOLLM=Claude 2.1, k=82024.10 | 80.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KE-RLLLM=Deepseek-Coder-7B2026.05 | 80.49 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KE-ICLLLM=Gemini-Pro-1.52026.05 | 80.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeSLLM=Gemini-Pro-1.5, Setting=ICL2026.05 | 80.17 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MARLOLLM=Mistral Large, k=82024.10 | 80 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-Coder-7B-InstructReward=-2026.06 | 79.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 73.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| T5-3B+PICARDBeam Size=4, Decoding=PICARD (top-2 guards), Database Content=true2021.09 | 79.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | 75.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CHESSworkflow=static, number of workflows=5, aggregation=majority voting2026.02 | 79.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RandomLLM=Llama 3 70B Instruct, k=82024.10 | 79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DAIL-SQLLLM=Gemini-Pro-1.5, Setting=ICL2026.05 | 78.91 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OmniLLM=Deepseek-Coder-7B, Setting=RL2026.05 | 78.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |