Semantic Validation on Spider 2.0
92.59AUPRCHEROSQL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HEROSQLBackbone=Qwen3-0.6B, Domain Setting=Out-of-Domain2025.12 | 92.59 | 64.32 | |
| HEROSQLBackbone=Gemma-3-0.3B, Domain Setting=Out-of-Domain2025.12 | 91.55 | 59.85 | |
| TEDBackbone=Qwen3-0.6B, Domain Setting=Out-of-Domain2025.12 | 91.44 | 58.06 | |
| HEROSQL - w/o ASTBackbone=Qwen3-0.6B, Domain Setting=Out-of-Domain2025.12 | 91.28 | 59.13 | |
| HEROSQL - w/o ASTBackbone=Gemma-3-0.3B, Domain Setting=Out-of-Domain2025.12 | 90.9 | 53.28 | |
| HEROSQL - w/o AST & LPBackbone=Qwen3-0.6B, Domain Setting=Out-of-Domain2025.12 | 90.83 | 59.25 | |
| ConfScoreBackbone=Gemma-3-0.3B, Domain Setting=Out-of-Domain2025.12 | 90.79 | 52.34 | |
| HEROSQL - w/o NDABackbone=Gemma-3-0.3B, Domain Setting=Out-of-Domain2025.12 | 90.78 | 52.56 | |
| COVEBackbone=Qwen3-0.6B, Domain Setting=Out-of-Domain2025.12 | 90.75 | 61.86 | |
| HEROSQL - w/o NDABackbone=Qwen3-0.6B, Domain Setting=Out-of-Domain2025.12 | 90.73 | 59.36 | |
| ConfScoreBackbone=Qwen3-0.6B, Domain Setting=Out-of-Domain2025.12 | 90.15 | 44.77 | |
| HEROSQL - w/o AST & LPBackbone=Gemma-3-0.3B, Domain Setting=Out-of-Domain2025.12 | 90.13 | 54.49 | |
| CoTBackbone=Gemma-3-0.3B, Domain Setting=Out-of-Domain2025.12 | 89.22 | 50 | |
| CoTBackbone=Qwen3-0.6B, Domain Setting=Out-of-Domain2025.12 | 89.15 | 48.32 | |
| TEDBackbone=Gemma-3-0.3B, Domain Setting=Out-of-Domain2025.12 | 89.14 | 49.56 | |
| PromptBackbone=Qwen3-0.6B, Domain Setting=Out-of-Domain2025.12 | 88.78 | 51.8 | |
| COVEBackbone=Gemma-3-0.3B, Domain Setting=Out-of-Domain2025.12 | 88.51 | 45.86 | |
| PromptBackbone=Gemma-3-0.3B, Domain Setting=Out-of-Domain2025.12 | 88.3 | 44.58 |