List your company

Text-to-SQL on Spider Robustness Suite SYN REALISTIC DK (dev)

78.9Execution Accuracy (SYN)SQLForge-LM
72.34874.04975.7577.451May 19, 2025

Evaluation results

MethodLinks
SQLForge-LMBackbone=CodeLlama-13B, Multiple Calls=false2025.0578.984.882.181.9
SENSEBackbone=CodeLlama-13B, Multiple Calls=false2025.0577.684.180.280.6
CodeSBackbone=StarCoder-15B, Multiple Calls=false2025.057783.170.776.9
RESDSQLBackbone=ROBERTa-3B, Multiple Calls=true2025.0576.981.96674.9
SQLForge-LMBackbone=CodeLlama-7B, Multiple Calls=false2025.0574.683.378.978.8
SENSEBackbone=CodeLlama-7B, Multiple Calls=false2025.0572.682.777.977.7