List your company

Text-to-SQL Error Repairing on ScienceBenchmark

63True Positive Repairs (EM)LLM-Value
6.8421.423650.58Jan 16, 2025

Evaluation results

MethodLinks
LLM-ValueLLM=GPT-3.5, Base Framework=MAC-SQL2025.016334
LLM-ExeLLM=Mistral, Base Framework=MAC-SQL2025.015962
LLM-ExtrLLM=Mistral, Base Framework=MAC-SQL2025.015865
LLM-PlainLLM=Mistral, Base Framework=MAC-SQL2025.015763
LLM-PlainLLM=GPT-3.5, Base Framework=MAC-SQL2025.015538
LLM-ExtrLLM=GPT-3.5, Base Framework=MAC-SQL2025.015539
LLM-ValueLLM=Mistral, Base Framework=MAC-SQL2025.015467
LLM-ExeLLM=GPT-3.5, Base Framework=MAC-SQL2025.015125
MapleDoctorLLM=Mistral, Base Framework=MAC-SQL2025.01429
MapleDoctorLLM=GPT-3.5, Base Framework=MAC-SQL2025.01388
LLM-ExeLLM=Qwen3, Base Framework=MAC-SQL2025.013814
LLM-ValueLLM=Qwen3, Base Framework=MAC-SQL2025.013734
MapleDoctorLLM=LLaMA, Base Framework=MAC-SQL2025.01373
LLM-PlainLLM=Qwen3, Base Framework=MAC-SQL2025.013525
LLM-ExtrLLM=Qwen3, Base Framework=MAC-SQL2025.013526
Rule-ExeLLM=Mistral, Base Framework=MAC-SQL2025.01354
LLM-ValueLLM=DSCoder, Base Framework=MAC-SQL2025.013436
MapleDoctorLLM=DSCoder, Base Framework=MAC-SQL2025.01345
LLM-ValueLLM=LLaMA, Base Framework=MAC-SQL2025.013331
LLM-ExeLLM=LLaMA, Base Framework=MAC-SQL2025.013233
Rule-ExeLLM=GPT-3.5, Base Framework=MAC-SQL2025.01306
LLM-ExeLLM=DSCoder, Base Framework=MAC-SQL2025.013030
LLM-ExtrLLM=DSCoder, Base Framework=MAC-SQL2025.012527
Rule-ExeLLM=LLaMA, Base Framework=MAC-SQL2025.01253
LLM-PlainLLM=DSCoder, Base Framework=MAC-SQL2025.012427
Rule-ExeLLM=Qwen3, Base Framework=MAC-SQL2025.01221
MapleDoctorLLM=Qwen3, Base Framework=MAC-SQL2025.011911
Rule-ExeLLM=DSCoder, Base Framework=MAC-SQL2025.01182
LLM-PlainLLM=LLaMA, Base Framework=MAC-SQL2025.01939
LLM-ExtrLLM=LLaMA, Base Framework=MAC-SQL2025.01945