Text-to-SQL on EHRSQL
85.2Execution AccuracySkillTrojan
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SkillTrojanModel=GLM-4.72026.04 | 85.2 | 62.1 | |
| AgentPoisonModel=GLM-4.72026.04 | 85 | 57.2 | |
| Non-attackModel=GLM-4.72026.04 | 84.1 | 0 | |
| BadChainModel=GLM-4.72026.04 | 84 | 31.8 | |
| GCGModel=GLM-4.72026.04 | 83.7 | 35.1 | |
| AutoDANModel=GLM-4.72026.04 | 82.2 | 46.8 | |
| CPAModel=GLM-4.72026.04 | 81.4 | 44.2 | |
| SkillTrojanModel=GLM-4.62026.04 | 81.3 | 72 | |
| AgentPoisonModel=GLM-4.62026.04 | 77.1 | 60.8 | |
| BadChainModel=GLM-4.62026.04 | 76.6 | 23.7 | |
| SkillTrojanModel=Qwen3-Coder2026.04 | 76.3 | 64.7 | |
| Non-attackModel=GLM-4.62026.04 | 76 | 0 | |
| GCGModel=GLM-4.62026.04 | 75.3 | 38.6 | |
| AutoDANModel=GLM-4.62026.04 | 73.6 | 55.6 | |
| AgentPoisonModel=Qwen3-Coder2026.04 | 72.4 | 62.5 | |
| CPAModel=GLM-4.62026.04 | 72.1 | 52.7 | |
| Non-attackModel=Qwen3-Coder2026.04 | 71.5 | 0 | |
| BadChainModel=Qwen3-Coder2026.04 | 70.7 | 18.4 | |
| GCGModel=Qwen3-Coder2026.04 | 70.2 | 41 | |
| AutoDANModel=Qwen3-Coder2026.04 | 68.9 | 51.4 | |
| CPAModel=Qwen3-Coder2026.04 | 67.8 | 57.9 | |
| KE-ICLLLM=Gemini-Pro-1.52026.05 | 55.46 | — | |
| Non-attackModel=Qwen3-VL-235B-A22B-Instruct2026.04 | 53.2 | 0 | |
| BadChainModel=Qwen3-VL-235B-A22B-Instruct2026.04 | 52.6 | 7.9 | |
| CodeSLLM=Gemini-Pro-1.5, Setting=ICL2026.05 | 52.28 | — | |
| AgentPoisonModel=Qwen3-VL-235B-A22B-Instruct2026.04 | 52 | 37.6 | |
| GCGModel=Qwen3-VL-235B-A22B-Instruct2026.04 | 51.8 | 24.9 | |
| DAIL-SQLLLM=Gemini-Pro-1.5, Setting=ICL2026.05 | 51 | — | |
| AutoDANModel=Qwen3-VL-235B-A22B-Instruct2026.04 | 49.5 | 19.7 | |
| CPAModel=Qwen3-VL-235B-A22B-Instruct2026.04 | 48.9 | 31.5 | |
| SkillTrojanModel=Qwen3-VL-235B-A22B-Instruct2026.04 | 48.4 | 26.7 | |
| KE-RLLLM=Qwen2.5-Coder-7B2026.05 | 47.93 | — | |
| KE-ICLLLM=GPT-4o2026.05 | 47.12 | — | |
| KE-RLLLM=Deepseek-Coder-7B2026.05 | 45.64 | — | |
| KE-RLLLM=Granite-3.1-8B2026.05 | 41.48 | — | |
| CodeSLLM=GPT-4o, Setting=ICL2026.05 | 41.27 | — | |
| DAIL-SQLLLM=GPT-4o, Setting=ICL2026.05 | 40.02 | — | |
| OmniSQL-32BModel Scale Category=10-50B2025.04 | 39.1 | — | |
| Think2SQL-14BModel Scale Category=10-50B2025.04 | 38.9 | — | |
| EVOLSQL-7BParameters=7B2026.01 | 38.6 | 61.9 | |
| OmniLLM=Qwen2.5-Coder-7B, Setting=RL2026.05 | 38 | — | |
| OmniSQL-7BModel Scale Category=5-10B2025.04 | 36.9 | — | |
| GATEBackbone LLM=Qwen3-8B2026.06 | 35 | — | |
| OmniSQL-7BParameters=7B2026.01 | 34.9 | 61.4 | |
| SQL-GENLLM=Qwen2.5-Coder-7B, Setting=RL2026.05 | 34.51 | — | |
| GATEBackbone LLM=GPT-5.4-mini2026.06 | 34.4 | — | |
| Think2SQL-4BModel Scale Category=< 5B2025.04 | 33.5 | — | |
| KE-ICLLLM=Qwen2.5-Coder-7B2026.05 | 33.43 | — | |
| SQL-GENLLM=Granite-3.1-8B, Setting=RL2026.05 | 33.1 | — | |
| Think2SQL-8BModel Scale Category=5-10B2025.04 | 32.9 | — | |
| ReActBackbone LLM=GPT-5.4-mini2026.06 | 32.2 | — | |
| OmniLLM=Deepseek-Coder-7B, Setting=RL2026.05 | 31.46 | — | |
| SFT (BIRD+Spider)Training Data=BIRD+Spider2026.01 | 31.4 | 55.8 | |
| ReFoRCEBackbone LLM=Qwen3-8B2026.06 | 31.2 | — | |
| OmniLLM=Granite-3.1-8B, Setting=RL2026.05 | 30.7 | — | |
| ReActBackbone LLM=Qwen3-8B2026.06 | 30.2 | — | |
| SQL-GENLLM=Deepseek-Coder-7B, Setting=RL2026.05 | 28.3 | — | |
| ReFoRCEBackbone LLM=GPT-5.4-mini2026.06 | 27.6 | — | |
| ReDelBackbone LLM=GPT-5.4-mini2026.06 | 26.8 | — | |
| ReDelBackbone LLM=Qwen3-8B2026.06 | 26.4 | — | |
| Base ModelBackbone=Qwen2.5-Coder-7B-Instruct2026.01 | 24.3 | 53.4 | |
| KE-ICLLLM=Deepseek-Coder-7B2026.05 | 23.51 | — | |
| CodeSLLM=Qwen2.5-Coder-7B, Setting=ICL2026.05 | 21.03 | — | |
| KE-ICLLLM=Granite-3.1-8B2026.05 | 20.63 | — | |
| DAIL-SQLLLM=Qwen2.5-Coder-7B, Setting=ICL2026.05 | 19.84 | — | |
| CodeSLLM=Deepseek-Coder-7B, Setting=ICL2026.05 | 14.88 | — | |
| CodeSLLM=Granite-3.1-8B, Setting=ICL2026.05 | 13.59 | — | |
| DAIL-SQLLLM=Deepseek-Coder-7B, Setting=ICL2026.05 | 13.41 | — | |
| DAIL-SQLLLM=Granite-3.1-8B, Setting=ICL2026.05 | 12.3 | — |