SQL Generation on BacktestBench synthetic (test)
100Execution Correctness Rate (ECR)Qwen3 235B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen3 235BAccessibility=Open-Source, Parameters=235B2026.05 | 100 | 97.11 | |
| Qwen3 32BAccessibility=Open-Source, Parameters=32B2026.05 | 100 | 90.72 | |
| Qwen3 MaxAccessibility=Closed-Source2026.05 | 100 | 94.73 | |
| Qwen3 30BAccessibility=Open-Source, Parameters=30B2026.05 | 99.97 | 92.86 | |
| Qwen3 8BAccessibility=Open-Source, Parameters=8B2026.05 | 99.97 | 86.89 | |
| Deepseek V3.2Accessibility=Open-Source2026.05 | 99.97 | 92.86 | |
| Qwen3 14BAccessibility=Open-Source, Parameters=14B2026.05 | 99.95 | 87.59 | |
| Qwen3 Coder PlusAccessibility=Closed-Source2026.05 | 99.95 | 94.03 | |
| Qwen3 Next 80BAccessibility=Open-Source, Parameters=80B2026.05 | 99.92 | 94.79 | |
| GLM 4.7Accessibility=Open-Source2026.05 | 99.92 | 94.86 | |
| Gemini 3 ProAccessibility=Closed-Source2026.05 | 99.64 | 98.7 | |
| MiniMax M2.1Accessibility=Open-Source2026.05 | 99.56 | 95.02 | |
| GPT OSS 120BAccessibility=Open-Source, Parameters=120B2026.05 | 99.43 | 96.58 | |
| Seed 1.8Accessibility=Closed-Source, Version=1.82026.05 | 99.4 | 97.45 | |
| Kimi Linear 48BAccessibility=Open-Source, Parameters=48B2026.05 | 99.22 | 48.46 | |
| Qwen3 4BAccessibility=Open-Source, Parameters=4B2026.05 | 98.85 | 69.94 | |
| Kimi K2 ThinkingAccessibility=Open-Source2026.05 | 98.2 | 96.06 | |
| Mimo V2 FlashAccessibility=Open-Source2026.05 | 97.84 | 80.42 | |
| GLM 4.7 FlashAccessibility=Open-Source2026.05 | 97.81 | 56.8 | |
| Ministral 3 14BAccessibility=Open-Source, Parameters=14B2026.05 | 97.18 | 72.26 | |
| GPT OSS 20BAccessibility=Open-Source, Parameters=20B2026.05 | 94.34 | 90.56 | |
| Ministral 3 8BAccessibility=Open-Source, Parameters=8B2026.05 | 92.23 | 46.35 | |
| Seed OSS 36BAccessibility=Open-Source, Parameters=36B2026.05 | 91.16 | 87.96 |