SQL Code Generation on InterCode SQL
64.01Success RateOPD-Evolver-9B
Evaluation results
| Method | Links | |
|---|---|---|
| OPD-Evolver-9BBackbone LLM=QWEN3.5-9B2026.06 | 64.01 | |
| QWEN3.5-397B-A17BBackbone LLM=QWEN3.5-397B-A17B2026.06 | 62.74 | |
| MemEvolveBackbone LLM=QWEN3.5-9B2026.06 | 61.15 | |
| EvolveRBackbone LLM=QWEN3.5-9B2026.06 | 60.51 | |
| STEP-3.5-FLASH (196B)Backbone LLM=STEP-3.5-FLASH (196B)2026.06 | 59.87 | |
| MempBackbone LLM=QWEN3.5-9B2026.06 | 58.28 | |
| ReasoningBankBackbone LLM=QWEN3.5-9B2026.06 | 57.96 | |
| CheatsheetBackbone LLM=QWEN3.5-9B2026.06 | 56.05 | |
| No MemoryBackbone LLM=QWEN3.5-9B2026.06 | 55.73 | |
| ExpeLBackbone LLM=QWEN3.5-9B2026.06 | 52.23 | |
| AWMBackbone LLM=QWEN3.5-9B2026.06 | 51.59 | |
| OPD-Evolver-4BBackbone LLM=QWEN3-4B2026.06 | 45.86 | |
| EvolveRBackbone LLM=QWEN3-4B2026.06 | 44.9 | |
| MemEvolveBackbone LLM=QWEN3-4B2026.06 | 44.59 | |
| ReasoningBankBackbone LLM=QWEN3-4B2026.06 | 43.95 | |
| MempBackbone LLM=QWEN3-4B2026.06 | 43.31 | |
| No MemoryBackbone LLM=QWEN3-4B2026.06 | 38.85 | |
| AWMBackbone LLM=QWEN3-4B2026.06 | 38.54 | |
| CheatsheetBackbone LLM=QWEN3-4B2026.06 | 37.9 | |
| ExpeLBackbone LLM=QWEN3-4B2026.06 | 34.08 | |
| LUMOS-IAll-13Bshot=2-/3-shot2023.11 | 7.3 | |
| Vicuna-v1.3-33B2023.11 | 6.7 | |
| LUMOS-IMath-13Bshot=2-/3-shot2023.11 | 5.8 | |
| Vicuna-v1.5-13B2023.11 | 4.8 | |
| LUMOS-IWeb-13Bshot=2-/3-shot2023.11 | 4.2 | |
| LUMOS-IMM-13Bshot=2-/3-shot2023.11 | 4 | |
| LUMOS-IQA-13Bshot=2-/3-shot2023.11 | 3.5 |