Text-to-model translation on NL4Opt Text2Zinc
98.78Execution Accuracy (Eacc)Chain-of-Thought (CoT)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Chain-of-Thought (CoT)LLM Calls=12026.04 | 98.78 | 78.37 | |
| CoT + Code & GrammarLLM Calls=32026.04 | 98.78 | 77.55 | |
| CoT + GrammarLLM Calls=22026.04 | 97.96 | 77.96 | |
| Agentic + CodeLLM Calls=52026.04 | 97.55 | 77.96 | |
| CoT + Code ValidationLLM Calls=22026.04 | 97.14 | 73.88 | |
| AgenticLLM Calls=42026.04 | 95.92 | 77.55 | |
| Zero ShotLLM Calls=12026.04 | 93.88 | 74.29 | |
| GalaLLM Calls=52026.04 | 90.61 | 71.02 |