Text-to-model translation on Mamo-Easy Text2Zinc
99.85Execution Accuracy (Eacc)CoT + Grammar
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoT + GrammarLLM Calls=22026.04 | 99.85 | 83.74 | |
| CoT + Code & GrammarLLM Calls=32026.04 | 99.85 | 83.13 | |
| CoT + Code ValidationLLM Calls=22026.04 | 99.54 | 82.98 | |
| Agentic + CodeLLM Calls=52026.04 | 99.23 | 84.97 | |
| Chain-of-Thought (CoT)LLM Calls=12026.04 | 98.62 | 83.9 | |
| AgenticLLM Calls=42026.04 | 98.01 | 83.9 | |
| GalaLLM Calls=52026.04 | 96.63 | 80.21 | |
| Zero ShotLLM Calls=12026.04 | 95.71 | 79.75 |