Text-to-model translation on Mamo-Complex Text2Zinc
96.68Execution AccuracyCoT + Code & Grammar
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoT + Code & GrammarLLM Calls=32026.04 | 96.68 | 55.92 | |
| CoT + GrammarLLM Calls=22026.04 | 95.26 | 54.03 | |
| CoT + Code ValidationLLM Calls=22026.04 | 94.31 | 54.98 | |
| Agentic + CodeLLM Calls=52026.04 | 92.89 | 54.03 | |
| GalaLLM Calls=52026.04 | 72.04 | 41.71 | |
| AgenticLLM Calls=42026.04 | 62.56 | 38.86 | |
| Chain-of-Thought (CoT)LLM Calls=12026.04 | 57.35 | 39.34 | |
| Zero ShotLLM Calls=12026.04 | 48.82 | 27.96 |