Text-to-Model Translation on CspLib n=11
90.91Execution AccuracyCoT + Grammar
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoT + GrammarLLM Calls=22026.04 | 90.91 | 45.45 | |
| CoT + Code & GrammarLLM Calls=32026.04 | 90.91 | 45.45 | |
| Agentic + CodeLLM Calls=52026.04 | 81.82 | 36.36 | |
| CoT + Code ValidationLLM Calls=22026.04 | 72.73 | 36.36 | |
| Zero ShotLLM Calls=12026.04 | 63.64 | 36.36 | |
| AgenticLLM Calls=42026.04 | 63.64 | 18.18 | |
| GalaLLM Calls=52026.04 | 63.64 | 45.45 | |
| Chain-of-Thought (CoT)LLM Calls=12026.04 | 54.55 | 27.27 | |
| Knowledge GraphLLM Calls=22026.04 | 36.36 | 27.27 |