Text-to-Model Translation on Hakank n=22
72.73Execution AccuracyCoT + Grammar
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoT + GrammarLLM Calls=22026.04 | 72.73 | 36.36 | |
| CoT + Code & GrammarLLM Calls=32026.04 | 72.73 | 45.45 | |
| Knowledge GraphLLM Calls=22026.04 | 68.18 | 31.82 | |
| Agentic + CodeLLM Calls=52026.04 | 68.18 | 36.36 | |
| CoT + Code ValidationLLM Calls=22026.04 | 63.64 | 36.36 | |
| Chain-of-Thought (CoT)LLM Calls=12026.04 | 59.09 | 36.36 | |
| GalaLLM Calls=52026.04 | 59.09 | 36.36 | |
| AgenticLLM Calls=42026.04 | 54.55 | 31.82 | |
| Zero ShotLLM Calls=12026.04 | 50 | 31.82 |