Text-to-Model Translation on NLP4LP (n=65)
76.92Execution AccuracyCoT + Grammar
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoT + GrammarLLM Calls=22026.04 | 76.92 | 38.46 | |
| Agentic + CodeLLM Calls=52026.04 | 76.92 | 29.23 | |
| Chain-of-Thought (CoT)LLM Calls=12026.04 | 70.77 | 35.38 | |
| CoT + Code & GrammarLLM Calls=32026.04 | 69.23 | 32.31 | |
| GalaLLM Calls=52026.04 | 66.15 | 32.31 | |
| CoT + Code ValidationLLM Calls=22026.04 | 64.62 | 30.77 | |
| Zero ShotLLM Calls=12026.04 | 63.08 | 32.31 | |
| Knowledge GraphLLM Calls=22026.04 | 52.31 | 32.31 | |
| AgenticLLM Calls=42026.04 | 50.77 | 32.31 |