Text-to-Model Translation on ComplexOR n=7
100Execution AccuracyZero Shot
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Zero ShotLLM Calls=12026.04 | 100 | 57.14 | |
| Chain-of-Thought (CoT)LLM Calls=12026.04 | 100 | 42.86 | |
| CoT + GrammarLLM Calls=22026.04 | 85.71 | 57.14 | |
| CoT + Code & GrammarLLM Calls=32026.04 | 85.71 | 57.14 | |
| AgenticLLM Calls=42026.04 | 85.71 | 57.14 | |
| Agentic + CodeLLM Calls=52026.04 | 85.71 | 57.14 | |
| GalaLLM Calls=52026.04 | 85.71 | 57.14 | |
| Knowledge GraphLLM Calls=22026.04 | 71.43 | 57.14 | |
| CoT + Code ValidationLLM Calls=22026.04 | 71.43 | 42.86 |