Natural Language to Signal Temporal Logic (NL-to-STL) Generation on STL-DivEn (test)
55.87STL Formula AccuracyKGST
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| KGSTbackbone=LLaMA 3-8B, refinement=GPT-4 with external knowledge2025.05 | 55.87 | 56.27 | 21.42 | |
| DeepSeekversion=DeepSeek-V32025.05 | 47.9 | 48.25 | 7.91 | |
| GPT-4version=gpt-4-0125-preview2025.05 | 47.33 | 47.41 | 8.31 | |
| GPT-4+Self-Refinerefinement=internal knowledge2025.05 | 44.22 | 44.66 | 5.21 | |
| GPT-3.5version=gpt-3.5-turbo-11062025.05 | 30.18 | 30.34 | 4.24 | |
| DeepSTLarchitecture=Transformers2025.05 | 19.86 | 18.83 | 2.93 |