Natural Language to Signal Temporal Logic (NL-to-STL) Generation on DeepSTL (test)
45.38STL Formula AccuracyKGST
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| KGSTbackbone=LLaMA 3-8B, refinement=GPT-4 with external knowledge2025.05 | 45.38 | 49.39 | 56.86 | |
| DeepSeekversion=DeepSeek-V32025.05 | 25.37 | 32.54 | 39.82 | |
| GPT-4version=gpt-4-0125-preview2025.05 | 22.62 | 30.48 | 28.81 | |
| GPT-4+Self-Refinerefinement=internal knowledge2025.05 | 22.03 | 30.19 | 26.82 | |
| GPT-3.5version=gpt-3.5-turbo-11062025.05 | 21.45 | 30.02 | 22.49 | |
| DeepSTLarchitecture=Transformers2025.05 | 20.02 | 29.16 | 33.32 |