Natural Language to STL Transformation on DeepSTL
81.43STL Formula AccuracyReasonSTL-Direct
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ReasonSTL-DirectBackbone=Qwen3-4B, Temperature=0.02026.05 | 81.43 | 82.57 | — | |
| Claude-Opus-4.7Temperature=0.12026.05 | 66.03 | 66.91 | — | |
| RESTL2026.05 | 59.85 | 63.27 | — | |
| DeepSeek-V42026.05 | 56.85 | 57.58 | — | |
| GPT-5.4Temperature=0.12026.05 | 51.5 | 52.2 | — | |
| Kimi-K2.6Temperature=1.02026.05 | 50.48 | 51.13 | — | |
| KGSTbackbone=LLAMA 3-8B, refinement=GPT-4 with external knowledge2025.05 | 45.38 | 49.39 | 56.86 | |
| KGST2026.05 | 45.38 | 49.39 | — | |
| DeepSeekversion=DeepSeek-V32025.05 | 25.37 | 32.54 | 39.82 | |
| GPT-4version=gpt-4-0125-preview2025.05 | 22.62 | 30.48 | 28.81 | |
| GPT-4+Self-Refinerefinement=GPT-4 self-refinement2025.05 | 22.03 | 30.19 | 26.82 | |
| GPT-3.5version=gpt-3.5-turbo-11062025.05 | 21.45 | 30.02 | 22.49 | |
| DeepSTLmodel_architecture=Transformers2025.05 | 20.02 | 29.16 | 33.32 | |
| DeepSTL2026.05 | 20.02 | 29.16 | — | |
| Qwen3-4BTemperature=0.02026.05 | 11.31 | 16.31 | — |