NL-to-STL Generation on STL-BENCH English
53Formula AccuracyReasonSTL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ReasonSTL2026.05 | 53 | — | |
| Claude-Opus-4.72026.05 | 53 | — | |
| ReasonSTLThink=✓, Tool=✓2026.05 | 51 | 56 | |
| DeepSeek-V42026.05 | 50 | — | |
| ReasonSTL-DirectThink=✗, Tool=✗2026.05 | 42 | 50 | |
| DeepSeek-V4Think=–, Tool=–2026.05 | 31 | 33 | |
| Claude-Opus-4.7Think=–, Tool=–2026.05 | 26 | 28 | |
| GPT-5.4Think=–, Tool=–2026.05 | 22 | 24 | |
| Kimi-K2.6Think=–, Tool=–2026.05 | 17 | 20 | |
| Qwen3-4BThink=✗, Tool=✗2026.05 | 7 | 11 |