NL-to-STL generation on STL-BENCH Chinese
47Formula AccuracyReasonSTL
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ReasonSTLThink=✓, Tool=✓2026.05 | 47 | 49 | — | |
| ReasonSTL-DirectThink=✗, Tool=✗2026.05 | 33 | 41 | — | |
| DeepSeek-V4Think=–, Tool=–2026.05 | 27 | 35 | — | |
| Claude-Opus-4.7Think=–, Tool=–2026.05 | 27 | 28 | — | |
| GPT-5.4Think=–, Tool=–2026.05 | 19 | 20 | — | |
| Kimi-K2.6Think=–, Tool=–2026.05 | 19 | 20 | — | |
| Qwen3-4BThink=✗, Tool=✗2026.05 | 9 | 13 | — | |
| Claude-Opus-4.72026.05 | — | — | 49 | |
| DeepSeek-V42026.05 | — | — | 51 | |
| ReasonSTL2026.05 | — | — | 51 |