NL-to-STL generation on STL-BENCH Manually Written (test)
54Formula AccuracyReasonSTL
Evaluation results
| Method | Links | |
|---|---|---|
| ReasonSTLThroughput (qps)=2.32026.05 | 54 | |
| Claude-Opus-4.7Throughput (qps)=2.22026.05 | 53 | |
| DeepSeek-V4Throughput (qps)=0.62026.05 | 51 | |
| ReasonSTL-DirectThroughput (qps)=6.12026.05 | 42 |