Workflow Generation and Execution on FlowBench 141 cases (694 tests)
0.355CRateES-P&E
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ES-P&EModel=GPT-52026.02 | 0.355 | 0.369 | 1 | 0.8085 | 0.8085 | |
| P&EModel=GPT-52026.02 | 0.326 | 0.357 | 0.4326 | 0.766 | 0.4326 | |
| ES-P&EModel=GPT-4.12026.02 | 0.298 | 0.333 | 0.9645 | 0.9291 | 0.9078 | |
| ES-ReActModel=GPT-4.12026.02 | 0.262 | 0.304 | 0.9787 | 0.8582 | 0.8369 | |
| ReActModel=GPT-52026.02 | 0.262 | 0.3 | 0.6879 | 0.6667 | 0.5461 | |
| ES-P&EModel=Qwen3-14B2026.02 | 0.241 | 0.275 | 0.8936 | 0.9433 | 0.8511 | |
| P&EModel=GPT-4.12026.02 | 0.241 | 0.304 | 0.9575 | 0.9716 | 0.9504 | |
| ES-ReActModel=GPT-52026.02 | 0.234 | 0.287 | 0.7872 | 0.9433 | 0.7376 | |
| ES-P&EModel=Qwen3-8B2026.02 | 0.213 | 0.259 | 0.8865 | 0.8723 | 0.7872 | |
| ES-ReActModel=Qwen3-14B2026.02 | 0.213 | 0.267 | 0.7447 | 0.922 | 0.7163 | |
| ES-ReActModel=Qwen3-32B2026.02 | 0.213 | 0.241 | 0.7305 | 0.7376 | 0.5745 | |
| ReActModel=GPT-4.12026.02 | 0.213 | 0.233 | 0.7426 | 0.8489 | 0.7194 | |
| P&EModel=Qwen3-14B2026.02 | 0.206 | 0.265 | 0.3121 | 0.7518 | 0.305 | |
| ReActModel=Qwen3-32B2026.02 | 0.193 | 0.218 | 0.7 | 0.4 | 0.3857 | |
| P&EModel=Qwen3-32B2026.02 | 0.191 | 0.251 | 0.2624 | 0.4681 | 0.2128 | |
| ES-P&EModel=Qwen3-32B2026.02 | 0.191 | 0.249 | 0.8936 | 0.8723 | 0.7943 | |
| ES-ReActModel=Qwen3-8B2026.02 | 0.184 | 0.232 | 0.6454 | 0.7305 | 0.5177 | |
| P&EModel=Qwen3-8B2026.02 | 0.177 | 0.21 | 0.2766 | 0.7305 | 0.234 | |
| ReActModel=Qwen3-14B2026.02 | 0.149 | 0.16 | 0.8085 | 0.3404 | 0.3121 | |
| ReActModel=Qwen3-8B2026.02 | 0.057 | 0.062 | 0.6383 | 0.3404 | 0.2553 |