Sequential Function Chart Generation on Industrial SFC Dataset 1.0 (test)
81.88Pass@k (Gen)Qwen2.5-Coder-14B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Qwen2.5-Coder-14BFine Tuning Type=NTP, Prompting Setting=3-shot, Generation Mode=structured, k=52025.12 | 81.88 | 81.88 | 77.99 | |
| Qwen3-14BFine Tuning Type=NTP, Prompting Setting=3-shot, Generation Mode=structured, k=52025.12 | 78.12 | 78.07 | 72.34 | |
| Qwen3-32BFine Tuning Type=NTP, Prompting Setting=3-shot, Generation Mode=structured, k=52025.12 | 74.93 | 74.89 | 71.13 | |
| Qwen2.5-Coder-32BFine Tuning Type=NTP + Masking, Prompting Setting=3-shot, Generation Mode=structured, k=52025.12 | 70.08 | 70.04 | 66.32 | |
| phi-4Fine Tuning Type=NTP, Prompting Setting=3-shot, Generation Mode=structured, k=52025.12 | 69.7 | 69.58 | 66.36 | |
| Qwen3-4BFine Tuning Type=NTP, Prompting Setting=3-shot, Generation Mode=structured, k=52025.12 | 68.87 | 68.16 | 64.48 | |
| Qwen2.5-Coder-7BFine Tuning Type=NTP, Prompting Setting=3-shot, Generation Mode=structured, k=52025.12 | 65.9 | 65.9 | 62.97 |