Procedure Planning on CrossTask T=3 (test)
59.35SRCLAD
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CLADUse ground truth a0 and at=false2025.03 | 59.35 | 84.56 | — | 80.61 | |
| PDPPUse ground truth a0 and at=true2025.03 | 50.94 | 83.65 | — | 79.44 | |
| KEPPUse ground truth a0 and at=true2025.03 | 47.98 | 82.66 | — | 77.94 | |
| SCHEMAUse ground truth a0 and at=true2025.03 | 47.06 | 82.35 | — | — | |
| ActionDiffusionUse ground truth a0 and at=true2025.03 | 46.35 | 82.12 | — | 77.48 | |
| LAPData curation setting=KEPP setting2026.03 | 41.14 | 70.13 | — | 65.98 | |
| ViterbiPlanNetParams (M)=5.572026.03 | 38.45 | 63.07 | 83.89 | — | |
| SCHEMAParams (M)=6.132026.03 | 37.24 | 62.69 | 83.94 | — | |
| PlanLLMParams (M)=384.942026.03 | 36.84 | 61.56 | 83.23 | — | |
| PDPPParams (M)=41.872026.03 | 36.73 | 61.96 | 83.2 | — | |
| LAP-TextData curation setting=KEPP setting, Input modality=text description2026.03 | 35.84 | 65.65 | — | 62.48 | |
| KEPPParams (M)=42.182026.03 | 34.93 | 60.34 | 82.67 | — | |
| PDPPData curation setting=KEPP setting2026.03 | 33.9 | 61.39 | — | — | |
| KEPPData curation setting=KEPP setting2026.03 | 33.27 | 60.16 | — | — | |
| ActionDiffusionData curation setting=KEPP setting2026.03 | 33.25 | 61.47 | — | 65.09 | |
| SCHEMAData curation setting=KEPP setting2026.03 | 32.34 | 57.3 | — | — | |
| Gemini 2.5 ProParams (M)=>100,0002026.03 | 29.18 | 57.9 | 81.48 | — | |
| SkipPlanData curation setting=KEPP setting2026.03 | 27.6 | 59.9 | — | — | |
| Qwen2.5-32BParams (M)=32,0002026.03 | 25.14 | 56.1 | 80.92 | — | |
| Qwen3-30BParams (M)=30,0002026.03 | 23.37 | 55.96 | 81.16 | — | |
| P3IVSupervision=L2022.05 | 23.34 | 49.96 | 73.89 | — | |
| Qwen3-30B + PKGParams (M)=30,000, PKG=true2026.03 | 23.31 | 56.15 | 81.06 | — | |
| PKG beam searchParams (M)=41.872026.03 | 22.38 | 55.74 | 80.92 | — | |
| Ext-GAILSupervision=V2022.05 | 21.27 | 49.46 | 61.7 | — | |
| Ext-GAIL w/o Aug.Supervision=V2022.05 | 18.01 | 43.86 | 57.16 | — | |
| DDNSupervision=V2022.05 | 12.18 | 31.29 | 47.48 | — | |
| Qwen2.5-VL-32BParams (M)=32,0002026.03 | 11.48 | 36.35 | 69.52 | — | |
| Retrieval-BasedSupervision=-2022.05 | 8.05 | 23.3 | 32.06 | — | |
| UPNSupervision=V2022.05 | 2.89 | 24.39 | 31.56 | — | |
| UAAASupervision=-2022.05 | 2.15 | 20.21 | 30.87 | — | |
| WLTDOSupervision=-2022.05 | 1.87 | 21.64 | 31.7 | — | |
| RandomSupervision=-2022.05 | 0.01 | 0.94 | 1.66 | — | |
| RandomUse ground truth a0 and at=true2025.03 | 0 | 1.02 | — | 1.7 |