Layout Generation on MesaTask-CTRC benchmark 1.0
46.5Quality Pass Rate (τ=7)PhyScene3D
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PhyScene3DBackbone=Qwen-3 VL 8B, Stages=SFT + Physically-Aware RL (GRPO)2026.06 | 46.5 | 39.1 | 27.3 | 7.85 | 9.45 | 9.91 | 8.29 | 9.15 | 8.93 | 41.6 | 3.86 | |
| GPT-4oParadigm=Zero-shot Generation, Source=Closed-source LLM2026.06 | 27.6 | 22.1 | 12.7 | 6.95 | 8.79 | 9.54 | 7.48 | 8.18 | 8.19 | 68.9 | 7.87 | |
| MesaTaskParadigm=End-to-End Model, Type=Supervised Baseline2026.06 | 21.1 | 18.6 | 14.8 | 7.78 | 9.23 | 9.78 | 8.15 | 9.05 | 8.8 | 78.3 | 8.19 | |
| I-Design-tableParadigm=Agent-Based Solver2026.06 | 19 | 9.7 | 4.5 | 4.39 | 8.2 | 9.2 | 5.34 | 5.55 | 6.53 | 39.1 | 5.94 | |
| GPT-4o-miniParadigm=Zero-shot Generation, Source=Closed-source LLM2026.06 | 17.9 | 13.4 | 8.8 | 6.18 | 8.45 | 9.15 | 6.82 | 7.34 | 7.59 | 80 | 13.61 | |
| ReferenceDescription=Samples drawn from MesaTask-10k2026.06 | 17.1 | 15.1 | 11.5 | 7.83 | 9.25 | 9.52 | 8.25 | 9.18 | 8.87 | 81.5 | 8.19 | |
| Holodeck-tableParadigm=Agent-Based Solver2026.06 | 2.7 | 0.6 | 0 | 2.35 | 6.21 | 7.38 | 3.61 | 3.43 | 4.6 | 2.7 | 0.47 |