Pairwise Scene Generation Evaluation on Procthor-10K (Easy)
75.6Score (SceneCritic) Method Aheuristic vs holodeck
Evaluation results
| Method | Links | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| heuristic vs holodeckMethod A=heuristic, Method B=holodeck2026.04 | 75.6 | 60 | 71.3 | 25 | 56.2 | 43.7 | 75 | 25 | 68.8 | 31.2 | 62.5 | 75 | 68.8 | -0.098 | 0.126 | -0.134 | — | — | — | — | — | — | |
| heuristic vs layoutgptMethod A=heuristic, Method B=layoutgpt2026.04 | 75.6 | 60 | 53.6 | 100 | 93.8 | 6.2 | 87.5 | 12.5 | 100 | 0 | 93.8 | 87.5 | 100 | 0.739 | 0.49 | 1 | — | — | — | — | — | — | |
| heuristic vs layoutvlmMethod A=heuristic, Method B=layoutvlm2026.04 | 75.6 | 60 | 81.9 | 60 | 75 | 25 | 87.5 | 12.5 | 68.8 | 31.2 | 75 | 87.5 | 68.8 | 0.164 | 0.49 | -0.134 | — | — | — | — | — | — | |
| holodeck vs layoutgptMethod A=holodeck, Method B=layoutgpt2026.04 | 71.3 | 25 | 53.6 | 100 | 87.5 | 12.5 | 93.8 | 6.2 | 93.8 | 6.2 | 87.5 | 93.8 | 93.8 | 0.512 | 0.727 | 0.691 | — | — | — | — | — | — | |
| holodeck vs layoutvlmMethod A=holodeck, Method B=layoutvlm2026.04 | 71.3 | 25 | 81.9 | 60 | 37.5 | 62.5 | 37.5 | 62.5 | 87.5 | 12.5 | 62.5 | 62.5 | 87.5 | -0.045 | -0.093 | 0.423 | — | — | — | — | — | — | |
| layoutgpt vs layoutvlmMethod A=layoutgpt, Method B=layoutvlm2026.04 | 53.6 | 100 | 81.9 | 60 | 12.5 | 87.5 | 6.2 | 93.8 | 12.5 | 87.5 | 87.5 | 93.8 | 87.5 | 0.512 | 0.727 | 0.423 | — | — | — | — | — | — |