3D Scene Graph Generation on MA3DSG-Bench SCP setting 1.0 (test)
26.4Triplet Recall@1SGFN
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SGFNApproach=Single-agent, Domain Size (# of rooms)=472026.02 | 26.4 | 9.6 | 14.1 | 56.2 | 29.4 | 38.6 | 32 | 25.8 | 28.6 | — | — | 61.8 | |
| MA3DSGApproach=Multi-agent, Domain Size (# of rooms)=252026.02 | 25.7 | 11.8 | 16.2 | 54.9 | 27.2 | 36.4 | 31.2 | 28.5 | 29.8 | 1.7 | 0.01 | 8.2 | |
| SGFNApproach=Single-agent, Domain Size (# of rooms)=252026.02 | 24.3 | 10.5 | 14.7 | 55.8 | 30.7 | 39.6 | 30.5 | 27.7 | 29 | — | — | 32.7 | |
| MA3DSGApproach=Multi-agent, Domain Size (# of rooms)=472026.02 | 24.2 | 9.5 | 13.7 | 55.8 | 25.6 | 35.1 | 27.7 | 22.2 | 24.6 | 3.7 | 0.02 | 14.8 | |
| SGFN + SGAlignerApproach=Multi-agent, Domain Size (# of rooms)=472026.02 | 22.5 | 8.8 | 12.7 | 50 | 31.3 | 38.5 | 27.5 | 24.6 | 25.9 | 364.2 | 107.1 | 16.6 | |
| SGFN + SG-PGMApproach=Multi-agent, Domain Size (# of rooms)=472026.02 | 22.5 | 8.8 | 12.6 | 50 | 31 | 38.3 | 28.1 | 25.4 | 26.7 | 364.1 | 32.1 | 15.3 | |
| MA3DSGApproach=Multi-agent, Domain Size (# of rooms)=52026.02 | 21.1 | 7.9 | 11.5 | 47.9 | 30.7 | 37.4 | 25.9 | 24 | 24.9 | 0.3 | 0 | 2.3 | |
| SGFNApproach=Single-agent, Domain Size (# of rooms)=152026.02 | 20.4 | 9.5 | 13 | 51.7 | 30.7 | 38.5 | 25.7 | 26.3 | 26 | — | — | 17.1 | |
| SGFNApproach=Single-agent, Domain Size (# of rooms)=52026.02 | 19.6 | 7.6 | 10.9 | 52.1 | 33.1 | 40.5 | 25.9 | 23.4 | 24.6 | — | — | 6.4 | |
| 3DSSGApproach=Single-agent, Domain Size (# of rooms)=252026.02 | 18.6 | 15.5 | 16.9 | 32.6 | 36.5 | 34.4 | 29.7 | 37.9 | 33.3 | — | — | — | |
| MA3DSGApproach=Multi-agent, Domain Size (# of rooms)=152026.02 | 17.3 | 8.9 | 11.7 | 51.4 | 25.5 | 34.1 | 21.1 | 22.1 | 21.6 | 1 | 0.01 | 4.3 | |
| 3DSSGApproach=Single-agent, Domain Size (# of rooms)=152026.02 | 16.8 | 15.5 | 16.1 | 31.5 | 37.7 | 34.4 | 26.3 | 39.2 | 31.5 | — | — | — | |
| SGFN + SGAlignerApproach=Multi-agent, Domain Size (# of rooms)=252026.02 | 16.2 | 6.9 | 9.7 | 51.1 | 31.7 | 39.1 | 20.9 | 20.2 | 20.5 | 181.7 | 56.5 | 9.1 | |
| SGFN + SG-PGMApproach=Multi-agent, Domain Size (# of rooms)=252026.02 | 16.2 | 6.9 | 9.7 | 51.3 | 31 | 38.6 | 21.2 | 20.4 | 20.8 | 181.6 | 16.9 | 8.5 | |
| 3DSSGApproach=Single-agent, Domain Size (# of rooms)=472026.02 | 14.8 | 9 | 11.2 | 34 | 33.4 | 33.7 | 25.9 | 26.8 | 26.3 | — | — | — | |
| SGFN + SGAlignerApproach=Multi-agent, Domain Size (# of rooms)=52026.02 | 14.7 | 5.4 | 7.9 | 50 | 34.3 | 40.7 | 18.9 | 16.4 | 17.6 | 43.2 | 11.7 | 2.5 | |
| SGFN + SG-PGMApproach=Multi-agent, Domain Size (# of rooms)=52026.02 | 13.8 | 5.2 | 7.5 | 52.1 | 33.8 | 41 | 19.6 | 17.3 | 18.4 | 43.1 | 3.5 | 2.4 | |
| SGFN + SGAlignerApproach=Multi-agent, Domain Size (# of rooms)=152026.02 | 10.8 | 4.7 | 6.6 | 46.1 | 30.5 | 36.7 | 14.6 | 15.8 | 15.2 | 116.2 | 35.1 | 4.9 | |
| SGFN + SG-PGMApproach=Multi-agent, Domain Size (# of rooms)=152026.02 | 10.6 | 4.6 | 6.5 | 46.7 | 30.4 | 36.8 | 14.8 | 16 | 15.4 | 116.1 | 10.5 | 4.5 | |
| 3DSSGApproach=Single-agent, Domain Size (# of rooms)=52026.02 | 3.1 | 3.2 | 3.1 | 27.1 | 32.5 | 29.5 | 15.2 | 19.5 | 17 | — | — | — |