Panoptic video scene graph generation on OpenPVSG v1 (test)
11.43Recall@20Ours - Convolution (IPS+T Baseline)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Ours - Convolution (IPS+T Baseline)vIoU threshold=0.12026.01 | 11.43 | 9.57 | 17.3 | 13.13 | 22.85 | 17.48 | |
| Motion-aware Contrastive Learning (Convolution)vIoU threshold=0.1, Segmentation Module=IPS+T2024.12 | 11.43 | 9.57 | 17.3 | 13.13 | 22.85 | 17.48 | |
| Ours - Transformer (IPS+T Baseline)vIoU threshold=0.12026.01 | 10.59 | 9.56 | 16.98 | 12.39 | 22.33 | 17.47 | |
| Motion-aware Contrastive Learning (Transformer)vIoU threshold=0.1, Segmentation Module=IPS+T2024.12 | 10.59 | 9.56 | 16.98 | 12.39 | 22.33 | 17.47 | |
| IPS+T - ConvolutionvIoU threshold=0.12026.01 | 10.06 | 8.98 | 14.99 | 12.21 | 18.13 | 15.47 | |
| IPS+T - ConvolutionvIoU threshold=0.1, Segmentation Module=IPS+T2024.12 | 10.06 | 8.98 | 14.99 | 12.21 | 18.13 | 15.47 | |
| IPS+T - TransformervIoU threshold=0.12026.01 | 9.01 | 6.69 | 14.88 | 11.28 | 17.51 | 13.2 | |
| IPS+T - TransformervIoU threshold=0.1, Segmentation Module=IPS+T2024.12 | 9.01 | 6.69 | 14.88 | 11.28 | 17.51 | 13.2 | |
| IPS+T - VanillavIoU threshold=0.12026.01 | 8.28 | 5.68 | 14.47 | 9.92 | 18.24 | 11.84 | |
| IPS+T - VanillavIoU threshold=0.1, Segmentation Module=IPS+T2024.12 | 8.28 | 5.68 | 14.47 | 9.92 | 18.24 | 11.84 | |
| Ours - Convolution (VPS Baseline)vIoU threshold=0.12026.01 | 8.18 | 8 | 12.9 | 11.47 | 14.22 | 13.59 | |
| Motion-aware Contrastive Learning (Convolution)vIoU threshold=0.1, Segmentation Module=VPS2024.12 | 8.18 | 8 | 12.9 | 11.47 | 14.22 | 13.59 | |
| IPS+T - Handcrafted filtervIoU threshold=0.12026.01 | 8.07 | 5.61 | 13.42 | 8.27 | 16.46 | 10.11 | |
| VPS - ConvolutionvIoU threshold=0.12026.01 | 8.07 | 7.84 | 11.01 | 9.78 | 12.89 | 10.77 | |
| IPS+T - Handcrafted filtervIoU threshold=0.1, Segmentation Module=IPS+T2024.12 | 8.07 | 5.61 | 13.42 | 8.27 | 16.46 | 10.11 | |
| VPS - ConvolutionvIoU threshold=0.1, Segmentation Module=VPS2024.12 | 8.07 | 7.84 | 11.01 | 9.78 | 12.89 | 10.77 | |
| Ours - Transformer (VPS Baseline)vIoU threshold=0.12026.01 | 6.71 | 6.94 | 10.27 | 8.68 | 13.42 | 12.09 | |
| Motion-aware Contrastive Learning (Transformer)vIoU threshold=0.1, Segmentation Module=VPS2024.12 | 6.71 | 6.94 | 10.27 | 8.68 | 13.42 | 12.09 | |
| VPS - TransformervIoU threshold=0.12026.01 | 6.5 | 5.75 | 9.64 | 8.25 | 12.26 | 9.51 | |
| VPS - TransformervIoU threshold=0.1, Segmentation Module=VPS2024.12 | 6.5 | 5.75 | 9.64 | 8.25 | 12.26 | 9.51 | |
| VPS - VanillavIoU threshold=0.12026.01 | 6.29 | 3.04 | 9.64 | 6.74 | 12.89 | 9.6 | |
| VPS - VanillavIoU threshold=0.1, Segmentation Module=VPS2024.12 | 6.29 | 3.04 | 9.64 | 6.74 | 12.89 | 9.6 | |
| VPS - Handcrafted filtervIoU threshold=0.12026.01 | 5.24 | 2.84 | 7.65 | 7.14 | 9.64 | 8.22 | |
| VPS - Handcrafted filtervIoU threshold=0.1, Segmentation Module=VPS2024.12 | 5.24 | 2.84 | 7.65 | 7.14 | 9.64 | 8.22 | |
| Ours - Convolution (IPS+T Baseline)vIoU threshold=0.52026.01 | 4.51 | 3.56 | 6.08 | 4.38 | 7.76 | 5.86 | |
| Motion-aware Contrastive Learning (Convolution)vIoU threshold=0.5, Segmentation Module=IPS+T2024.12 | 4.51 | 3.56 | 6.08 | 4.38 | 7.76 | 5.86 | |
| Ours - Transformer (IPS+T Baseline)vIoU threshold=0.52026.01 | 3.98 | 2.98 | 5.97 | 4.2 | 7.44 | 5.15 | |
| Motion-aware Contrastive Learning (Transformer)vIoU threshold=0.5, Segmentation Module=IPS+T2024.12 | 3.98 | 2.98 | 5.97 | 4.2 | 7.44 | 5.15 | |
| IPS+T - TransformervIoU threshold=0.52026.01 | 3.88 | 2.81 | 5.66 | 4.12 | 6.18 | 4.44 | |
| IPS+T - ConvolutionvIoU threshold=0.52026.01 | 3.88 | 2.55 | 5.24 | 3.29 | 6.71 | 5.36 | |
| IPS+T - TransformervIoU threshold=0.5, Segmentation Module=IPS+T2024.12 | 3.88 | 2.81 | 5.66 | 4.12 | 6.18 | 4.44 | |
| IPS+T - ConvolutionvIoU threshold=0.5, Segmentation Module=IPS+T2024.12 | 3.88 | 2.55 | 5.24 | 3.29 | 6.71 | 5.36 | |
| IPS+T - VanillavIoU threshold=0.52026.01 | 3.04 | 1.35 | 4.61 | 2.94 | 5.56 | 3.33 | |
| IPS+T - VanillavIoU threshold=0.5, Segmentation Module=IPS+T2024.12 | 3.04 | 1.35 | 4.61 | 2.94 | 5.56 | 3.33 | |
| IPS+T - Handcrafted filtervIoU threshold=0.52026.01 | 2.52 | 1.72 | 3.77 | 2.36 | 4.72 | 2.79 | |
| IPS+T - Handcrafted filtervIoU threshold=0.5, Segmentation Module=IPS+T2024.12 | 2.52 | 1.72 | 3.77 | 2.36 | 4.72 | 2.79 | |
| Ours - Convolution (VPS Baseline)vIoU threshold=0.52026.01 | 0.84 | 0.98 | 1.26 | 1.22 | 1.26 | 1.22 | |
| Motion-aware Contrastive Learning (Convolution)vIoU threshold=0.5, Segmentation Module=VPS2024.12 | 0.84 | 0.98 | 1.26 | 1.22 | 1.26 | 1.22 | |
| Ours - Transformer (VPS Baseline)vIoU threshold=0.52026.01 | 0.63 | 0.83 | 1.05 | 0.76 | 1.05 | 0.76 | |
| Motion-aware Contrastive Learning (Transformer)vIoU threshold=0.5, Segmentation Module=VPS2024.12 | 0.63 | 0.83 | 1.05 | 0.76 | 1.05 | 0.76 | |
| VPS - Handcrafted filtervIoU threshold=0.52026.01 | 0.42 | 0.13 | 0.52 | 0.5 | 0.94 | 0.92 | |
| VPS - TransformervIoU threshold=0.52026.01 | 0.42 | 0.61 | 0.73 | 0.76 | 1.05 | 0.92 | |
| VPS - ConvolutionvIoU threshold=0.52026.01 | 0.42 | 0.25 | 0.63 | 0.67 | 0.63 | 0.67 | |
| VPS - Handcrafted filtervIoU threshold=0.5, Segmentation Module=VPS2024.12 | 0.42 | 0.13 | 0.52 | 0.5 | 0.94 | 0.92 | |
| VPS - TransformervIoU threshold=0.5, Segmentation Module=VPS2024.12 | 0.42 | 0.61 | 0.73 | 0.76 | 1.05 | 0.92 | |
| VPS - ConvolutionvIoU threshold=0.5, Segmentation Module=VPS2024.12 | 0.42 | 0.25 | 0.63 | 0.67 | 0.63 | 0.67 | |
| VPS - VanillavIoU threshold=0.52026.01 | 0.21 | 0.1 | 0.21 | 0.1 | 0.31 | 0.18 | |
| VPS - VanillavIoU threshold=0.5, Segmentation Module=VPS2024.12 | 0.21 | 0.1 | 0.21 | 0.1 | 0.31 | 0.18 |