3D Instance Segmentation on ScanNet hidden v2 (test)

71.6Cabinet AP@0.5SoftGroup

Evaluation Results

Method
SoftGroup2022.03	71.6	76.1	80.8	86.2	84.4	76.9	62	59.4	84.5	69.9	24.3	65.5	82.4	79.1	—	100	71.6	100	73.4	—	—	98.1	—
HAIS2022.03	67.5	69.9	84.9	80.8	76.7	67.6	51.7	56	82	55.9	27.9	46.5	75.7	60	—	99.4	65.4	100	59.6	—	—	100	—
OccuSeg2022.03	57.6	67.2	75.8	84.2	79.7	56.3	56.7	46.7	68.2	45.1	47.7	52.4	50.4	55.7	—	100	75.1	100	58.5	—	—	100	—
SSTNet2022.03	55.6	69.8	69.7	80.3	72	69.2	55.6	50.9	88.8	70.2	38.7	41.7	62.6	60	—	100	82.4	100	58.5	—	—	100	—
PE2022.03	53.8	64.5	77.3	78.6	76.1	55.6	43.5	50.1	79.8	54.5	8.8	35	79.9	64.6	—	99.7	56.2	100	54.7	—	—	93.3	—
Dyco3D2022.03	53.1	64.1	84.1	80.2	76.4	65.7	43.8	56.8	89.3	43	11.5	44.8	58.8	55	—	98.7	53.4	100	53.7	—	—	85.7	—
3D-MPA2022.03	52.6	61.1	83.3	75.6	76.5	55.7	43.8	43	76.5	35.8	13.6	47	58.8	65	—	100	42.9	100	43.2	—	—	85.7	—
PointGroup2022.03	50.5	63.6	76.5	79.7	75.6	55.6	44.1	51.3	62.4	47.6	11.6	38.4	69.6	59.6	—	99.7	66.6	100	55.9	—	—	100	—
QueryFormer2023.12	50.4	78.7	70.2	73.3	74.5	59.2	47.9	36.3	39.3	53.3	27.6	37.3	52.7	69.7	—	95.8	43.6	92.6	53.4	58.3	—	72	—
GICN2022.03	48	63.8	89.5	67.6	83.6	59.9	44.7	47.3	80	36.5	14.4	35.4	73.7	70	—	100	56.9	100	40	—	—	100	—
PBNet2023.12	47.2	74.7	57.5	73.6	71.7	55.9	45.9	38.1	61.9	53.3	23.9	38.3	48.7	58.5	—	96.9	40.4	92.6	50.6	57.3	—	76.7	—
PBNet2022.07	47.2	—	57.5	73.6	71.7	55.9	45.9	38.1	61.9	53.3	23.9	38.3	48.7	58.5	—	96.9	40.4	92.6	50.6	57.3	—	76.7	—
MAFT2023.12	46	78.6	72.1	76.8	68.4	55.1	50.4	45	44.8	61.6	25.1	40.8	55.8	61.8	—	93.1	48.2	88.9	53.9	57.8	—	85.8	—
Spherical Mask2023.12	43.4	81.2	65.4	76.9	74.7	57.5	50.5	43.6	55.5	69.8	27.1	44.7	60.4	71.6	—	92.5	48	94.6	54.9	61.6	—	77.5	—
ISBNet2023.12	42.6	75.7	65.5	76.3	64.9	57.1	49.9	40.1	38.3	62.1	18	38.6	53.4	42.7	—	94.8	46.7	93.9	50.9	55.9	—	70.4	—
SPFormer2022.11	39.5	—	64	73.9	71.2	54	46.8	34.3	48.4	55.5	31.1	33.5	56.6	47.8	—	89.3	43.6	74.5	49.2	54.9	—	74.7	—
MASC2019.03	38.2	44.7	55.5	63.3	63.9	38.6	36.1	27.6	38.1	32.7	0.2	26	50.9	45.1	57.1	98	36.7	52.8	43.2	—	—	—	—
MASC2022.03	38.2	44.7	55.5	63.3	63.9	38.6	36.1	27.6	38.1	32.7	0.2	26	50.9	45.1	—	98	36.7	52.8	43.2	—	—	57.1	—
MASC2019.06	38.2	44.7	55.5	63.3	63.9	38.6	36.1	27.6	38.1	32.7	0.2	26	50.9	45.1	—	98	36.7	52.8	43.2	—	—	57.1	—
SoftGroup2022.11	38.1	—	57.9	69.4	64.3	49.2	38.7	38.8	37.2	31.9	7.2	30.3	67.7	58.2	—	90.7	31.8	66.7	53.1	50.4	—	75.4	—
SoftGroup2023.12	38.1	76.1	57.9	69.4	64.3	49.2	38.7	38.8	37.2	31.9	7.2	30.3	67.7	58.2	—	90.7	31.8	66.7	53.1	50.4	—	75.4	—
SoftGroup2022.07	38.1	—	57.9	69.4	64.3	49.2	38.7	38.8	37.2	31.9	7.2	30.3	67.7	58.2	—	90.7	31.8	66.7	53.1	50.4	—	75.4	—
DKNet2023.12	37.7	71.8	62.4	74.9	64	50.6	43.7	38.5	51.7	58.1	10.7	30.4	50.9	53.9	—	90.1	33.9	81.5	47.5	53.2	—	77.5	—
DKNet2022.07	37.7	—	62.4	74.9	64	50.6	43.7	38.5	51.7	58.1	10.7	30.4	50.9	53.9	—	90.1	33.9	81.5	47.5	53.2	—	77.5	—
OccuSeg2022.11	36.9	—	53.6	70.2	60.1	39.4	37.9	27.3	42.8	32.7	20.5	30.1	33.1	43.7	—	84.6	48.5	80.2	47.4	48.6	—	86.2	—
OCCuSeg2022.07	36.9	—	53.6	70.2	60.1	39.4	37.9	27.3	42.8	32.7	20.5	30.1	33.1	43.7	—	84.6	48.5	80.2	47.4	48.6	—	86.2	—
HAIS2022.11	36.4	—	56.1	67.3	56.3	43.4	30.8	34.4	45.7	28.8	4.6	19.4	54.7	45.4	—	88.9	26.2	70.4	42.6	45.7	—	71.1	—
HAIS2022.07	36.4	—	56.1	67.3	56.3	43.4	30.8	34.4	45.7	28.8	4.6	19.4	54.7	45.4	—	88.9	26.2	70.4	42.6	45.7	—	71.1	—
PointInst3D2022.07	35.5	—	50.7	70.3	55.3	44.2	31.3	29.3	33.8	28.8	8.9	20.8	39	40.1	—	91.3	24.2	81.5	37.3	43.8	—	66.6	—
GSPN2019.03	34.8	30.6	40.5	58.9	39.6	27.5	28.3	24.5	31.1	2.8	5.4	12.6	6.8	21.9	21.4	82.1	33.1	50	29	—	—	—	—
GSPN2022.03	34.8	30.6	40.5	58.9	39.6	27.5	28.3	24.5	31.1	2.8	5.4	12.6	6.8	21.9	—	82.1	33.1	50	29	—	—	21.4	—
R-PointNet2019.06	34.8	30.6	40.5	58.9	39.6	27.5	28.3	24.5	31.1	2.8	5.4	12.6	6.8	21.9	—	82.1	33.1	50	29	—	—	21.4	—
MaskGroup2022.07	33	—	51.6	65.8	57.5	42.5	25.6	36.2	47.1	30.9	2.9	24.9	52.6	38.4	—	87.7	36.8	77.8	40	43.4	—	29.6	—
MTML2022.03	32.7	54.9	80.7	64.7	68.8	57.1	41.8	39.6	58.8	18.2	0.4	18	81.5	44.5	—	100	44.2	100	36.4	—	—	100	—
TD3D2023.12	32.2	75.1	51.1	73.5	67.1	51	34.9	32.9	43.4	28.3	10.1	35.5	51.2	51.4	—	90.8	26.8	85.2	46.8	48.9	—	67.6	—
SSTNet2022.11	31.6	—	54.9	69.3	63.7	45.7	33	29	49.7	57.6	17.8	19.8	37.7	51.5	—	94.3	49.4	73.8	46.3	50.6	—	85.7	—
SSTNet2022.07	31.6	—	54.9	69.3	63.7	45.7	33	29	49.7	57.6	17.8	19.8	37.7	51.5	—	94.3	49.4	73.8	46.3	50.6	—	85.7	—
3D-Bonet2022.03	30.1	48.8	67.2	48.4	49.9	51.3	34.1	43.9	59	12.5	9.8	30.6	62	43.4	—	90.9	40.2	100	25.9	—	—	79.6	—
3D-BoNet2019.06	30.1	48.8	67.2	48.4	49.9	51.3	34.1	43.9	59	12.5	9.8	30.6	62	43.4	—	90.9	40.2	100	25.9	—	—	79.6	—
3D-MPA2022.11	27.7	—	48.4	59.1	57.4	35.2	21.7	21.3	29.9	19.3	4.7	21.2	33.2	41.3	—	84.9	19.5	45.7	27.8	35.5	—	41	—
3D-MPA2022.07	27.7	—	48.4	59.1	57.4	35.2	21.7	21.3	29.9	19.3	4.7	21.2	33.2	41.3	—	84.9	19.5	45.7	27.8	35.5	—	41	—
MTML2019.06	27.2	48.1	66.6	70.9	60.1	48.7	36.1	38.4	37.7	9.5	0.1	25.4	57.9	43.2	—	93.8	18.4	100	31.8	—	—	100	—
PanopticFusion2019.03	25.9	47.8	71.2	55	59.1	26.7	25	35.9	59.5	43.7	0	17.5	61.3	41.1	85.7	94.4	48.5	66.7	43.4	—	—	—	—
PanopticFusion2022.03	25.9	47.8	71.2	55	59.1	26.7	25	35.9	59.5	43.7	0	17.5	61.3	41.1	—	94.4	48.5	66.7	43.4	—	—	85.7	—
Dyco3D2022.11	25.9	—	51.8	66.6	58.7	43.8	23.1	31.7	44.7	23.2	5	16.6	25.1	33.1	—	85	22.9	64.2	36.2	39.5	—	53.5	—
ResNet-Backbone2019.06	25.9	45.9	73.7	58.7	53.6	59	41.6	30.4	15.9	12.8	13.8	21.7	47.5	31.5	—	87.3	41.1	100	40.8	—	—	71.4	—
PanopticFusion2019.06	25.9	47.8	71.2	55	59.1	26.7	25	35.9	59.5	43.7	0	17.5	61.3	41.1	—	94.4	48.5	66.7	43.4	—	—	85.7	—
Dyco3d2022.07	25.9	—	51.8	66.6	58.7	43.8	23.1	31.7	44.7	23.2	5	16.6	25.1	33.1	—	85	22.9	64.2	36.2	39.5	—	53.5	—
RPGN2022.07	24.9	—	50.8	65.8	60.9	40.6	23.4	26.7	36.7	27	1.6	13.1	67.3	43.4	—	84.2	27.4	63	38.3	42.8	—	74.8	—
PE2022.11	24.3	—	46.7	62.4	54.1	35	21.9	27.3	44.6	23.9	2.2	10.6	57.7	48.7	—	81.8	22.5	66.7	34	39.6	—	47.5	—
PointGroup2022.11	24.3	—	49.6	64.5	56.2	34.1	21.1	29.1	41.5	21.7	2.1	11.4	57	42.8	—	86	25.6	63.9	35.9	40.7	—	66.6	—
PointGroup2022.07	24.3	—	49.6	64.5	56.2	34.1	21.1	29.1	41.5	21.7	2.1	11.4	57	42.8	—	86	25.6	63.9	35.9	40.7	—	66	—
PE2022.07	24.3	—	46.7	62.4	54.1	35	21.9	27.3	44.6	23.9	2.2	10.6	57.7	48.7	—	81.8	22.5	66.7	34	39.6	—	47.5	—
GICN2022.11	19.8	—	37.1	46.9	52.5	32.9	21.2	24.1	34.4	12.7	5.2	9.3	56.4	34.7	—	72.9	20.6	58	21.2	34.1	—	53.7	—
3D-SIS2019.03	19	38.2	43.2	57.7	69.9	27.1	32	23.5	24.5	7.5	1.3	3.3	26.3	42.2	85.7	88.3	11.7	100	24	—	—	—	—
3D-SIS2022.03	19	38.2	43.2	57.7	69.9	27.1	32	23.5	24.5	7.5	1.3	3.3	26.3	42.2	—	88.3	11.7	100	24	—	—	85.7	—
3D-SIS2019.06	19	38.2	43.2	57.7	69.9	27.1	32	23.5	24.5	7.5	1.3	3.3	26.3	42.2	—	88.3	11.7	100	24	—	—	85.7	—
UNet-Backbone2019.06	18.9	31.9	71.5	47.9	61.5	35.5	20.1	9.3	23.3	10.7	0.8	6.7	21.8	12.3	—	91.6	15	66.7	17.3	—	—	43.8	—
GSPN2022.07	14	—	17.3	35.9	22.1	12.8	13.4	9.4	11.3	0.8	1.2	3.9	2.3	8.9	—	56.3	11.7	35.6	12.3	15.8	—	14.9	—
3D-BoNet2022.11	13.7	—	32.4	34.5	30.1	30.3	16.2	17.8	25.1	5.2	3.1	6.9	41.9	20.2	—	65.1	14.7	51.9	13.1	25.3	—	33.8	—
3D-BoNet2023.12	13.7	48.8	32.4	34.5	30.1	30.3	16.2	17.8	25.1	5.2	3.1	6.9	41.9	20.2	—	65.1	14.7	51.9	13.1	25.3	—	33.8	—
3D-Bonet2022.07	13.7	—	32.4	34.5	30.1	30.3	16.2	17.8	25.1	5.2	3.1	6.9	41.9	20.2	—	65.1	14.7	51.9	13.1	25.3	—	33.8	—
MTML2022.11	10.7	—	38	43	49.1	31.3	17.9	16.2	18.2	7	0.1	5.7	42.2	22.9	—	65	16.1	57.7	16.2	28.2	—	51.1	—
SGPN2019.03	6.5	14.3	39	27.5	35.1	16.8	8.7	13.8	16.9	1.4	2.9	0	6.9	2.7	0	43.8	11.2	20.8	4.3	—	—	—	—
SGPN2022.03	6.5	14.3	39	27.5	35.1	16.8	8.7	13.8	16.9	1.4	2.9	0	6.9	2.7	—	43.8	11.2	20.8	4.3	—	—	0	—
SGPN2019.06	6.5	14.3	39	27.5	35.1	16.8	8.7	13.8	16.9	1.4	2.9	0	6.9	2.7	—	43.8	11.2	20.8	4.3	—	—	0	—
MaskRCNN2019.06	5.3	5.8	0.2	0.2	10.7	2	4.5	0.6	0	23.8	0.2	0	2.1	6.5	—	11	1.4	33.3	2.4	—	—	0	—
3D-BEVIS2019.06	3.5	24.8	56.6	39.4	60.4	18.1	9.9	17.1	7.6	2.5	2.7	9.8	3.5	9.8	—	85.4	12.6	66.7	3	—	—	37.5	—
SGPN2022.07	1.3	—	13.4	14.4	12.2	9.5	2.8	5.4	3.1	0.3	0.6	0	0.8	0.9	—	17.5	2.1	2.3	1.7	4.9	—	0	—
3D-BEVIS2020.03	—	24.8	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	11.7	40.1	—	—
3D-BoNet2020.03	—	48.8	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	25.3	68.7	—	—
3D-BoNet2023.03	—	25.3	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	68.7
3D-MPA2020.03	—	61.1	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	35.5	73.7	—	—
3D-SIS2020.03	—	38.2	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	16.1	55.8	—	—
Di&Co3D2023.03	—	47.7	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	80.2
DKNet2023.03	—	53.2	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	81.5
DyCo3D2023.03	—	39.5	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	76.1
GSPN2020.03	—	30.6	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	15.8	54.4	—	—
HAIS2023.03	—	45.7	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	80.3
ISBNet2023.03	—	55.9	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	84.5
MTML2020.03	—	54.9	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	28.2	73.1	—	—
MTML2023.03	—	28.2	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	73.1
OccuSeg2023.03	—	44.3	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	74.2
PE2023.03	—	39.6	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	77.6
PointGroup2023.03	—	40.7	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	77.8
PointInst3D2023.03	—	43.8	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—
RPGN2023.03	—	42.8	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	80.6
SGPN2020.03	—	14.3	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	4.9	39	—	—
SGPN2023.03	—	4.9	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	26.1
SoftGroup2023.03	—	50.4	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	86.5
SSTNet2023.03	—	50.6	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	78.9