3D Dense Captioning on Scan2Cap (val)
77.19CIDEr (@0.5)Chat-Scene
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Chat-SceneVenue=null2023.12 | 77.19 | — | — | — | — | 81.94 | 38.23 | 29.01 | 60.57 | 36.34 | 28.01 | 58.12 | |
| LEOVenue=ICML242023.12 | 68.4 | — | — | — | — | — | — | — | — | 36.9 | 27.7 | 57.8 | |
| Vote2Cap-DETR++Venue=T-PAMI242023.12 | 67.58 | — | — | — | — | 76.36 | 41.37 | 28.7 | 60 | 37.05 | 26.89 | 55.64 | |
| 3D-VisTAVenue=ICCV232023.12 | 66.9 | — | — | — | — | 71 | 36.5 | 28.4 | 57.6 | 34 | 27.1 | 54.3 | |
| LL3DAVenue=CVPR242023.12 | 65.19 | — | — | — | — | 74.17 | 41.41 | 27.76 | 59.53 | 36.79 | 25.97 | 55.06 | |
| D3NetVenue=ECCV222023.12 | 62.64 | — | — | — | — | — | — | — | — | 35.68 | 25.72 | 53.9 | |
| Vote2Cap-DETRVenue=CVPR232023.12 | 61.81 | — | — | — | — | 71.45 | 39.34 | 28.25 | 59.33 | 34.46 | 26.22 | 54.4 | |
| 3D-VLPVenue=CVPR232023.12 | 54.94 | — | — | — | — | 70.73 | 41.03 | 28.14 | 59.72 | 32.31 | 24.83 | 51.51 | |
| 3DJCGVenue=CVPR222023.12 | 49.48 | — | — | — | — | 64.7 | 40.17 | 27.66 | 59.23 | 31.03 | 24.22 | 50.8 | |
| X-Trans2CapVenue=CVPR222023.12 | 43.87 | — | — | — | — | 61.83 | 35.65 | 26.61 | 54.7 | 25.05 | 22.46 | 45.28 | |
| Scan2CapVenue=CVPR212023.12 | 39.08 | — | — | — | — | 56.82 | 34.18 | 26.29 | 55.27 | 23.32 | 21.97 | 44.48 | |
| GPT4Scene (HDM)Vision modal.=B,I, # of tokens=80002026.05 | 0.863 | — | — | — | — | — | — | — | — | 0.406 | — | — | |
| 3DRSVision modal.=I, # of tokens=80002026.05 | 0.861 | — | — | — | — | — | — | — | — | 0.416 | — | — | |
| LLaVA-3D2024.09 | 0.841 | — | — | — | — | — | — | — | — | 0.426 | 0.29 | 0.634 | |
| Video-3D-LLMVision modal.=I, # of tokens=80002026.05 | 0.838 | — | — | — | — | — | — | — | — | 0.414 | — | — | |
| PAR3Dcategory=Generalist 3D-MLLMs2026.06 | 0.814 | — | — | — | — | — | — | — | — | 0.373 | 0.275 | 0.579 | |
| 3DGraphLLMcategory=Generalist 3D-MLLMs2026.06 | 0.81 | — | — | — | — | — | — | — | — | 0.365 | — | — | |
| LLaVA-3DVision modal.=D,I, # of tokens=30962026.05 | 0.792 | — | — | — | — | — | — | — | — | 0.411 | — | — | |
| 3D-LLaVAModality=PC, Model Type=3D LMM2025.01 | 0.788 | — | — | — | — | — | — | — | — | 0.369 | 0.271 | 0.577 | |
| 3D-LLaVAcategory=Generalist 3D-MLLMs2026.06 | 0.788 | — | — | — | — | — | — | — | — | 0.369 | 0.271 | 0.577 | |
| Chat-SceneModality=PC+I, Model Type=3D LMM2025.01 | 0.772 | — | — | — | — | — | — | — | — | 0.364 | 0.28 | 0.581 | |
| ChatScene2024.09 | 0.772 | — | — | — | — | — | — | — | — | 0.363 | 0.28 | 0.581 | |
| Descrip3DVision modal.=I2026.05 | 0.772 | — | — | — | — | — | — | — | — | 0.345 | — | — | |
| Chat-Scenecategory=Generalist 3D-MLLMs2026.06 | 0.772 | — | — | — | — | — | — | — | — | 0.364 | 0.28 | 0.581 | |
| Chat-SceneVision modal.=P2026.05 | 0.771 | — | — | — | — | — | — | — | — | 0.363 | — | — | |
| Proxy3DVision modal.=I, # of tokens=700, Backbone=Qwen2.5-VL-7B2026.05 | 0.733 | — | — | — | — | — | — | — | — | 0.347 | — | — | |
| LEOModality=PC+I, Model Type=3D LMM2025.01 | 0.724 | — | — | — | — | — | — | — | — | 0.382 | 0.279 | 0.581 | |
| LEOcategory=Generalist 3D-MLLMs2026.06 | 0.724 | — | — | — | — | — | — | — | — | 0.382 | 0.279 | 0.581 | |
| Grounded 3D-LLMModality=PC, Model Type=3D LMM2025.01 | 0.706 | — | — | — | — | — | — | — | — | 0.355 | — | — | |
| Grounded 3D-LLMcategory=Generalist 3D-MLLMs2026.06 | 0.706 | — | — | — | — | — | — | — | — | 0.355 | — | — | |
| LEO2024.09 | 0.684 | — | — | — | — | — | — | — | — | 0.369 | 0.277 | 0.578 | |
| LL3DAModality=PC, Model Type=Finetuned 3D LMM2025.01 | 0.652 | — | — | — | — | — | — | — | — | 0.368 | 0.26 | 0.551 | |
| LL3DA2024.09 | 0.652 | — | — | — | — | — | — | — | — | 0.368 | 0.26 | 0.55 | |
| LL3DAcategory=Finetuned 3D-MLLMs2026.06 | 0.652 | — | — | — | — | — | — | — | — | 0.368 | 0.26 | 0.551 | |
| Vote2Cap-DETRModality=PC, Model Type=Specialist Model2025.01 | 0.618 | — | — | — | — | — | — | — | — | 0.345 | 0.262 | 0.544 | |
| Vote2Cap-DETR2024.09 | 0.618 | — | — | — | — | — | — | — | — | 0.345 | 0.262 | 0.544 | |
| Vote2Cap-DETRcategory=Specialist Models2026.06 | 0.618 | — | — | — | — | — | — | — | — | 0.345 | 0.262 | 0.544 | |
| 3D-VisTAModality=PC, Model Type=Specialist Model2025.01 | 0.616 | — | — | — | — | — | — | — | — | 0.341 | 0.268 | 0.55 | |
| 3D-VisTA2024.09 | 0.616 | — | — | — | — | — | — | — | — | 0.341 | 0.268 | 0.55 | |
| 3D-VisTAcategory=Specialist Models2026.06 | 0.616 | — | — | — | — | — | — | — | — | 0.341 | 0.268 | 0.55 | |
| 3D-VLP2024.09 | 0.55 | — | — | — | — | — | — | — | — | 0.323 | 0.248 | 0.515 | |
| 3D-VLPModality=PC, Model Type=Specialist Model2025.01 | 0.549 | — | — | — | — | — | — | — | — | 0.323 | 0.248 | 0.515 | |
| 3DJCGModality=PC, Model Type=Specialist Model2025.01 | 0.495 | — | — | — | — | — | — | — | — | 0.31 | 0.242 | 0.508 | |
| UniT3DModality=PC, Model Type=Specialist Model2025.01 | 0.467 | — | — | — | — | — | — | — | — | 0.272 | 0.219 | 0.46 | |
| UniT3Dcategory=Specialist Models2026.06 | 0.467 | — | — | — | — | — | — | — | — | 0.272 | 0.219 | 0.46 | |
| D3NetModality=PC, Model Type=Specialist Model2025.01 | 0.461 | — | — | — | — | — | — | — | — | 0.303 | 0.244 | 0.517 | |
| SpaCap3DModality=PC, Model Type=Specialist Model2025.01 | 0.44 | — | — | — | — | — | — | — | — | 0.253 | 0.223 | 0.454 | |
| MOREModality=PC, Model Type=Specialist Model2025.01 | 0.409 | — | — | — | — | — | — | — | — | 0.229 | 0.217 | 0.444 | |
| Scan2CapModality=PC, Model Type=Specialist Model2025.01 | 0.391 | — | — | — | — | — | — | — | — | 0.233 | 0.22 | 0.448 | |
| Scan2Cap2024.09 | 0.391 | — | — | — | — | — | — | — | — | 0.233 | 0.22 | 0.448 | |
| Scan2CapVision modal.=P, # of tokens=2562026.05 | 0.391 | — | — | — | — | — | — | — | — | 0.233 | — | — | |
| Scan2Capcategory=Specialist Models2026.06 | 0.391 | — | — | — | — | — | — | — | — | 0.233 | 0.22 | 0.448 | |
| Qwen2-VL-7BVision modal.=I, # of tokens=80002026.05 | 0 | — | — | — | — | — | — | — | — | 0.038 | — | — | |
| 3D-IDEIoU Threshold=0.52026.03 | — | 0.4076 | 0.2879 | 0.6213 | 0.7902 | — | — | — | — | — | — | — | |
| 3D-LLaVAModality=PC, Training Protocol=3D MLLMs, Parameter Count=7B2026.03 | — | 0.369 | 0.271 | 0.577 | — | — | — | — | — | — | — | — | |
| 3D-LLM(BLIP2-flant5)Learning strategy=LLM-based Methods, # 3D Data for Alignment=675K2025.03 | — | 0.081 | 0.131 | 0.332 | — | — | — | — | — | — | — | — | |
| 3D-LLM(Flamingo)Learning strategy=LLM-based Methods, # 3D Data for Alignment=675K2025.03 | — | 0.059 | 0.114 | 0.299 | — | — | — | — | — | — | — | — | |
| 3D-VisTALearning strategy=Closed-set, full sup.2025.03 | — | 0.34 | 0.271 | 0.543 | 0.669 | — | — | — | — | — | — | — | |
| 3D-VisTAMethod type=Expert models2025.12 | — | 0.341 | 0.268 | 0.55 | 0.616 | — | — | — | — | — | — | — | |
| 3D-VisTAModality=PC, Training Protocol=Specialist Models, Parameter Count=7B2026.03 | — | 0.341 | 0.268 | 0.55 | — | — | — | — | — | — | — | — | |
| 3D-VisTAIoU Threshold=0.52026.03 | — | 0.34 | 0.271 | 0.543 | 0.669 | — | — | — | — | — | — | — | |
| 3D-VLPMethod type=Expert models2025.12 | — | 0.323 | 0.248 | 0.515 | 0.55 | — | — | — | — | — | — | — | |
| 3D-VLPModality=PC, Training Protocol=Specialist Models, Parameter Count=7B2026.03 | — | 0.323 | 0.248 | 0.515 | — | — | — | — | — | — | — | — | |
| 3DJCGMethod type=Expert models2025.12 | — | 0.31 | 0.242 | 0.508 | 0.495 | — | — | — | — | — | — | — | |
| 3DJCGModality=PC, Training Protocol=Specialist Models, Parameter Count=7B2026.03 | — | 0.31 | 0.242 | 0.508 | — | — | — | — | — | — | — | — | |
| baselineIoU Threshold=0.5, Note=baseline from Video-3D2026.03 | — | 0.2998 | 0.2418 | 0.5766 | 0.3153 | — | — | — | — | — | — | — | |
| Chat-3D v2Learning strategy=LLM-based Methods, # 3D Data for Alignment=38K2025.03 | — | 0.318 | 0.223 | 0.502 | 0.639 | — | — | — | — | — | — | — | |
| Chat-SceneLearning strategy=LLM-based Methods, # 3D Data for Alignment=38K2025.03 | — | 0.363 | — | — | 0.771 | — | — | — | — | — | — | — | |
| Chat-SceneMethod type=3D LMMs2025.12 | — | 0.363 | 0.28 | 0.581 | 0.772 | — | — | — | — | — | — | — | |
| Chat-SceneModality=PC+I, Training Protocol=3D MLLMs, Parameter Count=7B2026.03 | — | 0.364 | 0.28 | 0.581 | — | — | — | — | — | — | — | — | |
| ChatSceneIoU Threshold=0.52026.03 | — | 0.3634 | 0.2801 | 0.5812 | 0.7719 | — | — | — | — | — | — | — | |
| CVPMethod type=3D LMMs2025.12 | — | 0.417 | 0.289 | 0.622 | 0.905 | — | — | — | — | — | — | — | |
| D3NetModality=PC, Training Protocol=Specialist Models, Parameter Count=7B2026.03 | — | 0.303 | 0.244 | 0.517 | — | — | — | — | — | — | — | — | |
| Grounded 3D-LLMLearning strategy=LLM-based Methods, # 3D Data for Alignment=107K2025.03 | — | 0.355 | — | — | 0.706 | — | — | — | — | — | — | — | |
| Grounded 3D-LLMModality=PC, Training Protocol=3D MLLMs, Parameter Count=7B2026.03 | — | 0.355 | — | — | — | — | — | — | — | — | — | — | |
| Inst3D-LMMLearning strategy=LLM-based Methods, # 3D Data for Alignment=38K2025.03 | — | 0.383 | 0.275 | 0.572 | 0.797 | — | — | — | — | — | — | — | |
| LEOMethod type=3D LMMs2025.12 | — | 0.369 | 0.277 | 0.578 | 0.684 | — | — | — | — | — | — | — | |
| LEOModality=PC+I, Training Protocol=3D MLLMs, Parameter Count=7B2026.03 | — | 0.382 | 0.279 | 0.581 | — | — | — | — | — | — | — | — | |
| LL3DALearning strategy=LLM-based Methods, # 3D Data for Alignment=38K2025.03 | — | 0.359 | 0.256 | 0.546 | 0.652 | — | — | — | — | — | — | — | |
| LL3DAMethod type=3D LMMs2025.12 | — | 0.368 | 0.26 | 0.55 | 0.652 | — | — | — | — | — | — | — | |
| LL3DAModality=PC, Training Protocol=Finetuned 3D MLLMs, Parameter Count=7B2026.03 | — | 0.368 | 0.26 | 0.551 | — | — | — | — | — | — | — | — | |
| LLaVALearning strategy=LLM-based Methods, mode=zero-shot2025.03 | — | 0.015 | 0.083 | 0.196 | 0.032 | — | — | — | — | — | — | — | |
| LLaVA-3DMethod type=3D LMMs2025.12 | — | 0.411 | 0.302 | 0.634 | 0.792 | — | — | — | — | — | — | — | |
| LLaVA-3DIoU Threshold=0.52026.03 | — | 0.4112 | 0.3021 | 0.6341 | 0.7921 | — | — | — | — | — | — | — | |
| MOREModality=PC, Training Protocol=Specialist Models, Parameter Count=7B2026.03 | — | 0.229 | 0.217 | 0.444 | — | — | — | — | — | — | — | — | |
| SAGE-7BModality=PC, Training Protocol=3D MLLMs, Parameter Count=7B2026.03 | — | 0.378 | 0.281 | 0.586 | — | — | — | — | — | — | — | — | |
| Scan2CapLearning strategy=Closed-set, full sup.2025.03 | — | 0.224 | 0.214 | 0.435 | 0.352 | — | — | — | — | — | — | — | |
| Scan2CapMethod type=Expert models2025.12 | — | 0.233 | 0.22 | 0.448 | 0.391 | — | — | — | — | — | — | — | |
| Scan2CapModality=PC, Training Protocol=Specialist Models, Parameter Count=7B2026.03 | — | 0.233 | 0.22 | 0.448 | — | — | — | — | — | — | — | — | |
| Scan2CapIoU Threshold=0.52026.03 | — | 0.2332 | 0.2197 | 0.4448 | 0.3908 | — | — | — | — | — | — | — | |
| SpaCap3DModality=PC, Training Protocol=Specialist Models, Parameter Count=7B2026.03 | — | 0.253 | 0.223 | 0.454 | — | — | — | — | — | — | — | — | |
| UniT3DModality=PC, Training Protocol=Specialist Models, Parameter Count=7B2026.03 | — | 0.272 | 0.219 | 0.46 | — | — | — | — | — | — | — | — | |
| VG-LLMIoU Threshold=0.52026.03 | — | 0.415 | 0.289 | 0.626 | 0.8 | — | — | — | — | — | — | — | |
| Video-3D-LLMMethod type=3D LMMs2025.12 | — | 0.413 | 0.289 | 0.623 | 0.838 | — | — | — | — | — | — | — | |
| Vote2Cap-DETRMethod type=Expert models2025.12 | — | 0.345 | 0.262 | 0.544 | 0.618 | — | — | — | — | — | — | — | |
| Vote2Cap-DETRModality=PC, Training Protocol=Specialist Models, Parameter Count=7B2026.03 | — | 0.345 | 0.262 | 0.544 | — | — | — | — | — | — | — | — |