3D Dense Captioning on Scan2Cap
89CIDEr @0.5Video-3D LLM†
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Video-3D LLM†Video sequence length=32, 3D Encoder=Additional 3D encoder2026.04 | 89 | — | — | — | — | 42.6 | — | — | — | — | — | — | — | — | |
| Geo3DPrunerVideo sequence length=32, 3D Encoder=Additional 3D encoder, Reduction ratio=↓60%2026.04 | 87.3 | — | — | — | — | 42.4 | — | — | — | — | — | — | — | — | |
| 3DRSModel category=Generalists, 3D geometric inputs=Yes2026.03 | 86.1 | — | — | — | — | 41.6 | — | — | — | — | — | — | — | — | |
| Geo3DPrunerVideo sequence length=32, 3D Encoder=Additional 3D encoder, Reduction ratio=↓80%2026.04 | 85.7 | — | — | — | — | 42.1 | — | — | — | — | — | — | — | — | |
| GAP-MLLM-3B (Ours)3D Input=✗2026.03 | 84.7 | — | — | — | — | 42.1 | — | — | — | — | — | — | 63.1 | — | |
| Utonia2026.03 | 83.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Video-3D LLM (Uniform)3D Generalist=true, sampling strategy=uniform sampling2024.11 | 83.8 | — | — | — | — | 41.3 | — | — | — | — | — | — | — | — | |
| Baseline2026.03 | 83.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Video-3D LLMModel category=Generalists, 3D geometric inputs=Yes2026.03 | 83.8 | — | — | — | — | 41.3 | — | — | — | — | — | — | — | — | |
| Video-3D LLM(Uniform)3D Input=✓2026.04 | 83.8 | — | — | — | — | 42.4 | 28.9 | 62.3 | — | — | — | — | — | — | |
| Video-3D LLMVideo sequence length=322026.04 | 83.8 | — | — | — | — | 41.3 | — | — | — | — | — | — | — | — | |
| Sonata2026.03 | 83.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Geo3DPrunerVideo sequence length=32, 3D Encoder=Additional 3D encoder, Reduction ratio=↓90%2026.04 | 82.3 | — | — | — | — | 41.3 | — | — | — | — | — | — | — | — | |
| VID-LLMModel category=Generalists, 3D geometric inputs=No2026.03 | 81.5 | — | — | — | — | 40.6 | — | — | — | — | — | — | — | — | |
| GUIDE-9B3D Input=✗2026.04 | 81.2 | — | — | — | — | 41 | 30.1 | 62 | — | — | — | — | — | — | |
| Inst3D-LMMLLM=Qwen3-8B2026.07 | 80.9 | — | — | — | — | 38 | — | — | — | — | — | — | — | — | |
| CAIRNLLM=Qwen3-8B2026.07 | 80.7 | — | — | — | — | 36.2 | — | — | — | — | — | — | — | — | |
| VG-LLM (w/ GAP)3D Input=✗2026.03 | 80.6 | — | — | — | — | 41.1 | — | — | — | — | — | — | 62.8 | — | |
| PQ3D3D Generalist=true2024.11 | 80.3 | — | — | — | — | 36 | — | — | — | — | — | — | — | — | |
| PQ3DModel category=Generalists, 3D geometric inputs=Yes2026.03 | 80.3 | — | — | — | — | 36 | — | — | — | — | — | — | — | — | |
| PQ3DLLM=None2026.07 | 80.3 | — | — | — | — | 36 | — | — | — | — | — | — | — | — | |
| Video-3D LLM (MC)3D Generalist=true, sampling strategy=maximum coverage2024.11 | 80 | — | — | — | — | 40.2 | — | — | — | — | — | — | — | — | |
| Video-3D LLM3D Input=✓2026.03 | 80 | — | — | — | — | 40.2 | — | — | — | — | — | — | 61.7 | — | |
| VG LLM-8BInput 2D=true2026.03 | 80 | — | — | — | — | 41.5 | 28.9 | 62.6 | — | — | — | — | — | — | |
| Video-3D LLMInput 2D=true, Input 3D=true2026.03 | 80 | — | — | — | — | 40.2 | 28.5 | 61.7 | — | — | — | — | — | — | |
| VG LLM-8BModel category=Generalists, 3D geometric inputs=No2026.03 | 80 | — | — | — | — | 41.5 | — | — | — | — | — | — | — | — | |
| Video-3D LLM(MC)3D Input=✓2026.04 | 80 | — | — | — | — | 40.2 | 28.5 | 61.7 | — | — | — | — | — | — | |
| VG LLM-8B3D Input=✗2026.04 | 80 | — | — | — | — | 41.5 | 28.9 | 62.6 | — | — | — | — | — | — | |
| VG LLMVideo sequence length=322026.04 | 80 | — | — | — | — | 41.5 | — | — | — | — | — | — | — | — | |
| Video-3D-LLMVenue=CVPR’25, Input=RGBD2026.06 | 80 | — | — | — | — | 40.2 | 28.5 | 61.7 | — | — | — | — | — | — | |
| Inst3D-LLMModel category=Generalists, 3D geometric inputs=Yes2026.03 | 79.7 | — | — | — | — | 38.3 | — | — | — | — | — | — | — | — | |
| Inst3D-LMMLLM=Vicuna-7B-v1.52026.07 | 79.7 | — | — | — | — | 38.3 | — | — | — | — | — | — | — | — | |
| Concerto2026.03 | 79.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaVA-3D3D Generalist=true2024.11 | 79.2 | — | — | — | — | 41.1 | — | — | — | — | — | — | — | — | |
| LLaVA-3D3D Input=✓2026.03 | 79.2 | — | — | — | — | 41.1 | — | — | — | — | — | — | 63.4 | — | |
| LLaVA-3DInput 2D=true, Input 3D=true2026.03 | 79.2 | — | — | — | — | 41.1 | 30.2 | 63.4 | — | — | — | — | — | — | |
| LLaVA-3DModel category=Generalists, 3D geometric inputs=Yes2026.03 | 79.2 | — | — | — | — | 41.1 | — | — | — | — | — | — | — | — | |
| LLaVA-3D3D Input=✓2026.04 | 79.2 | — | — | — | — | 41.1 | 30.2 | 63.4 | — | — | — | — | — | — | |
| LLaVA-3D2026.04 | 79.2 | — | — | — | — | 41.1 | — | — | — | — | — | — | — | — | |
| LLaVA-3DVenue=ICCV’25, Input=RGBD2026.06 | 79.2 | — | — | — | — | 41.1 | 30.2 | 63.4 | — | — | — | — | — | — | |
| Motion-MLLM-4BInput 2D=true, Input M (egomotion)=true2026.03 | 79 | — | — | — | — | 41.6 | 30 | 64 | — | — | — | — | — | — | |
| 3D-IDEModel category=Generalists, 3D geometric inputs=No2026.03 | 79 | — | — | — | — | 40.7 | — | — | — | — | — | — | — | — | |
| DVTIEPruning Ratio=35%2026.04 | 78.92 | — | — | — | — | 36.35 | — | — | — | — | — | — | — | — | |
| 3D-LLaVAModel category=Generalists, 3D geometric inputs=Yes2026.03 | 78.8 | — | — | — | — | 36.9 | — | — | — | — | — | — | — | — | |
| VG-LLM-4B3D Input=✗2026.03 | 78.6 | — | — | — | — | 40.9 | — | — | — | — | — | — | 62.4 | — | |
| VG LLM-4BInput 2D=true2026.03 | 78.6 | — | — | — | — | 40.9 | 28.6 | 62.4 | — | — | — | — | — | — | |
| VG LLM-4BModel category=Generalists, 3D geometric inputs=No2026.03 | 78.6 | — | — | — | — | 40.9 | — | — | — | — | — | — | — | — | |
| VG LLM-4B3D Input=✗2026.04 | 78.6 | — | — | — | — | 40.9 | 28.6 | 62.4 | — | — | — | — | — | — | |
| 3DGraphLLMLLM=Qwen3-8B2026.07 | 78.1 | — | — | — | — | 35.2 | — | — | — | — | — | — | — | — | |
| Chat-SceneModel Category=LLM-based Models2023.12 | 77.1 | — | — | — | — | 36.3 | — | — | — | — | — | — | — | — | |
| ChatScene3D Generalist=true2024.11 | 77.1 | — | — | — | — | 36.3 | — | — | — | — | — | — | — | — | |
| Chat-Scene3D Input=✓2026.03 | 77.1 | — | — | — | — | 36.5 | — | — | — | — | — | — | — | — | |
| Chat-SceneInput 2D=true, Input 3D=true2026.03 | 77.1 | — | — | — | — | 36.5 | — | — | — | — | — | — | — | — | |
| ChatSceneModel category=Generalists, 3D geometric inputs=Yes2026.03 | 77.1 | — | — | — | — | 36.3 | — | — | — | — | — | — | — | — | |
| Chat-Scene3D Input=✓2026.04 | 77.1 | — | — | — | — | 36.5 | — | — | — | — | — | — | — | — | |
| Chat-SceneVenue=NIPS’24, Input=PC2026.06 | 77.1 | — | — | — | — | 36.5 | — | — | — | — | — | — | — | — | |
| Chat-SceneLLM=Vicuna-7B-v1.52026.07 | 77.1 | — | — | — | — | 36.3 | — | — | — | — | — | — | — | — | |
| Occ-VLMInput=RGB2026.06 | 76.5 | — | — | — | — | 38.9 | 28.2 | 60.8 | — | — | — | — | — | — | |
| Baseline (Chat-Scene)Pruning Ratio=0%2026.04 | 76.35 | — | — | — | — | 35.86 | — | — | — | — | — | — | — | — | |
| FastVPruning Ratio=35%2026.04 | 75.94 | — | — | — | — | 35.89 | — | — | — | — | — | — | — | — | |
| Fast3DPruning Ratio=35%2026.04 | 75.83 | — | — | — | — | 35.56 | — | — | — | — | — | — | — | — | |
| DVTIEPruning Ratio=65%2026.04 | 75.43 | — | — | — | — | 35.52 | — | — | — | — | — | — | — | — | |
| Chat-SceneLLM=Qwen3-8B2026.07 | 75.4 | — | — | — | — | 34.5 | — | — | — | — | — | — | — | — | |
| DVTIEPruning Ratio=90%2026.04 | 75.33 | — | — | — | — | 35.96 | — | — | — | — | — | — | — | — | |
| GPT4Scene-HDLLM=Qwen2-VL-7B2026.07 | 74.4 | — | — | — | — | 37.9 | — | — | — | — | — | — | — | — | |
| ToMePruning Ratio=35%2026.04 | 73.43 | — | — | — | — | 35.76 | — | — | — | — | — | — | — | — | |
| Fast3DPruning Ratio=65%2026.04 | 73.24 | — | — | — | — | 34.77 | — | — | — | — | — | — | — | — | |
| Random PruningPruning Ratio=35%2026.04 | 72.77 | — | — | — | — | 34.9 | — | — | — | — | — | — | — | — | |
| LEO3D Generalist=true2024.11 | 72.4 | — | — | — | — | 38.2 | — | — | — | — | — | — | — | — | |
| LEO3D Input=✓2026.03 | 72.4 | — | — | — | — | 38.2 | — | — | — | — | — | — | 58.1 | — | |
| LEOModel category=Generalists, 3D geometric inputs=Yes2026.03 | 72.4 | — | — | — | — | 38.2 | — | — | — | — | — | — | — | — | |
| LEO3D Input=✓2026.04 | 72.4 | — | — | — | — | 38.2 | 27.9 | 58.1 | — | — | — | — | — | — | |
| LEO2026.04 | 72.4 | — | — | — | — | 38.2 | — | — | — | — | — | — | — | — | |
| LEOLLM=Vicuna-7B-v1.12026.07 | 72.4 | — | — | — | — | 38.2 | — | — | — | — | — | — | — | — | |
| Grounded 3D-LLM3D Generalist=true2024.11 | 70.6 | — | — | — | — | 35.5 | — | — | — | — | — | — | — | — | |
| Grounded 3D-LLMModel category=Generalists, 3D geometric inputs=Yes2026.03 | 70.6 | — | — | — | — | 35.5 | — | — | — | — | — | — | — | — | |
| Grounded 3D-LLMLLM=Tiny-Vicuna-1B2026.07 | 70.6 | — | — | — | — | 35.5 | — | — | — | — | — | — | — | — | |
| Grounded 3D-LLM3D Input=✓2026.03 | 70.2 | — | — | — | — | 35 | — | — | — | — | — | — | — | — | |
| Grounded 3D-LLMInput 3D=true2026.03 | 70.2 | — | — | — | — | 35 | — | — | — | — | — | — | — | — | |
| Grounded 3D-LLM3D Input=✓2026.04 | 70.2 | — | — | — | — | 35 | — | — | — | — | — | — | — | — | |
| Fast3DPruning Ratio=90%2026.04 | 69.25 | — | — | — | — | 32.98 | — | — | — | — | — | — | — | — | |
| LEOModel Category=LLM-based Models2023.12 | 68.4 | — | — | — | — | 36.9 | — | — | — | — | — | — | — | — | |
| Vote2Cap-DETR++Model Category=Expert Models2023.12 | 67.6 | — | — | — | — | 37.1 | — | — | — | — | — | — | — | — | |
| Vote2Cap-DETR++LLM=None2026.07 | 67.6 | — | — | — | — | 37.1 | — | — | — | — | — | — | — | — | |
| 3D-VisTAtraining=pre-trained2023.08 | 66.9 | 71 | 36.5 | 28.4 | 57.6 | 34 | 27.1 | 54.3 | — | — | — | — | — | — | |
| 3D-VisTAModel Category=Expert Models2023.12 | 66.9 | — | — | — | — | 34 | — | — | — | — | — | — | — | — | |
| 3D-VisTA3D Generalist=false2024.11 | 66.9 | — | — | — | — | 34 | — | — | — | — | — | — | — | — | |
| 3D-VisTAInput 3D=true2026.03 | 66.9 | — | — | — | — | 34 | — | — | — | — | — | — | — | — | |
| 3D-VisTAModel category=Specialists2026.03 | 66.9 | — | — | — | — | 34 | — | — | — | — | — | — | — | — | |
| 3D-VisTALLM=None2026.07 | 66.9 | — | — | — | — | 34 | — | — | — | — | — | — | — | — | |
| LL3DAModel Category=LLM-based Models2023.12 | 65.2 | — | — | — | — | 36.8 | — | — | — | — | — | — | — | — | |
| LL3DA3D Input=✓2026.03 | 65.2 | — | — | — | — | 36.8 | — | — | — | — | — | — | 55 | — | |
| LL3DA3D Input=✓2026.04 | 65.2 | — | — | — | — | 36.8 | 26 | 55 | — | — | — | — | — | — | |
| LL3DAVenue=CVPR’24, Input=PC2026.06 | 65.2 | — | — | — | — | 36.8 | 26 | 55 | — | — | — | — | — | — | |
| LL3DALLM=OPT-1.3B2026.07 | 65.2 | — | — | — | — | 36.8 | — | — | — | — | — | — | — | — | |
| Chat-3D v23D Generalist=true2024.11 | 63.9 | — | — | — | — | 31.8 | — | — | — | — | — | — | — | — | |
| Chat-3D-v23D Input=✓2026.03 | 63.9 | — | — | — | — | 31.8 | — | — | — | — | — | — | — | — | |
| Chat-3D v2Model category=Generalists, 3D geometric inputs=Yes2026.03 | 63.9 | — | — | — | — | 31.8 | — | — | — | — | — | — | — | — | |
| Chat-3D-v23D Input=✓2026.04 | 63.9 | — | — | — | — | 31.8 | — | — | — | — | — | — | — | — | |
| LL3DA3D Generalist=true2024.11 | 62.9 | — | — | — | — | 36 | — | — | — | — | — | — | — | — |