3D Question Answering on ScanQA (test)
13.9BLEU-4NaviLLM
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| NaviLLM2023.12 | 13.9 | — | 16.6 | 40.2 | 80.8 | 26.3 | — | |
| SIG3D2024.06 | 12.4 | 39.5 | 13.4 | 35.9 | 68.8 | — | — | |
| ScanQAExplicit object representations=true2023.07 | 12 | 31.6 | 13.5 | 34.3 | 67.3 | 23.5 | — | |
| ScanQA2023.12 | 12 | — | 13.5 | 34.3 | 67.3 | 23.5 | — | |
| 3D-LLM2024.06 | 12 | 39.3 | 14.5 | 35.7 | 69.4 | — | — | |
| 3D-LLMBackbone=BLIP2-flant52023.07 | 11.6 | 38.3 | 14.9 | 35.3 | 69.6 | 19.1 | — | |
| 3D-LLMBackbone=BLIP2-flant52023.12 | 11.6 | — | 14.9 | 35.3 | 69.6 | 19.1 | — | |
| 3D-LLMBackbone=BLIP2-opt2023.07 | 10.7 | 37.3 | 14.3 | 34.5 | 67.1 | 19.1 | — | |
| ScanQA2024.06 | 10.1 | 30.2 | 13.1 | 33.3 | 64.9 | — | — | |
| 3D-LLMBackbone=Flamingo2023.07 | 8.4 | 32.6 | 13.5 | 34.8 | 65.6 | 23.2 | — | |
| 3D-LLMBackbone=Flamingo2023.12 | 8.4 | — | 13.5 | 34.8 | 65.6 | 23.2 | — | |
| Flamingo2024.06 | 8.4 | 25.6 | 11.3 | 31.1 | 55 | — | — | |
| SR+MCANPoint cloud encoder=ScanRefer2024.06 | 7.9 | 26.9 | 11.5 | 30 | 55.4 | — | — | |
| ScanRefer+MCANExplicit object representations=true2023.07 | 7.5 | 27.9 | 11.9 | 30.7 | 57.4 | 20.6 | — | |
| ScanRefer+MCAN2023.12 | 7.5 | — | 11.9 | 30.7 | 57.4 | 20.6 | — | |
| VN+MCANPoint cloud encoder=VoteNet2024.06 | 6.2 | 28 | 11.4 | 29.8 | 54.7 | — | — | |
| VoteNet+MCANExplicit object representations=true2023.07 | 6 | 29.5 | 12 | 30.9 | 58.2 | 19.7 | — | |
| VoteNet+MCAN2023.12 | 6 | — | 12 | 30.9 | 58.2 | 19.7 | — | |
| BLIP22024.06 | 5.9 | 29.7 | 11.3 | 26.6 | 45.7 | — | — | |
| SingleImage+MCAN2023.07 | 0 | 16.5 | 8.4 | 21.5 | 38.6 | 15.8 | — | |
| 3D-LLMModel Training Strategy=Separate Model For Each Task2023.12 | — | — | — | — | — | — | 19.1 | |
| NaviLLMModel Training Strategy=Unified Model For All Tasks2023.12 | — | — | — | — | — | — | 26.3 |