# 3D Question Answering on ScanQA (Nov 2023 Leaderboard Test)

Source: [3D Question Answering on ScanQA (Nov 2023 Leaderboard Test)](<https://www.sota2.com/research/sota/3d-question-answering-on-scanqa-nov-2023-leaderboard-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:41:35.362Z

Evaluation of 3D Question Answering performance on the specific snapshot of the ScanQA test set results reported on the leaderboard as of November 2023, using standard generation metrics and Exact Match.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | 3D Question Answering |
| Dataset (including any stated split) | ScanQA Leaderboard as of Nov 2023 (test) |
| Metric | EM |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 5 of 5 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [EM](<https://www.sota2.com/research/sota/3d-question-answering-on-scanqa-nov-2023-leaderboard-test.md?metric=EM>): Higher is better; unit: PERCENT.

- [BLEU-1](<https://www.sota2.com/research/sota/3d-question-answering-on-scanqa-nov-2023-leaderboard-test.md?metric=BLEU-1>): Higher is better; unit: PERCENT.

- [BLEU-4](<https://www.sota2.com/research/sota/3d-question-answering-on-scanqa-nov-2023-leaderboard-test.md?metric=BLEU-4>): Higher is better; unit: PERCENT.

- [ROUGE](<https://www.sota2.com/research/sota/3d-question-answering-on-scanqa-nov-2023-leaderboard-test.md?metric=ROUGE>): Higher is better; unit: PERCENT.

- [METEOR](<https://www.sota2.com/research/sota/3d-question-answering-on-scanqa-nov-2023-leaderboard-test.md?metric=METEOR>): Higher is better; unit: PERCENT.

- [CIDEr](<https://www.sota2.com/research/sota/3d-question-answering-on-scanqa-nov-2023-leaderboard-test.md?metric=CIDEr>): Higher is better; unit: PERCENT.

## Recorded results

| Method | EM | Result date | Report paper |
| --- | --- | --- | --- |
| mare | 30.82 | Not recorded | 68fa72be1e209131cea58557 |
| NaviLLM | 24.77 | Not recorded | 68fa72be1e209131cea58557 |
| bubble-bee | 23.78 | Not recorded | 68fa72be1e209131cea58557 |
| Optimus-prime | 23.01 | Not recorded | 68fa72be1e209131cea58557 |
| ML | 21.37 | Not recorded | 68fa72be1e209131cea58557 |

### mare

EM: 30.82. Result ID: 699290a23a6c3148490fd10e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72be1e209131cea58557.md>)

Recorded evaluation configuration: {"Rank":"1"}. This record does not assert independent reproduction.

### NaviLLM

EM: 24.77. Result ID: 699290a23a6c3148490fd10f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72be1e209131cea58557.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72be1e209131cea58557.md>)

[Implementation](<https://github.com/zd11024/NaviLLM>)

Recorded evaluation configuration: {"Rank":"2"}. This record does not assert independent reproduction.

### bubble-bee

EM: 23.78. Result ID: 699290a23a6c3148490fd110.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72be1e209131cea58557.md>)

Recorded evaluation configuration: {"Rank":"3"}. This record does not assert independent reproduction.

### Optimus-prime

EM: 23.01. Result ID: 699290a23a6c3148490fd111.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72be1e209131cea58557.md>)

Recorded evaluation configuration: {"Rank":"4"}. This record does not assert independent reproduction.

### ML

EM: 21.37. Result ID: 699290a23a6c3148490fd112.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72be1e209131cea58557.md>)

Recorded evaluation configuration: {"Rank":"5"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.