Object localization on ReaSCAN (test)
99.6Success Rate A1Syntax Guided Transformer
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Syntax Guided Transformer2024.12 | 99.6 | 97.3 | 99.6 | 95.4 | 90.1 | 92.5 | 21.7 | 85.2 | |
| NeSyCoCo (w/o Embedding)word embedding=false2024.12 | 99.5 | 91.7 | 99.1 | 97.6 | 97.7 | 94.5 | 95.9 | 96.5 | |
| GroCoT2024.12 | 99.4 | 85.7 | 95.4 | 90.4 | 83.5 | 70.2 | 27.7 | 78.5 | |
| NeSyCoCo (Full)2024.12 | 99.1 | 94.1 | 98.5 | 98.9 | 98.7 | 96.8 | 95.9 | 97.5 | |
| NeSyCoCo (w/o Emb. w/o Soft Reasoner)word embedding=false, soft reasoner=false, improved prompting=true2024.12 | 97.8 | 80.3 | 95.5 | 93 | 95.8 | 92.2 | 92.9 | 92.5 | |
| LEFTdependency parsing=false2024.12 | 97.8 | 80.3 | 95.5 | 93 | 95.8 | 92.2 | 91.5 | 92.3 |