Stored Household Item location prediction on Stored Household Item Challenge 1.0 (dev)
35.83AccuracyHuman Annotator
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Human AnnotatorType=Estimated2025.12 | 35.83 | 36.1 | |
| NOAM GPT-4IoU threshold=1, LLM Backbone=GPT-42025.12 | 28.11 | 28.7 | |
| NOAM LLaMA-3.3IoU threshold=1, LLM Backbone=LLaMA-3.32025.12 | 23.51 | 24.4 | |
| GPT-4o APIIoU threshold=>= 0.52025.12 | 13.78 | 13.8 | |
| Grounding-DINOPrompt configuration=no item in prompt, IoU threshold=12025.12 | 11.35 | 12.8 | |
| Grounding-DINOIoU threshold=>= 0.952025.12 | 9.49 | 11.2 | |
| Gemini-1.5-flashIoU threshold=>= 0.52025.12 | 8.11 | 8.3 | |
| Qwen-2.5IoU threshold=>= 0.52025.12 | 7.57 | 9.9 | |
| RandomIoU threshold=12025.12 | 7.55 | 8.3 | |
| Grounding-DINOIoU threshold=12025.12 | 5.89 | 11.2 | |
| LLaMA-4IoU threshold=>= 0.52025.12 | 3.24 | 7.1 | |
| Gemini-2.5-flashIoU threshold=>= 0.52025.12 | 2.43 | 3 | |
| Kosmos-2IoU threshold=>= 0.52025.12 | 0.46 | 1.3 |