# Ontology Grounding on Pixar Kitchen Set opaque regime SOMA-HOME ontology (test)

Source: [Ontology Grounding on Pixar Kitchen Set opaque regime SOMA-HOME ontology (test)](<https://www.sota2.com/research/sota/ontology-grounding-on-pixar-kitchen-set-opaque-regime-soma-home-ontology-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-09T10:24:14.347Z

Evaluation of Ontology Grounding performance on the test split of the Pixar Kitchen Set dataset, specifically using the opaque regime defined by the SOMA-HOME ontology. Metrics reported are EMA and HF1.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Ontology Grounding |
| Dataset (including any stated split) | Pixar Kitchen Set opaque regime SOMA-HOME ontology (test) |
| Metric | EMA |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [EMA](<https://www.sota2.com/research/sota/ontology-grounding-on-pixar-kitchen-set-opaque-regime-soma-home-ontology-test.md?metric=EMA>): Higher is better; unit: SCALAR.

- [HF1](<https://www.sota2.com/research/sota/ontology-grounding-on-pixar-kitchen-set-opaque-regime-soma-home-ontology-test.md?metric=HF1>): Higher is better; unit: PERCENT.

## Recorded results

| Method | EMA | Result date | Report paper |
| --- | --- | --- | --- |
| Gemini 3 Flash | 48 | Not recorded | 6a279e5390966ade919bf9b0 |
| Gemini 3 Flash | 48 | Not recorded | 6a279e5390966ade919bf9b0 |
| Gemini 2.5 Flash | 33 | Not recorded | 6a279e5390966ade919bf9b0 |
| Gemini 2.5 Flash | 30 | Not recorded | 6a279e5390966ade919bf9b0 |
| Qwen 3.5-27B | 26 | Not recorded | 6a279e5390966ade919bf9b0 |
| Qwen 3.5-27B | 21 | Not recorded | 6a279e5390966ade919bf9b0 |
| Qwen 3.5-27B | 19 | Not recorded | 6a279e5390966ade919bf9b0 |
| Qwen 3.5-27B | 17 | Not recorded | 6a279e5390966ade919bf9b0 |
| Gemini 2.5 Flash | 6 | Not recorded | 6a279e5390966ade919bf9b0 |
| Gemini 3 Flash | 6 | Not recorded | 6a279e5390966ade919bf9b0 |
| Qwen 3.5-27B | 6 | Not recorded | 6a279e5390966ade919bf9b0 |
| LV-NN | 2 | Not recorded | 6a279e5390966ade919bf9b0 |
| S-BERT | 2 | Not recorded | 6a279e5390966ade919bf9b0 |
| Multiverse | 0 | Not recorded | 6a279e5390966ade919bf9b0 |

### Gemini 3 Flash

EMA: 48. Result ID: 6a27e937fbc373592c25528c.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

Recorded evaluation configuration: {"Prompting Strategy":"B"}. This record does not assert independent reproduction.

### Gemini 3 Flash

EMA: 48. Result ID: 6a27e937fbc373592c25528d.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

Recorded evaluation configuration: {"Prompting Strategy":"C"}. This record does not assert independent reproduction.

### Gemini 2.5 Flash

EMA: 33. Result ID: 6a27e937fbc373592c255289.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

Recorded evaluation configuration: {"Prompting Strategy":"B"}. This record does not assert independent reproduction.

### Gemini 2.5 Flash

EMA: 30. Result ID: 6a27e937fbc373592c25528a.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

Recorded evaluation configuration: {"Prompting Strategy":"C"}. This record does not assert independent reproduction.

### Qwen 3.5-27B

EMA: 26. Result ID: 6a27e937fbc373592c255290.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

Recorded evaluation configuration: {"Model Mode":"instruct","Prompting Strategy":"C"}. This record does not assert independent reproduction.

### Qwen 3.5-27B

EMA: 21. Result ID: 6a27e937fbc373592c255292.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

Recorded evaluation configuration: {"Model Mode":"thinking","Prompting Strategy":"C"}. This record does not assert independent reproduction.

### Qwen 3.5-27B

EMA: 19. Result ID: 6a27e937fbc373592c255291.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

Recorded evaluation configuration: {"Model Mode":"thinking","Prompting Strategy":"B"}. This record does not assert independent reproduction.

### Qwen 3.5-27B

EMA: 17. Result ID: 6a27e937fbc373592c25528f.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

Recorded evaluation configuration: {"Model Mode":"instruct","Prompting Strategy":"B"}. This record does not assert independent reproduction.

### Gemini 2.5 Flash

EMA: 6. Result ID: 6a27e937fbc373592c255288.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

Recorded evaluation configuration: {"Prompting Strategy":"A"}. This record does not assert independent reproduction.

### Gemini 3 Flash

EMA: 6. Result ID: 6a27e937fbc373592c25528b.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

Recorded evaluation configuration: {"Prompting Strategy":"A"}. This record does not assert independent reproduction.

### Qwen 3.5-27B

EMA: 6. Result ID: 6a27e937fbc373592c25528e.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

Recorded evaluation configuration: {"Model Mode":"instruct","Prompting Strategy":"A"}. This record does not assert independent reproduction.

### LV-NN

EMA: 2. Result ID: 6a27e937fbc373592c255286.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

Recorded evaluation configuration: {"type":"Log-Volume Nearest Neighbor"}. This record does not assert independent reproduction.

### S-BERT

EMA: 2. Result ID: 6a27e937fbc373592c255287.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66441e209131cea0f5ab.md>)

[Implementation](<https://github.com/UKPLab/sentence-transformers>)

Recorded evaluation configuration: {"type":"Cosine Similarity","model":"all-MiniLM-L6-v2"}. This record does not assert independent reproduction.

### Multiverse

EMA: 0. Result ID: 6a27e937fbc373592c255285.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa710a1e209131cea4ed23.md>)

Recorded evaluation configuration: {"type":"Dictionary Mapping"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
