# See & Sniff: Learning Visuo-Olfactory Representations

Source: [See & Sniff: Learning Visuo-Olfactory Representations](<https://www.sota2.com/research/paper/6a3e07d1206542251d7cb77e>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-27T05:00:11.922Z

Authors: Seongyu Kim, Seungwoo Lee, Hyeonggon Ryu, Joon Son Chung, Arda Senocak

Published: 2026-06-25T17:26:47.000Z

## Abstract

While modern multimodal models integrate vision with language, audio, or touch, olfaction remains largely unexplored due to the lack of paired visuo-olfactory data. We introduce SmellNet-V, a scalable visuo-olfactory dataset built on the insight that odor identity is largely invariant to visual transformations within a semantic category. This allows us to synthetically pair smell-only samples with semantically aligned in-the-wild web images, converting a unimodal olfactory dataset into a cross-modal benchmark without costly co-collection. Building on this dataset, we propose See & Sniff, a self-supervised framework that learns joint visuo-olfactory representations via dense local alignment and naturally produces smell saliency maps for spatial grounding of odor sources. We further introduce pixel-level smell localization task and a benchmark for evaluation. Our method surpasses smell-only baselines by 7% in smell classification from smell alone and generalizes to cross-modal retrieval and smell localization, establishing visuo-olfactory learning as a new direction in multimodal perception.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Smell Classification on SmellNet smell-only (test)](<https://www.sota2.com/research/sota/smell-classification-on-smellnet-smell-only-test.md>): Smell Classification; SmellNet smell-only (test); Accuracy: 63.75.

- [Family-wise Classification on SmellNet V (unseen ingredients)](<https://www.sota2.com/research/sota/family-wise-classification-on-smellnet-v-unseen-ingredients.md>): Family-wise Classification; SmellNet V (unseen ingredients); Accuracy: 52.97.

- [Smell-to-Vision Retrieval on SmellNet-V (test)](<https://www.sota2.com/research/sota/smell-to-vision-retrieval-on-smellnet-v-test.md>): Smell-to-Vision Retrieval; SmellNet-V (test); R@1: 56.14.

- [Vision-to-Smell Retrieval on SmellNet-V (test)](<https://www.sota2.com/research/sota/vision-to-smell-retrieval-on-smellnet-v-test.md>): Vision-to-Smell Retrieval; SmellNet-V (test); Recall@1: 63.2.

- [Smell-to-Smell Retrieval on SmellNet (test)](<https://www.sota2.com/research/sota/smell-to-smell-retrieval-on-smellnet-test.md>): Smell-to-Smell Retrieval; SmellNet (test); R@1: 69.44.

- [Smell Localization on SmellNet-V-Source](<https://www.sota2.com/research/sota/smell-localization-on-smellnet-v-source.md>): Smell Localization; SmellNet-V-Source; mIoU: 0.6456.

- [Interactive Localization on SmellNet InteractiveSource V (test)](<https://www.sota2.com/research/sota/interactive-localization-on-smellnet-interactivesource-v-test.md>): Interactive Localization; SmellNet InteractiveSource V (test); IIoU: 35.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.27307>)

[PDF](<https://arxiv.org/pdf/2606.27307.pdf>)