# Query by Semantic Sketch

Source: [Query by Semantic Sketch](<https://www.sota2.com/research/paper/68fa66791e209131cea106c9>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:50:28.993Z

Authors: Luca Rossetto, Ralph Gasser, Heiko Schuldt

Published: 2019-09-27T07:27:04.000Z

## Abstract

Sketch-based query formulation is very common in image and video retrieval as these techniques often complement textual retrieval methods that are based on either manual or machine generated annotations. In this paper, we present a retrieval approach that allows to query visual media collections by sketching concept maps, thereby merging sketch-based retrieval with the search for semantic labels. Users can draw a spatial distribution of different concept labels, such as "sky", "sea" or "person" and then use these sketches to find images or video scenes that exhibit a similar distribution of these concepts. Hence, this approach does not only take the semantic concepts themselves into account, but also their semantic relations as well as their spatial context. The efficient vector representation enables efficient retrieval even in large multimedia collections. We have integrated the semantic sketch query mode into our retrieval engine vitrivr and demonstrated its effectiveness.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1909.12526>)

[PDF](<https://arxiv.org/pdf/1909.12526.pdf>)
