# Multilingual Conceptual Coverage in Text-to-Image Models

Source: [Multilingual Conceptual Coverage in Text-to-Image Models](<https://www.sota2.com/research/paper/68fa6f911e209131cea464e8>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-02T02:51:25.090Z

Authors: Michael Saxon, William Yang Wang

Published: 2023-06-02T17:59:09.000Z

## Abstract

We propose "Conceptual Coverage Across Languages" (CoCo-CroLa), a technique for benchmarking the degree to which any generative text-to-image system provides multilingual parity to its training language in terms of tangible nouns. For each model we can assess "conceptual coverage" of a given target language relative to a source language by comparing the population of images generated for a series of tangible nouns in the source language to the population of images generated for each noun under translation in the target language. This technique allows us to estimate how well-suited a model is to a target language as well as identify model-specific weaknesses, spurious correlations, and biases without a-priori assumptions. We demonstrate how it can be used to benchmark T2I models in terms of multilinguality, and how despite its simplicity it is a good proxy for impressive generalization.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2306.01735>)

[PDF](<https://arxiv.org/pdf/2306.01735.pdf>)

[Implementation](<https://github.com/michaelsaxon/CoCoCroLa>)