# Don't Learn, Ground: A Case for Natural Language Inference with Visual Grounding

Source: [Don't Learn, Ground: A Case for Natural Language Inference with Visual Grounding](<https://www.sota2.com/research/paper/6923e675d6b0dd334292eedc>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-11-25T05:03:07.149Z

Authors: Daniil Ignatev, Ayman Santeer, Albert Gatt, Denis Paperno

Published: 2025-11-21T16:23:17.000Z

## Abstract

We propose a zero-shot method for Natural Language Inference (NLI) that leverages multimodal representations by grounding language in visual contexts. Our approach generates visual representations of premises using text-to-image models and performs inference by comparing these representations with textual hypotheses. We evaluate two inference techniques: cosine similarity and visual question answering. Our method achieves high accuracy without task-specific fine-tuning, demonstrating robustness against textual biases and surface heuristics. Additionally, we design a controlled adversarial dataset to validate the robustness of our approach. Our findings suggest that leveraging visual modality as a meaning representation provides a promising direction for robust natural language understanding.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2511.17358>)

[PDF](<https://arxiv.org/pdf/2511.17358.pdf>)
