# LingoQA: Visual Question Answering for Autonomous Driving Source: [LingoQA: Visual Question Answering for Autonomous Driving]() Collection index: [llms.txt]() Data updated: 2025-10-27T06:40:00.946Z Authors: Ana-Maria Marcu, Long Chen, Jan H\\"unermann, Alice Karnsund, Benoit Hanotte, Prajwal Chidananda, Saurabh Nair, Vijay Badrinarayanan, Alex Kendall, Jamie Shotton, Elahe Arani, Oleg Sinavski Published: 2023-12-21T18:40:34.000Z ## Abstract We introduce LingoQA, a novel dataset and benchmark for visual question answering in autonomous driving. The dataset contains 28K unique short video scenarios, and 419K annotations. Evaluating state-of-the-art vision-language models on our benchmark shows that their performance is below human capabilities, with GPT-4V responding truthfully to 59.6% of the questions compared to 96.6% for humans. For evaluation, we propose a truthfulness classifier, called Lingo-Judge, that achieves a 0.95 Spearman correlation coefficient to human evaluations, surpassing existing techniques like METEOR, BLEU, CIDEr, and GPT-4. We establish a baseline vision-language model and run extensive ablation studies to understand its performance. We release our dataset and benchmark as an evaluation platform for vision-language models in autonomous driving. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Video Question Answering on LingoQA (test)](): Video Question Answering; LingoQA (test); Ling-Judge: 60.8. - [Autonomous Driving Question Answering on LingoQA (val)](): Autonomous Driving Question Answering; LingoQA (val); Lingo-J: 60.8. ## Original sources and implementation [Original paper]() [PDF]()