# Deep learning evaluation using deep linguistic processing

Source: [Deep learning evaluation using deep linguistic processing](<https://www.sota2.com/research/paper/68fa64631e209131cea043f3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:02:11.964Z

Authors: Alexander Kuhnle, Ann Copestake

Published: 2017-06-05T13:53:56.000Z

## Abstract

We discuss problems with the standard approaches to evaluation for tasks like visual question answering, and argue that artificial data can be used to address these as a complement to current practice. We demonstrate that with the help of existing 'deep' linguistic processing technology we are able to create challenging abstract datasets, which enable us to investigate the language understanding abilities of multimodal deep learning models in detail, as compared to a single performance value on a static and monolithic dataset.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1706.01322>)

[PDF](<https://arxiv.org/pdf/1706.01322.pdf>)