# CommAI: Evaluating the first steps towards a useful general AI

Source: [CommAI: Evaluating the first steps towards a useful general AI](<https://www.sota2.com/research/paper/68fa63391e209131ce9fd98c>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:36:15.201Z

Authors: Marco Baroni, Armand Joulin, Allan Jabri, Germ\\\`an Kruszewski, Angeliki Lazaridou, Klemen Simonic, Tomas Mikolov

Published: 2017-01-31T09:20:17.000Z

## Abstract

With machine learning successfully applied to new daunting problems almost every day, general AI starts looking like an attainable goal. However, most current research focuses instead on important but narrow applications, such as image classification or machine translation. We believe this to be largely due to the lack of objective ways to measure progress towards broad machine intelligence. In order to fill this gap, we propose here a set of concrete desiderata for general AI, together with a platform to test machines on how well they satisfy such desiderata, while keeping all further complexities to a minimum.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1701.08954>)

[PDF](<https://arxiv.org/pdf/1701.08954.pdf>)