# SoccerChat: Integrating Multimodal Data for Enhanced Soccer Game Understanding

Source: [SoccerChat: Integrating Multimodal Data for Enhanced Soccer Game Understanding](<https://www.sota2.com/research/paper/68fa78b31e209131cea79d09>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:57:16.741Z

Authors: Sushant Gautam, Cise Midoglu, Vajira Thambawita, Michael A. Riegler, P{\\aa}l Halvorsen, Mubarak Shah

Published: 2025-05-22T13:01:51.000Z

## Abstract

The integration of artificial intelligence in sports analytics has transformed soccer video understanding, enabling real-time, automated insights into complex game dynamics. Traditional approaches rely on isolated data streams, limiting their effectiveness in capturing the full context of a match. To address this, we introduce SoccerChat, a multimodal conversational AI framework that integrates visual and textual data for enhanced soccer video comprehension. Leveraging the extensive SoccerNet dataset, enriched with jersey color annotations and automatic speech recognition (ASR) transcripts, SoccerChat is fine-tuned on a structured video instruction dataset to facilitate accurate game understanding, event classification, and referee decision making. We benchmark SoccerChat on action classification and referee decision-making tasks, demonstrating its performance in general soccer event comprehension while maintaining competitive accuracy in referee decision making. Our findings highlight the importance of multimodal integration in advancing soccer analytics, paving the way for more interactive and explainable AI-driven sports analysis. https://github.com/simula/SoccerChat

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Sports Video Understanding on DeepSport (test)](<https://www.sota2.com/research/sota/sports-video-understanding-on-deepsport-test.md>): Sports Video Understanding; DeepSport (test); Fine-Grained Recognition Accuracy: 31.43.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2505.16630>)

[PDF](<https://arxiv.org/pdf/2505.16630.pdf>)