# SUMMIR: A Hallucination-Aware Framework for Ranking Sports Insights from LLMs

Source: [SUMMIR: A Hallucination-Aware Framework for Ranking Sports Insights from LLMs](<https://www.sota2.com/research/paper/69d5e0de7d0172140ec758bf>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-09T05:00:04.534Z

Authors: Nitish Kumar, Sannu Kumar, S Akash, Manish Gupta, Ankith Karat, Sriparna Saha

Published: 2026-03-30T12:45:15.000Z

## Abstract

With the rapid proliferation of online sports journalism, extracting meaningful pre-game and post-game insights from articles is essential for enhancing user engagement and comprehension. In this paper, we address the task of automatically extracting such insights from articles published before and after matches. We curate a dataset of 7,900 news articles covering 800 matches across four major sports: Cricket, Soccer, Basketball, and Baseball. To ensure contextual relevance, we employ a two-step validation pipeline leveraging both open-source and proprietary large language models (LLMs). We then utilize multiple state-of-the-art LLMs (GPT-4o, Qwen2.5-72B-Instruct, Llama-3.3-70B-Instruct, and Mixtral-8x7B-Instruct-v0.1) to generate comprehensive insights. The factual accuracy of these outputs is rigorously assessed using a FactScore-based methodology, complemented by hallucination detection via the SummaC (Summary Consistency) framework with GPT-4o. Finally, we propose SUMMIR (Sentence Unified Multimetric Model for Importance Ranking), a novel architecture designed to rank insights based on user-specific interests. Our results demonstrate the effectiveness of this approach in generating high-quality, relevant insights, while also revealing significant differences in factual consistency and interestingness across LLMs. This work contributes a robust framework for automated, reliable insight generation from sports news content. The source code is availble here https://github.com/nitish-iitp/SUMMIR.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Insight Ranking on Sports Insights](<https://www.sota2.com/research/sota/insight-ranking-on-sports-insights.md>): Insight Ranking; Sports Insights; NDCG@2: 0.858.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.04947>)

[PDF](<https://arxiv.org/pdf/2604.04947.pdf>)
