# Leveraging LLMs in Scholarly Knowledge Graph Question Answering

Source: [Leveraging LLMs in Scholarly Knowledge Graph Question Answering](<https://www.sota2.com/research/paper/68fa713a1e209131cea4fe84>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T02:00:40.638Z

Authors: Tilahun Abedissa Taffa, Ricardo Usbeck

Published: 2023-11-16T12:13:49.000Z

## Abstract

This paper presents a scholarly Knowledge Graph Question Answering (KGQA) that answers bibliographic natural language questions by leveraging a large language model (LLM) in a few-shot manner. The model initially identifies the top-n similar training questions related to a given test question via a BERT-based sentence encoder and retrieves their corresponding SPARQL. Using the top-n similar question-SPARQL pairs as an example and the test question creates a prompt. Then pass the prompt to the LLM and generate a SPARQL. Finally, runs the SPARQL against the underlying KG - ORKG (Open Research KG) endpoint and returns an answer. Our system achieves an F1 score of 99.0%, on SciQA - one of the Scholarly-QALD-23 challenge benchmarks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2311.09841>)

[PDF](<https://arxiv.org/pdf/2311.09841.pdf>)