# Knowledge-Graph Grounding Helps LLMs Only for Out-of-Training Knowledge: A Controlled Study on Clinical Question Answering

Source: [Knowledge-Graph Grounding Helps LLMs Only for Out-of-Training Knowledge: A Controlled Study on Clinical Question Answering](<https://www.sota2.com/research/paper/6a3a1387206542251d7c75ce>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-25T05:00:34.132Z

Authors: Madhulatha Mandarapu, Sandeep Kunkunuru

Published: 2026-06-21T10:12:17.000Z

## Abstract

A recent Nature Medicine study reports that general-purpose frontier LLMs outperform specialized retrieval-augmented clinical tools on medical benchmarks, and that retrieval can hurt strong models. We ask the natural follow-up: does structured knowledge-graph (KG) grounding change this, and when does grounding help at all? We contribute two results. First, a reproduction: the study's headline HealthBench score (~88) is the Consensus variant, not full HealthBench, where frontier models and ideal completions both score ~46-47 under a physician-calibrated grader (agreement 82.5%); we reproduce GPT-5.2 Consensus =90.9 and flag a score-deflating grader bug. Second, a knowledge-boundary result. Using a graph+vector engine (samyama-graph) over the public biomedical KG PrimeKG, neither naive triple retrieval nor an agentic natural-language-to-Cypher loop (82% successful queries) improves MedQA across a weak-to-strong model ladder (all \|Delta\| \<= 3.4). On a synthetic counterfactual KG, and on a hybrid benchmark mixing known and novel facts, the identical pipeline lifts out-of-training accuracy from chance to ~100% (+68 to +79) while adding nothing on known facts (a no-LLM arm answers both). Across three regimes (no-knowledge, graph-aided, hybrid), grounding helps only insofar as the decisive fact lies outside the model's training -- public-KG facts are redundant, private and novel data are where it pays -- matching the study's institutional-data caveat.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Medical Question Answering on HealthBench Consensus](<https://www.sota2.com/research/sota/medical-question-answering-on-healthbench-consensus.md>): Medical Question Answering; HealthBench Consensus; Score: not recorded.

- [Medical Question Answering on HealthBench oss\_eval (full)](<https://www.sota2.com/research/sota/medical-question-answering-on-healthbench-oss_eval-full.md>): Medical Question Answering; HealthBench oss\_eval (full); Overall Score: not recorded.

- [Medical Question Answering on MedQA (matched 500)](<https://www.sota2.com/research/sota/medical-question-answering-on-medqa-matched-500.md>): Medical Question Answering; MedQA (matched 500); Score: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.22419>)

[PDF](<https://arxiv.org/pdf/2606.22419.pdf>)