# FACT-AUDIT: An Adaptive Multi-Agent Framework for Dynamic Fact-Checking Evaluation of Large Language Models Source: [FACT-AUDIT: An Adaptive Multi-Agent Framework for Dynamic Fact-Checking Evaluation of Large Language Models]() Collection index: [llms.txt]() Data updated: 2025-10-26T04:13:53.325Z Authors: Hongzhan Lin, Yang Deng, Yuxuan Gu, Wenxuan Zhang, Jing Ma, See-Kiong Ng, Tat-Seng Chua Published: 2025-02-25T07:44:22.000Z ## Abstract Large Language Models (LLMs) have significantly advanced the fact-checking studies. However, existing automated fact-checking evaluation methods rely on static datasets and classification metrics, which fail to automatically evaluate the justification production and uncover the nuanced limitations of LLMs in fact-checking. In this work, we introduce FACT-AUDIT, an agent-driven framework that adaptively and dynamically assesses LLMs' fact-checking capabilities. Leveraging importance sampling principles and multi-agent collaboration, FACT-AUDIT generates adaptive and scalable datasets, performs iterative model-centric evaluations, and updates assessments based on model-specific responses. By incorporating justification production alongside verdict prediction, this framework provides a comprehensive and evolving audit of LLMs' factual reasoning capabilities, to investigate their trustworthiness. Extensive experiments demonstrate that FACT-AUDIT effectively differentiates among state-of-the-art LLMs, providing valuable insights into model strengths and limitations in model-centric fact-checking analysis. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Claim Verification on AVeriTeC Golden (dev)](): Claim Verification; AVeriTeC Golden (dev); Accuracy: 81.8. - [Claim Verification on AVeriTeC Retrieved (I) (dev)](): Claim Verification; AVeriTeC Retrieved (I) (dev); Accuracy: 67.2. - [Claim Verification on AVeriTeC Retrieved (H) (dev)](): Claim Verification; AVeriTeC Retrieved (H) (dev); Accuracy: 64.4. ## Original sources and implementation [Original paper]() [PDF]()