# Using Captum to Explain Generative Language Models Source: [Using Captum to Explain Generative Language Models]() Collection index: [llms.txt]() Data updated: 2025-10-26T02:05:39.468Z Authors: Vivek Miglani, Aobo Yang, Aram H. Markosyan, Diego Garcia-Olano, Narine Kokhlikyan Published: 2023-12-09T07:35:24.000Z ## Abstract Captum is a comprehensive library for model explainability in PyTorch, offering a range of methods from the interpretability literature to enhance users' understanding of PyTorch models. In this paper, we introduce new features in Captum that are specifically designed to analyze the behavior of generative language models. We provide an overview of the available functionalities and example applications of their potential for understanding learned associations within generative language models. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Explanation Generation on Alpaca avg prompt instance](): Explanation Generation; Alpaca avg prompt instance; Inference Time (s): 1169.648. - [Explanation Generation on tldr\_news avg prompt instance](): Explanation Generation; tldr\_news avg prompt instance; Latency (s): 1727.602. - [Explanation Generation on MHC avg prompt instance](): Explanation Generation; MHC avg prompt instance; Time (s): 1806.551. - [Faithfulness Measurement on MHC](): Faithfulness Measurement; MHC; BLEU: 68.8. - [Faithfulness Measurement on tldr\_news](): Faithfulness Measurement; tldr\_news; BLEU: 75.9. - [Faithfulness Measurement on Alpaca](): Faithfulness Measurement; Alpaca; BLEU: 0.515. ## Original sources and implementation [Original paper]() [PDF]()