# LLM-Based Visualization Evaluation: How Well Do Literacy-Stratified Personas Approximate Human Judgments? Source: [LLM-Based Visualization Evaluation: How Well Do Literacy-Stratified Personas Approximate Human Judgments?]() Collection index: [llms.txt]() Data updated: 2026-06-11T05:00:05.512Z Authors: Swaroop Panda Published: 2026-06-08T19:21:32.000Z ## Abstract Evaluating data visualizations across diverse user populations continues to pose a significant methodological challenge within visualization research. We propose a theorized evaluation framework, Literacy-Stratified LLM Evaluation (LSLE), which formalizes a two-stage process. The first stage involves constructing visualization literacy personas grounded in established frameworks such as VLAT. The second stage directs large language models to adopt these personas as simulated evaluators of visualization artifacts. We ground the framework in an epistemic analysis that characterizes the conditions under which LLM persona simulation may produce plausible proxies for literacy-dependent perception - and, critically, the conditions under which it does not - engaging directly with emerging critiques of LLM-as-participant paradigms from the VIS and HCI literature. To empirically test LSLE's boundaries, we benchmark its outputs against openly available human response data from the validation studies of two established instruments: VLAT and BeauVIS. Using the same stimuli and assessment items as the original human studies, we compare LSLE persona responses across literacy strata against published human distributions and against default (non-persona) LLM baselines. Our analysis reveals where literacy-stratified personas converge with and diverge from human response patterns - identifying task types and evaluation dimensions where persona simulation approximates human variability and where it systematically fails. We discuss implications for the responsible use of LLM-assisted evaluation as a complement to empirical methods, and propose boundary conditions for when LSLE may be most appropriate: early-stage design exploration and rapid comparative screening rather than summative evaluation. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Visualization Literacy on VLAT 53 items](): Visualization Literacy; VLAT 53 items; Mean Proportion Correct: 98. - [Item Difficulty Rank-Order Correlation on VLAT 53 items](): Item Difficulty Rank-Order Correlation; VLAT 53 items; Spearman's Rho: 0.44. - [Aesthetic rating on Beauvis Beamtree low stimulus](): Aesthetic rating; Beauvis Beamtree low stimulus; Composite Rating: 4.4. - [Aesthetic rating on Beauvis Star tree med stimulus](): Aesthetic rating; Beauvis Star tree med stimulus; Composite Aesthetic Score: 5. - [Aesthetic rating on Beauvis Sunburst high stimulus](): Aesthetic rating; Beauvis Sunburst high stimulus; Composite Aesthetic Score: 5.47. ## Original sources and implementation [Original paper]() [PDF]()