Human-centered evaluation of data visualization understanding in vision-language models

Article Open access Published: 19 October 2026 Scientific Reports ( 2026 ) Cite this article We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record.

All legal disclaimers apply. Abstract Data visualization understanding, which requires integrating visual, numerical, and linguistic inputs, poses fundamental challenges to multimodal reasoning. Recently developed vision-language models (VLMs) are promising candidates for modeling the computations that underlie human data visualization understanding.

However, whether they are useful as scientific models depends not only on whether they produce correct answers, but also on whether they approximate the patterns of responses that people produce on the same tasks. This distinction has been ignored in recent evaluations of VLMs, which typically only report overall model performance (accuracy) without direct comparison to human behavioral patterns .

Here we introduce a human-centered evaluation protocol that compares VLM and human responses on the same set of visualization understanding items. We apply this protocol to eight VLMs across six assessments: five established tests of human visualization literacy, plus one designed specifically for evaluating VLMs.

Models performed worse than humans on average, and this gap persisted even under lenient criteria. Moreover, while relative performance across items was somewhat correlated between models and humans, all models produced error patterns that were reliably distinct from those of humans.

We envision our evaluation protocol helping to identify artificial systems that might serve as promising scientific cognitive models of how humans reason about data visualizations. Acknowledgements We thank members of the Cognitive Tools Lab for their input and feedback. Funding This work was supported by NSF CAREER Award #2436199, NSF DRL #2400471, and awards from ONR Science of Autonomy, the Stanford Human-Centered AI Institute (HAI), and Stanford Accelerator for Learning.

Author information Authors and Affiliations Stanford University, Santa Clara, USA Arnav Verma, Kushin Mukherjee, Christopher Potts & Judith E. Fan Massachusetts Institute of Technology, Cambridge, USA Arnav Verma University of California, Los Angeles, USA Elisa Kreiss Authors Arnav Verma Kushin Mukherjee Christopher Potts Elisa Kreiss Judith E.

Fan Corresponding authors Correspondence to Arnav Verma or Judith E. Fan . Ethics declarations Competing interests The authors declare no competing interests.

Additional information Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Rights and permissions Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material.

You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material.

If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions About this article Cite this article Verma, A., Mukherjee, K., Potts, C. et al. Human-centered evaluation of data visualization understanding in vision-language models. Sci Rep (2026). https://doi.org/10.1038/s41598-026-68119-z Download citation Received : 08 October 2025 Accepted : 19 August 2026 Published : 19 October 2026 DOI : https://doi.org/10.1038/s41598-026-68119-z Keywords