# Enhancing Visual Programming for Visual Reasoning via Probabilistic Graphs Source: [Enhancing Visual Programming for Visual Reasoning via Probabilistic Graphs]() Collection index: [llms.txt]() Data updated: 2025-12-18T05:00:04.345Z Authors: Wentao Wan, Kaiyu Wu, Qingyang Ma, Nan Kang, Yunjie Chen, Liang Lin, Keze Wang Published: 2025-12-16T10:07:40.000Z ## Abstract Recently, Visual Programming (VP) based on large language models (LLMs) has rapidly developed and demonstrated significant potential in complex Visual Reasoning (VR) tasks. Previous works to enhance VP have primarily focused on improving the quality of LLM-generated visual programs. However, they have neglected to optimize the VP-invoked pre-trained models, which serve as modules for the visual sub-tasks decomposed from the targeted tasks by VP. The difficulty is that there are only final labels of targeted VR tasks rather than labels of sub-tasks. Besides, the non-differentiable nature of VP impedes the direct use of efficient gradient-based optimization methods to leverage final labels for end-to-end learning of the entire VP framework. To overcome these issues, we propose EVPG, a method to Enhance Visual Programming for visual reasoning via Probabilistic Graphs. Specifically, we creatively build a directed probabilistic graph according to the variable dependency relationships during the VP executing process, which reconstructs the non-differentiable VP executing process into a differentiable exact probability inference process on this directed probabilistic graph. As a result, this enables the VP framework to utilize the final labels for efficient, gradient-based optimization in end-to-end supervised learning on targeted VR tasks. Extensive and comprehensive experiments demonstrate the effectiveness and advantages of our EVPG, showing significant performance improvements for VP on three classical complex VR tasks: GQA, NLVRv2, and Open Images. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Visual Reasoning on GQA balanced (test-dev)](): Visual Reasoning; GQA balanced (test-dev); Accuracy: 58.9. - [Visual Question Answering on Open Images cross-task (test)](): Visual Question Answering; Open Images cross-task (test); Accuracy: 44.7. - [Visual Reasoning on Open Images (test)](): Visual Reasoning; Open Images (test); Accuracy: 85.1. - [Visual Reasoning on NLVR2 (test)](): Visual Reasoning; NLVR2 (test); Accuracy: 81.2. - [Visual Question Answering on GQA (Accuracy)](): Visual Question Answering; GQA; Accuracy: 58. ## Original sources and implementation [Original paper]() [PDF]()