# Toward Mining Visual Log of Software

Source: [Toward Mining Visual Log of Software](<https://www.sota2.com/research/paper/68fa62d81e209131ce9fb8ed>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:28:18.594Z

Authors: Hung Pham, Tam Nguyen, Phong Vu, Tung Nguyen

Published: 2016-10-27T18:13:54.000Z

## Abstract

In this paper, we define visual log of a software system as data capturing the interactions between its users and its graphic user interface (GUI), such as screen-shots and screen recordings. We vision that mining such visual log could be useful for bug reproducing and debugging, automated GUI testing, user interface designing, question answering of common usages in software support, etc. Toward that vision, we propose a core framework for mining visual log of software. This framework focuses on detecting GUI elements and changes in visual log, removing users' private data, recognizing user interactions with GUI elements, and learning GUI usage patterns. We also performed a small study on the characteristics of GUI elements in mobile apps. The findings from this study suggested several heuristics to design techniques for recognizing GUI elements and interactions.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1610.08911>)

[PDF](<https://arxiv.org/pdf/1610.08911.pdf>)
