# AQuA: Automated Question-Answering in Software Tutorial Videos with Visual Anchors

Source: [AQuA: Automated Question-Answering in Software Tutorial Videos with Visual Anchors](<https://www.sota2.com/research/paper/68fa727c1e209131cea56b17>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T06:31:06.454Z

Authors: Saelyne Yang, Jo Vermeulen, George Fitzmaurice, Justin Matejka

Published: 2024-03-08T10:55:43.000Z

## Abstract

Tutorial videos are a popular help source for learning feature-rich software. However, getting quick answers to questions about tutorial videos is difficult. We present an automated approach for responding to tutorial questions. By analyzing 633 questions found in 5,944 video comments, we identified different question types and observed that users frequently described parts of the video in questions. We then asked participants (N=24) to watch tutorial videos and ask questions while annotating the video with relevant visual anchors. Most visual anchors referred to UI elements and the application workspace. Based on these insights, we built AQuA, a pipeline that generates useful answers to questions with visual anchors. We demonstrate this for Fusion 360, showing that we can recognize UI elements in visual anchors and generate answers using GPT-4 augmented with that visual information and software documentation. An evaluation study (N=16) demonstrates that our approach provides better answers than baseline methods.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2403.05213>)

[PDF](<https://arxiv.org/pdf/2403.05213.pdf>)