# From 3D Pose to Prose: Biomechanics-Grounded Vision--Language Coaching

Source: [From 3D Pose to Prose: Biomechanics-Grounded Vision--Language Coaching](<https://www.sota2.com/research/paper/69cb55250636e5bf4afc9fdf>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-01T05:00:04.436Z

Authors: Yuyang Ji, Yixuan Shen, Shengjie Zhu, Yu Kong, Feng Liu

Published: 2026-03-27T19:26:28.000Z

## Abstract

We present BioCoach, a biomechanics-grounded vision--language framework for fitness coaching from streaming video. BioCoach fuses visual appearance and 3D skeletal kinematics, through a novel three-stage pipeline: an exercise-specific degree-of-freedom selector that focuses analysis on salient joints; a structured biomechanical context that pairs individualized morphometrics with cycle and constraint analysis; and a vision--biomechanics conditioned feedback module that applies cross-attention to generate precise, actionable text. Using parameter-efficient training that freezes the vision and language backbones, BioCoach yields transparent, personalized reasoning rather than pattern matching. To enable learning and fair evaluation, we augment QEVD-fit-coach with biomechanics-oriented feedback to create QEVD-bio-fit-coach, and we introduce a biomechanics-aware LLM judge metric. BioCoach delivers clear gains on QEVD-bio-fit-coach across lexical and judgment metrics while maintaining temporal triggering; on the original QEVD-fit-coach, it improves text quality and correctness with near-parity timing, demonstrating that explicit kinematics and constraints are key to accurate, phase-aware coaching.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Video-to-Feedback Generation on QEVD-fit-coach (original annotations)](<https://www.sota2.com/research/sota/video-to-feedback-generation-on-qevd-fit-coach-original-annotations.md>): Video-to-Feedback Generation; QEVD-fit-coach (original annotations); METEOR: 12.9.

- [Streaming Biomechanical Feedback Generation on QEVD bio-fit-coach](<https://www.sota2.com/research/sota/streaming-biomechanical-feedback-generation-on-qevd-bio-fit-coach.md>): Streaming Biomechanical Feedback Generation; QEVD bio-fit-coach; METEOR Score: 0.312.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2603.26938>)

[PDF](<https://arxiv.org/pdf/2603.26938.pdf>)