# Cross-Modal Perceptionist: Can Face Geometry be Gleaned from Voices? Source: [Cross-Modal Perceptionist: Can Face Geometry be Gleaned from Voices?]() Collection index: [llms.txt]() Data updated: 2026-01-04T03:40:42.831Z Authors: Cho-Ying Wu, Chin-Cheng Hsu, Ulrich Neumann Published: 2022-03-18T10:03:07.000Z ## Abstract This work digs into a root question in human perception: can face geometry be gleaned from one's voices? Previous works that study this question only adopt developments in image synthesis and convert voices into face images to show correlations, but working on the image domain unavoidably involves predicting attributes that voices cannot hint, including facial textures, hairstyles, and backgrounds. We instead investigate the ability to reconstruct 3D faces to concentrate on only geometry, which is much more physiologically grounded. We propose our analysis framework, Cross-Modal Perceptionist, under both supervised and unsupervised learning. First, we construct a dataset, Voxceleb-3D, which extends Voxceleb and includes paired voices and face meshes, making supervised learning possible. Second, we use a knowledge distillation mechanism to study whether face geometry can still be gleaned from voices without paired voices and 3D face data under limited availability of 3D face scans. We break down the core question into four parts and perform visual and numerical analyses as responses to the core question. Our findings echo those in physiology and neuroscience about the correlation between voices and facial structures. The work provides future human-centric cross-modal learning with explainable foundations. See our project page: https://choyingw.github.io/works/Voice2Mesh/index.html ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [3D Face Reconstruction on Voxceleb-3D (test)](): 3D Face Reconstruction; Voxceleb-3D (test); RMSE: 1.21. - [3D Face Reconstruction from Voice on Voxceleb-3D](): 3D Face Reconstruction from Voice; Voxceleb-3D; Error Rate (ER): 1.52. - [3D Facial Landmark Alignment on BFM Face 68 facial landmarks (test)](): 3D Facial Landmark Alignment; BFM Face 68 facial landmarks (test); NME: 0.2723. ## Original sources and implementation [Original paper]() [PDF]()