Facial expression and attributes recognition based on multi-task learning of lightweight neural networks
About
In this paper, the multi-task learning of lightweight convolutional neural networks is studied for face identification and classification of facial attributes (age, gender, ethnicity) trained on cropped faces without margins. The necessity to fine-tune these networks to predict facial expressions is highlighted. Several models are presented based on MobileNet, EfficientNet and RexNet architectures. It was experimentally demonstrated that they lead to near state-of-the-art results in age, gender and race recognition on the UTKFace dataset and emotion classification on the AffectNet dataset. Moreover, it is shown that the usage of the trained models as feature extractors of facial regions in video frames leads to 4.5% higher accuracy than the previously known state-of-the-art single models for the AFEW and the VGAF datasets from the EmotiW challenges. The models and source code are publicly available at https://github.com/HSE-asavchenko/face-emotion-recognition.
Key insights
What problem does this multi-task face model solve?
It trains lightweight CNNs to jointly predict age, gender, ethnicity, and emotion from tightly cropped faces, arguing that fine-tuning face-recognition features is necessary for accurate expression classification.
How does it differ from prior work?
Unlike complex ensembles, it uses a simplified sequential pipeline on MobileNet, EfficientNet, and RexNet, achieving 59.27% AFEW and 69.84% VGAF accuracy, about 4.5% above prior single models.
Where can these models be deployed?
The paper documents an Android demo application and notes suitability for mobile and embedded systems, but reports only benchmark results, not production deployment.
What limitations do the authors acknowledge?
They note that only traditional classifiers like SVMs were used on extracted features, so not all results match state-of-the-art, and suggest future work with graph convolutional networks or transformers for improved quality.