# Exploring Self-Supervised Regularization for Supervised and Semi-Supervised Learning Source: [Exploring Self-Supervised Regularization for Supervised and Semi-Supervised Learning]() Collection index: [llms.txt]() Data updated: 2026-01-04T03:09:14.794Z Authors: Phi Vu Tran Published: 2019-06-25T06:42:05.000Z ## Abstract Recent advances in semi-supervised learning have shown tremendous potential in overcoming a major barrier to the success of modern machine learning algorithms: access to vast amounts of human-labeled training data. Previous algorithms based on consistency regularization can harness the abundance of unlabeled data to produce impressive results on a number of semi-supervised benchmarks, approaching the performance of strong supervised baselines using only a fraction of the available labeled data. In this work, we challenge the long-standing success of consistency regularization by introducing self-supervised regularization as the basis for combining semantic feature representations from unlabeled data. We perform extensive comparative experiments to demonstrate the effectiveness of self-supervised regularization for supervised and semi-supervised image classification on SVHN, CIFAR-10, and CIFAR-100 benchmark datasets. We present two main results: (1) models augmented with self-supervised regularization significantly improve upon traditional supervised classifiers without the need for unlabeled data; (2) together with unlabeled data, our models yield semi-supervised performance competitive with, and in many cases exceeding, prior state-of-the-art consistency baselines. Lastly, our models have the practical utility of being efficiently trained end-to-end and require no additional hyper-parameters to tune for optimal performance beyond the standard set for training neural networks. Reference code and data are available at https://github.com/vuptran/sesemi ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on CIFAR-100 (50,000 labels) + Tiny Images (500,000 unlabeled)](): Image Classification; CIFAR-100 (50,000 labels) + Tiny Images (500,000 unlabeled); Error Rate: 22.49. - [Image Classification on CIFAR-100 (50k labels) + Tiny Images (237k unlabeled)](): Image Classification; CIFAR-100 (50,000 labels) + Tiny Images (237,203 unlabeled); Error Rate: 22.49. - [Image Classification on CIFAR-10 (test) (Error Rate)](): Image Classification; CIFAR-10 (test); Error Rate: 4.7. - [Image Classification on CIFAR-100 (test)](): Image Classification; CIFAR-100 (test); Accuracy: not recorded. - [Image Classification on CIFAR-100](): Image Classification; CIFAR-100; Accuracy: not recorded. - [Image Classification on SVHN (test)](): Image Classification; SVHN (test); Accuracy: not recorded. ## Original sources and implementation [Original paper]() [PDF]() [Implementation]()