Semi-Supervised Learning with Ladder Networks

About

We combine supervised learning with unsupervised learning in deep neural networks. The proposed model is trained to simultaneously minimize the sum of supervised and unsupervised cost functions by backpropagation, avoiding the need for layer-wise pre-training. Our work builds on the Ladder network proposed by Valpola (2015), which we extend by combining the model with supervision. We show that the resulting model reaches state-of-the-art performance in semi-supervised MNIST and CIFAR-10 classification, in addition to permutation-invariant MNIST classification with all labels.

Antti Rasmus, Harri Valpola, Mikko Honkala, Mathias Berglund, Tapani Raiko · 2015

Related benchmarks

Task	Dataset	Result
Image Classification	CIFAR-10 (test)	Accuracy—	3,381
Image Classification	MNIST (test)	Accuracy—	894
Image Classification	CIFAR-100	Accuracy—	691
Image Classification	CIFAR-10	Accuracy—	507
Image Classification	CIFAR-100 (test)	Top-1 Accuracy—	429
Image Classification	SVHN	Accuracy92.5	395
Image Classification	STL-10 (test)	Accuracy—	380
Image Classification	CIFAR100	Accuracy62.1	378
Image Classification	CIFAR-100 standard (test)	Top-1 Accuracy—	195
Image Classification	STL-10	Top-1 Accuracy—	146