Enhanced Image Classification With a Fast-Learning Shallow Convolutional Neural Network
About
We present a neural network architecture and training method designed to enable very rapid training and low implementation complexity. Due to its training speed and very few tunable parameters, the method has strong potential for applications requiring frequent retraining or online training. The approach is characterized by (a) convolutional filters based on biologically inspired visual processing filters, (b) randomly-valued classifier-stage input weights, (c) use of least squares regression to train the classifier output weights in a single batch, and (d) linear classifier-stage output units. We demonstrate the efficacy of the method by applying it to image classification. Our results match existing state-of-the-art results on the MNIST (0.37% error) and NORB-small (2.2% error) image classification databases, but with very fast training times compared to standard deep network approaches. The network's performance on the Google Street View House Number (SVHN) (4% error) database is also competitive with state-of-the art methods.
Key insights
What problem does this shallow CNN solve?
Deep networks train slowly, hindering online or frequent retraining. The paper combines fixed convolutional filters, random classifier input weights, and single-batch least-squares output-weight training to enable fast learning with few tunable parameters.
How does it differ from prior work?
Unlike deep networks, only the output layer is trained, via least squares. On MNIST it achieved 0.37% error versus 0.39% prior best without augmentation; on NORB-small, 2.21% versus 2.53%, across four benchmark datasets.
Where could fast retraining be useful?
The authors state the method's training speed and few tunable parameters give strong potential for applications requiring frequent retraining or online training. They demonstrate benchmarks only; no production deployment is documented.
What limitations do the authors note?
The method is not competitive on CIFAR-10 (24.14% vs 9.78% state-of-the-art), with up to 20% train-test gap. Authors suggest future work: limited Stage 1 filter training and data augmentation to address harder datasets.