Automatic Construction of Pattern Classifiers Capable of Continuous Incremental Learning and Unlearning Tasks Based on Compact-Sized Probabilistic Neural Network
About
This paper proposes a novel approach to pattern classification using a probabilistic neural network model. The strategy is based on a compact-sized probabilistic neural network capable of continuous incremental learning and unlearning tasks. The network is constructed/reconstructed using a simple, one-pass network-growing algorithm with no hyperparameter tuning. Then, given the training dataset, its structure and parameters are automatically determined and can be dynamically varied in continual incremental and decremental learning situations. The algorithm proposed in this work involves no iterative or arduous matrix-based parameter approximations but a simple data-driven updating scheme. Simulation results using nine publicly available databases demonstrate the effectiveness of this approach, showing that compact-sized probabilistic neural networks constructed have a much smaller number of hidden units compared to the original probabilistic neural network model and yet can achieve a similar classification performance to that of multilayer perceptron neural networks in standard classification tasks, while also exhibiting sufficient capability in continuous class incremental learning and unlearning tasks.
Key insights
How does this paper build classifiers that learn and forget dynamically?
It proposes a compact probabilistic neural network (CS-PNN) built by a one-pass algorithm needing no hyperparameter tuning. The network grows or shrinks dynamically for incremental learning and unlearning without iterative matrix operations.
How does CS-PNN compare to prior PNN and DNN methods?
On nine public datasets, CS-PNN matched original PNN accuracy using 4 to 46% of hidden units. In class-incremental learning, it outperformed a DNN using iCaRL, which degraded severely as new classes increased.
What practical use does the CS-PNN support?
The paper demonstrates a prototype benchmark for flexible pattern recognition requiring continual class addition and removal. It documents no production deployment, but suggests CS-PNN could provide a rapid engine for higher-level intelligent systems.
What limitations and future work do the authors identify?
Evaluation was restricted to nine datasets with relatively few classes. CS-PNN used more hidden units than DNNs in standard tasks. Future work will investigate larger databases, more classes, and parallel implementation for faster testing.