Biologically inspired incremental learning for high-dimensional spaces

Alexander Gepperth (1, 2) , Thomas Hecht (1) , Mathieu Lefort (1, 2) , Ursula Körner (3)

1 RV - Robotique et Vision
2 Flowers - Flowing Epigenetic Robots and Systems
3 HRI - Honda Research Institute Europe GmbH

We propose an incremental, highly parallelizable, and constant-time complexity neural learning architecture for multi-class classification (and regression) problems that remains resource-efficient even when the number of input dimensions is very high (≥ 1000). This so-called projection-prediction (PROPRE) architecture is strongly inspired by biological information processing in that it uses a prototype-based, topologically organized hidden layer trained with the SOM learning rule that updates hidden layer weights whenever an error occurs. The SOM learning adapts only the weights of localized neural sub-populations that are similar to the input, which explicitly avoids the catastrophic forgetting effect of MLPs in case new input statistics are presented. The readout layer applies linear regression to hidden layer activities subjected to a transfer function, making the whole system capable of representing strongly non-linear decision boundaries. The resource-efficiency of the algorithm stems from the fact of approximating similarity in the input space by proximity in the SOM layer due to the topological SOM projection property. This avoids the storage of inter-cluster distances (quadratic in number of hidden layer) or input space covariance matrices (quadratic in input dimensions) as K-means, RBF or LWPR would have to do. Tests on the popular MNIST handwritten digit benchmark show that the algorithm compares favorably to state-of-the-art results, and parallelizability is demonstrated by analyzing the efficiency of a parallel GPU implementation of the architecture.

Domaines: Apprentissage [cs.LG]

Alexander Gepperth, Thomas Hecht, Mathieu Lefort, Ursula Körner. Biologically inspired incremental learning for high-dimensional spaces. Joint IEEE International Conference Developmental Learning and Epigenetic Robotics (ICDL-EPIROB), Sep 2015, Providence, United States. ⟨10.1109/DEVLRN.2015.7346155⟩. ⟨hal-01250961⟩