A study on energy aware supervised learning of cortical algorithms -

Abstract

Cortical algorithms (CA)s modeled by MountCastle after the human visual cortex, have superior performance when compared to the first and second artificial neural networks generations. However, despite their superior “hypothetical” performance CAs remain less widely used due to their expensive long training and computational requirements which would prevent them from running in an online learning mode on energy aware computing nodes and on large datasets for applications restricting the training time of the ML model. Motivated to reduce CA supervised training complexity with minimal impact of accuracy; we present in this work a new soft weight update rule based on the linearization of the exponential weight update function. Our suggested approach leads to a major reduction in CA training and complexity with a minimal impact on performance as empirically and theoretically proven. Experimental results on 6 publicly available databases show that the proposed rule is capable of reducing the required training time by an average of 50percent at the expense of a minor degradation in the performance not to exceed 0.7percent. The improvement introduced by this rule is further demonstrated by a major reduction (about 50percent) in the number of required computations.. In addition, we propose an entropy-based cost function, and an entropy based weight update rule for CA and test its generalization capabilities on 2 publicly available datasets. Experimental results show that the proposed entropy-based rule outperforms the original exponential weight update rule. Finally in effort to make the overall supervised training process of CA more energy aware especially in the context of non-informative features, we test our proposed soft weight update rule together with the entropy-based concepts in some fixed and variable feature reduction schemes on 4 publicly available databases. Experimental results on these databases validate the merit of our proposed approach in providing an efficient feature selection coupled with an improvement i