We describe a mathematical framework that provides a hierarchical architecture for learning and cognition. It combines a wavelet preprocessor, a group invariant feature extractor and a hierarchical learning algorithm.  There are two feedback loops, one to the feature extractor and one to the wavelet preprocessor. It can incorporate metric and non-metric dissimilarity measures like Bregman divergences. Two universal learning algorithms are used: Learning Vector Quantization for supervised and Self-Organizing Map for unsupervised learning. We demonstrate its superior performance on a variety of practical problems. The underlying mathematics support design and evaluation of deep neural networks. We close with current work on microelectronic implementations that emulate architectural abstractions of the cortex of higher-level animals and humans w.r.t. to sound and vision sensing and cognition -- “Cortex-on-a-Chip”.