CNN as Guided Multi-layer RECOS Transform

Abstract: There is a resurging interest in developing a neural-network-based solution to the supervised machine learning problem. The convolutional neural network (CNN) will be studied in this note. To begin with, we introduce a RECOS transform as a basic building block of CNNs. The "RECOS" is an acronym for "REctified-COrrelations on a Sphere". It consists of two main concepts: 1) data clustering on a sphere and 2) rectification. Afterwards, we interpret a CNN as a network that implements the guided multi-layer RECOS transform with three highlights. First, we compare the traditional single-layer and modern multi-layer signal analysis approaches, point out key ingredients that enable the multi-layer approach, and provide a full explanation to the operating principle of CNNs. Second, we discuss how guidance is provided by labels through backpropagation (BP) in the training. Third, we show that a trained network can be greatly simplified in the testing stage demanding only one-bit representation for both filter weights and inputs.