OrthoMaps: an efficient convolutional neural network with orthogonal feature maps for tiny image classification

In image processing domain of deep learning, the big size and complexity of the visual data require a large number of learnable variables. Subsequently, the training process consumes enormous computation and memory resources. Based on residual modules, the authors developed a new model architecture that has a minimal number of parameters and layers that enabled us to classify tiny images using much less computation and memory costs. Also, the summation of correlations between pairs of feature maps as an additive penalty in the objective function was used. This technique encourages the kernels to be learned in a way that elicit uncorrelated representations from the input images. Also, employing Fractional pooling helped to have deeper networks that consequently resulted in more informative representation. Moreover, employing periodic learning rate curves, multiple machines are trained with a less total cost. In the training phase, a random augmentation to the input data that prevent the model from being overfitted was applied. Applying MNIST and CIFAR-10 datasets to the proposed model resulted in the classification accuracy of 99.72 and 93.98, respectively.