An Improved Adaptive Optimization Technique for Image Classification

In deep learning, the optimization techniques are the most part dependent on gradient descent methods, such as SGD, Adam; adopt the leading place in the area of optimization methods. Conventional methodologies which depend on stochastic gradients are non-adaptive because the prescribed parameter worth's usage should be tuned for every application. However, the generalization performance of the stochastic optimizers is far superior to the adaptive methods, whereas Adam and its variants cannot maintain this without a fast convergence rate in deep neural networks. To improve this generalization performance, we need to diminish the oscillation of the weights, the general problem of the accuracy fall. To stable the generalization performance, we have introduced Mean-ADAM, a variance of Adam, extends the updated weights by an external weight to diminish the oscillation and overcome a superior accuracy rate than all other adaptive gradient methods till the conclusion of the training. Therefore, we substantially improve the generalization performance, permit it to contend with the existing algorithms on different image classification datasets such as Mnist, Cifar 10, Cifar 100, ImageNet, etc. We have attained 82% at 150 ^th epochs with Cifar 10 and 99.49% with Mnist where the Adam has indicated 76% and 99.43% individually. The graphical statistics and the quantitative results can indicate that the proposed method is more likely to perform in more diverse image datasets.