Optimizing CNN Model Performance for MNIST and CIFAR Classification Using Rectified Sigmoid and ReS Activation Functions

In this work, we implemented and evaluated the accuracy of two commonly used activation functions in convolutional neural networks (CNNs), Rectified Linear Unit (ReLU) or/and Sigmoid, as well as a newer activation function called Residual (Res) activation function. trained CNN models with each of these activation functions on MNIST and CIFAR datasets and evaluate their performance. Overall, this work provides insights into the performance of different activation functions in CNNs.in CIFAR dataset Res Activation function provides slightly better accuracy than the Leaky ReLU and PReLU activation function and rectify_sigmoid achieving a slightly higher accuracy of 89.82%, compared to the activation function of res.