Facial expression conveys a variety of emotional and intentional message from human beings, and automated facial expression recognition (FER) has become an ongoing and promising research topic in the field of computer vision. However, the primary challenge of FER is learning to discriminate similar features among different emotion categories. In this paper, a hybrid architecture using Efficient Channel Attention (ECA) residual network ResNet-18, and feature manipulation network is proposed to tackle the above challenge. First, the ECA residual network effectively extract input features with local cross-channel interaction. Then, the feature decomposition network (FDN), feature reconstruction network (FRN) modules are added to decompose and aggregate latent features for enhancing the compactness of intra-category features and discrimination of inter-category features. Finally, an expression prediction network is connected to FRN to draw the final expression classification result. To examine the efficacy of the suggested approach, the model is trained independently using in-the-lab (CK+) and in-the-wild (RAF-DB) datasets. Several important evaluation metrics such as confusion matrix, Grad-CAM are reported, and the ablation study is conducted for demonstrating the efficacy and interpretability of the proposed network. It achieves the state-of-the-art accuracy compared to the existing facial recognition work, at 99.70% in CK+ and 89.17% in RAF-DB.