State Space Model (SSM) has shown potential in medical image analysis due to its advantages in long sequence modeling. As the most popular SSM at present, traditional Mamba prefers global dependency modeling and ignores some local details. However, in breast cancer image classification, local details of lesions are crucial to the accuracy. To address this challenge, we proposed three branch network HybridMamba in this paper, which combines the local feature extraction of convolutional neural network and global context modeling of Transformer to improve the perception of coarse-fine grained lesion features. In the Mamba branch of HybridMamba, an 8-direction Scanning Path is proposed, and Patch Wise Scanning Attention (PWSA) is used to aggregate different scanning sequences to improve the local perception ability. Due to the significant differences between the training dataset and testing dataset, a Patch Statistical Normalization (PSN) is proposed in the Transformer branch of HybridMamba to mitigate performance loss from in data distribution difference. In the experiments, the proposed HybridMamba is evaluated with 9 different models on DBT images of 150 female patients with breast cancer collected in the hospital. The experimental results showed that compared with the optimal VMamba in the comparative models, HybridMamba achieved improvements of 6.81, 6.04, 5.26, and 8.66 in Accuracy (ACC), Area Under the Curve (AUC), Sensitivity (SEN), and Specificity (SPE), respectively, which indicates the effectiveness of the proposed method.