As an indispensable material in daily life, effective quality control of metal requires real-time surface defect detection.However, metal surface defects present challenges for existing detection algorithms due to their characteristics, such as major distinctions in interclass similarity, multi-scale variations, and small sizes.In this paper, we propose an improved model based on YOLOv8m to achieve better performance in defect detection.Firstly, we introduce new feature extraction modules, C2f_ASCF and C2f_E-ASCF, which are deployed in different regions of the model to enhance its feature extraction capabilities.In stage of feature fusion, we design the FFDPN module to aggregate multiple channel information precisely.Additionally, a mechanism is employed to disseminate contextual information between different scales, significantly enhancing the network's performance to extract and fuse multiscale features.Moreover, the model incorporates a Lightweight Efficient Shared Convolutional Detection Head (LE-SCDH), which greatly reduces computational requirements compared to the conventional YOLOv8m head, without compromising performance.This is further enhanced by the Focaler-PIoUv2 loss function, which enables the model to excel in detecting challenging targets.We experimented with two industrial metal surface defect datasets to validate the rationality of the improvements on the model: The GC10-DET and NEU-DET datasets.The results demonstrate that our model improves the mAP by 7.2% and 12.2%, respectively, while reducing the total number of model parameters from 25.8 million to 15.7 million, which generally improves the reliability of metal surface defect detection.