ABSTRACT To ensure precise and rapid identification of casting surface defects and to support the subsequent realisation of high‐precision grinding, this study introduces a method for detecting casting surface defects using a lightweight YOLOv5 framework. The enhanced model integrates the ShuffleNetV2 high‐efficiency CNN architecture into the YOLOv5 foundation, substantially reducing network parameters to achieve a lightweight model. Additionally, the Convolutional Block Attention Module (CBAM) attention mechanism is incorporated to enhance the model's capability to detect defects. The ReLU activation function replaces the SiLU function in the convolutional layer, decreasing the computational load and boosting efficiency. Subsequently, the optimised model is quantised and implemented on the RV1126 embedded development board, successfully performing image inference. To validate the effectiveness of the proposed method, a dataset of casting surface defects was designed and constructed. The optimised model has a file size of 7.6 MB, representing 55.4% of the original model, with about 50.6% of the original model's parameters. The onboard inference speed of the improved model is 50 ms per image, which is 9.1% faster than the traditional YOLOv5 model. These results offer valuable insights for future casting surface defect detection technologies.