Surface defect detection (SDD) is essential for quality assurance in consumer electronics manufacturing. Although existing SDD methods have achieved high accuracy in complex scenarios, their real-world deployment still faces two major challenges. First, high-accuracy models typically involve large model sizes, high computational costs, and slow inference speeds, which hinder real-time applications. Second, although lightweight architectures help to mitigate these issues, they often suffer from weakened feature representations, leading to performance degradation. To address these challenges, we propose a lightweight yet effective Contextual Collaborative Aggregation Network (CCANet) for SDD. CCANet employs a hybrid feature extraction backbone that integrates the first four stages of EfficientNet-B0 with a Context-Aware Vision Transformer (CAViT), enhancing both local and global feature representations while reducing model complexity. We further introduce a Context Collaborative Module (CCM) to leverage both low-level texture details and high-level semantic features. Additionally, we design an Asymmetric Enhancement Module (AEM) that uses asymmetric convolution kernels to explore orthogonal features and gradually fuse multi-level representations. Extensive experiments on four public SDD datasets validate the robustness and generalization of CCANet. For a 352× 352 input image, it requires only 1.85M parameters and 1.69G FLOPs, achieving real-time inference at 33.61 FPS.