ABSTRACT Recent advancements in machine learning have emerged as a transformative force to revolutionize the way of designing metasurfaces and accelerating plenty of adaptive applications in sensing, displaying, imaging, and cloaking. However, the established intelligent design agents typically work for a fixed scenario, and the associated transfer learning techniques mostly lack robust and interpretable knowledge migration. Here, we propose an attention‐guided transfer learning framework that enables adaptive cross‐frequency knowledge transfer for robust broadband metasurface inverse design. The core of attention‐guided transfer learning lies in a physics‐informed attention mechanism that evaluates source‐frequency knowledge for a target frequency, selectively emphasizing generalizable knowledge while suppressing source‐specific effects to ensure effective‐only knowledge migration. Comparative results across 5–10 GHz band demonstrate that the attention‐guided transfer learning achieves over 10% prediction accuracy enhancement compared to conventional methods, while reducing the data requirements by 20%. Moreover, visual analyses confirm the model's physical intuition, with its focus shifting from global patterns at lower frequencies to local details at higher frequencies, mirroring wavelength‐dependent electromagnetic behavior. Our work opens a new avenue for robust cross‐scenario metasurface design by leveraging selective knowledge transfer to enhance the adaptability of intelligent electromagnetic systems.