Infrared small target detection is essential for security monitoring, maritime rescue, and public safety, yet its performance often degrades severely under domain shifts caused by varying imaging conditions and sensor characteristics. Existing domain generalization methods alleviate this issue to some extent, but cross-domain detection remains challenging when targets exhibit weak contrast and occupy only a few pixels. This motivates us to investigate whether large-scale pretrained visual priors, combined with infrared-specific feature learning, can improve performance on unseen domains. To this end, we propose EIRSAM, a SAM-based adaptation network for domain-generalized infrared small target detection. Built upon a frozen SAM backbone, EIRSAM introduces a dual-branch encoder–decoder architecture to jointly exploit the transferable visual priors of the foundation model and the fine-grained structural cues of infrared imagery. A feature-gated fusion module is designed to balance domain-general semantic representations and infrared-specific local structures, while a hierarchical mask decoder equipped with mask-guided category self-attention and bi-directional cross-mask interaction is developed to suppress background interference and refine target-focused prediction. In addition, a hierarchical loss strategy is introduced to stabilize coarse-to-fine optimization and improve prediction consistency under cross-domain conditions. Extensive experiments on multiple infrared small target benchmarks demonstrate that EIRSAM consistently improves detection accuracy and false alarm suppression on unseen domains, validating its effectiveness for robust cross-domain infrared small target detection.