Multimodal machine learning in the AEC industry: a lifecycle-aligned review of strategies, challenges, and informatics frameworks

The architecture, engineering, and construction (AEC) industry generates diverse data streams across design, construction, and operations, yet most artificial intelligence systems remain unimodal, limiting their potential. Multimodal machine learning (MMML) offers a framework for integrating heterogeneous inputs, such as images, BIM models, sensor logs, and textual records, to enable richer representations and context-aware decision support. This review delivers a lifecycle-aligned synthesis of MMML in AEC, based on a systematic screening of 71 peer-reviewed studies (2018–2025) coded by modality, fusion strategy, architecture, task, and application domain. Findings show that images, text, and numerical data dominate existing pipelines, whereas audio, physiological, and higher-order modality combinations are rare. Intermediate fusion, primarily through CNN-based architectures, remains the prevailing approach, with limited use of transformers, diffusion models, or graph neural networks. Applications cluster in construction-phase safety monitoring and operations-phase inspection, with predictive maintenance gaining momentum post-2021. Generative design and decision support remain underrepresented, often relying on simulation-driven methods. Key barriers include weak alignment mechanisms, limited robustness to missing modalities, insufficient external validation, and the absence of standardized benchmarks. Advancing MMML in AEC requires lifecycle-spanning benchmarks, adoption of next-generation architectures, and embedding robustness and validation as standard practice. Emerging platforms, such as BIM integration, digital twins, and federated MMML, offer promising testbeds for scalable deployment. This review provides a comprehensive overview of current progress, challenges, and future directions for multimodal ML in the built environment.