Semiotics-based prompt engineering for architectural text-to-image generation processes

Text-to-image generative AI tools have gained significant attention in the architectural community; however, they are currently being used by trial-and-error with simple textual inputs. This is largely due to the lack of established frameworks for crafting prompts that yield semantically rich architectural outputs. This paper proposes using semiotics as an analytical method facilitating text-to-image generation processes. Two experiments were conducted to investigate the effects of semiotic analysis and adding context modifiers to prompts on the relevancy of outputs of three mainstream text-to-image generation tools (DALL-E, Midjourney, and Stable Diffusion). The results indicate the effectiveness of the proposed method and reveal opportunities and limitations of current text-to-image generative models in architecture. It is concluded that a human-centered approach to Human-AI interaction is needed to overcome issues regarding control, transparency, and data quality.