StepIdeator: Utilizing Mixed Representations to Support Step-by-step Design With Generative AI

Abstract In product conceptual design, designers utilize multiple design representations to iteratively ideate, externalize, and refine concepts. Mixed representations, defined as the simultaneous presentation of multiple representations, foster deeper insights and facilitate broader exploration compared to focusing on a single representation. However, designers often struggle with the cumbersome process of creating, transforming, and refining these representations. Advanced AI capabilities now significantly lower the barriers to creating text, images, and 3D models, presenting substantial potential for application in design practice. However, current design tools based on generative AI tend to produce polished product images directly, hindering designers from continuously deliberating on mixed representations, and thus limiting their creative potential. Therefore, based on a formative study, we propose StepIdeator, a step-by-step design tool driven by generative AI. This tool facilitates seamless transition and refinement of mixed design representations. Through a comparative study (N=16), we validated StepIdeator's effectiveness in improving idea externalization and enhancing creativity. Furthermore, the results revealed that designers perceived greater confidence, ownership, and sense of contribution when collaborating with StepIdeator.