Examining How the Large Language Models Impact the Conceptual Design with Human Designers: A Comparative Case Study

Advances in artificial intelligence have led to breakthroughs in large language models (LLMs), like ChatGPT, opening up exciting possibilities for conceptual design. However, it's essential to gain an in-depth understanding of how LLMs impact conceptual design output, process, and human designers' perception. To this end, we chose ChatGPT as an example and conducted the investigation with 30 participants divided into Human-LLMs groups and human-human groups. The results indicated that there was no significant difference between their outputs, but the incorporation of LLMs shortened the completion time with fewer design steps and less time allocated to the late stages of design. Despite being perceived as less efficient and trusted, LLMs can still be viewed as potential collaborators, with humans holding the leadership. These findings offer the HCI community a thorough comprehension of how LLMs influence creativity-related practices, providing valuable insights for designing future interactions with LLMs.