PromptInfuser: How Tightly Coupling AI and UI Design Impacts Designers' Workflows
Abstract
AI tools are ubiquitous and designers are increasingly mocking up AI-based applications.
While large language model (LLM) prompting has democratized AI prototyping, designers are still prototyping AI functionality separately from their UI design. In this paper, we investigate how tightly coupling both prompt and UI design affects designers' workflows and thought processes. To ground our research, we developed PromptInfuser, a Figma widget that enables users to create semi-functional mockups, by connecting UI elements to the inputs and outputs of LLM prompts. In a user study with 14 professional designers we compare PromptInfuser to their current disconnected workflow. Participants felt that PromptInfuser enabled prototypes that better communicated a product idea, hued more closely to the envisioned artifact, and better anticipated UI issues and potential technical constraints. PromptInfuser also transformed designers' workflow into a back and forth, tandem improvement of both prompt and UI, which in turn, helped them to (1) identify layout and prompt incompatibilities and (2) reflect upon and improve their total solution. Taken together, these findings inform future systems for prototyping ML-based applications.
While large language model (LLM) prompting has democratized AI prototyping, designers are still prototyping AI functionality separately from their UI design. In this paper, we investigate how tightly coupling both prompt and UI design affects designers' workflows and thought processes. To ground our research, we developed PromptInfuser, a Figma widget that enables users to create semi-functional mockups, by connecting UI elements to the inputs and outputs of LLM prompts. In a user study with 14 professional designers we compare PromptInfuser to their current disconnected workflow. Participants felt that PromptInfuser enabled prototypes that better communicated a product idea, hued more closely to the envisioned artifact, and better anticipated UI issues and potential technical constraints. PromptInfuser also transformed designers' workflow into a back and forth, tandem improvement of both prompt and UI, which in turn, helped them to (1) identify layout and prompt incompatibilities and (2) reflect upon and improve their total solution. Taken together, these findings inform future systems for prototyping ML-based applications.
