AI-assisted vs. non-AI assisted product ideation: An experimental study

Topic

This Master Thesis examines how AI-assisted ideation influences the creativity of product ideas compared to non-AI-assisted ideation. Using a controlled experimental design, participants generated product ideas either with ChatGPT support or without AI assistance. The study evaluates idea creativity through the dimensions of novelty and usefulness and additionally explores how AI literacy influences ideation outcomes. The research focuses specifically on early-stage digital product ideation and investigates not only idea quality, but also efficiency and diversity within idea portfolios.

Relevance

Generative AI tools such as ChatGPT are increasingly being integrated into creative and innovation processes within organizations. However, many companies still lack clarity regarding when AI truly improves idea generation and what trade-offs may emerge. This research is relevant for practitioners because it shows that AI can improve efficiency and help generate more structured and useful ideas, while simultaneously reducing diversity across idea portfolios. The findings help organizations better understand how to integrate AI into innovation processes without unintentionally limiting conceptual exploration and creativity diversity.

Results

The findings show that AI-assisted participants completed ideation tasks significantly faster than non-AI-assisted participants. AI-assisted ideas were also evaluated more positively in terms of usefulness and meaningfulness. However, no significant differences were found for novelty-related dimensions. Additionally, AI-assisted ideas appeared more homogeneous and clustered more closely together, suggesting lower collective idea diversity. The study further indicates that AI literacy plays an important role, as users with more AI experience appeared better able to generate differentiated outputs and navigate AI-supported ideation processes more effectively.

Implications for practitioners

• AI can improve ideation efficiency and reduce the time required to generate structured product ideas.
• AI-assisted ideation may enhance usefulness and clarity of ideas, especially in early-stage innovation tasks.
• Overreliance on AI may reduce diversity within idea portfolios and lead to more homogeneous solutions.
• Organizations should combine AI-assisted and non-AI-assisted ideation approaches to balance optimization and exploration.
• Developing employees’ AI literacy and prompting skills is important to maximize the value of AI-supported creativity.

Methods

This study used a two-phase between-subjects experimental design. In the first phase, participants were randomly assigned to either an AI-assisted condition, where they used ChatGPT during ideation, or a non-AI-assisted condition. All participants completed the same standardized digital product ideation task focused on improving information management in messaging applications. In the second phase, independent raters evaluated the generated ideas using validated creativity scales measuring novelty and usefulness based on Im and Workman (2004). Statistical analyses included Wilcoxon rank-sum tests, Kolmogorov–Smirnov tests, and exploratory Principal Component Analysis (PCA) visualizations to examine creativity patterns, diversity, and clustering effects between conditions.