AI in User Research: How to Discover the Jobs to be Done from Customer Interview

With the help of AI, you can streamline the analysis process and gain deeper insights into your customers' needs and motivations. But how exactly is AI used in user research?

AI can be used in user research to analyze customer interviews, organize insights into key themes and challenges, and translate the findings into practical insights that anyone can understand. However, users should be cautious about consistency and hallucinations in AI-generated output.

Let's examine a case study that illustrates how AI tools can be employed in user research.

Shavin Peiries is a product designer who used to run a design agency called “Very Bad Wizards”. They focused on helping corporate businesses apply design thinking to projects.

A large part of Shavin’s work involves interviewing customers to understand their problems. This involves speaking to people, transcribing the conversations, highlighting insights, organizing them into themes, and then synthesizing the findings. It can take days to analyze a batch of interviews because the work can be so subjective.

Shavin figured out a simple chatGPT workflow that organized all the highlights from the interviews into themes.

INPUT PROMPT 1: Group these customer quotes into crisp, actionable themes. Include the number of times a quote appears for each theme, who said the quote, and a snippet of the quote relating to the theme. Do it in the table format.

Then Shavin had the brilliant idea to get ChatGPT to rewrite the themes in the job story format so that they were easier for everyone to understand.

INPUT PROMPT 2: Now could you write them in the job stories format starting with When I...?

The final step involved identifying the most important customer challenges his clients would need to address. Shavin asked ChatGPT to take the customer quotes and create problem statements in the “how might we” format to reframe problems in the previous step as opportunities.

INPUT PROMPT 3: Generate 5 problem statements in the HMW format so that we can be discovered by these types of customers

ChatGPT successfully turned a collection of quotes into a set of key customer challenges that Shavin’s client could then brainstorm solutions for. All in half the time it usually takes. Plus, the opportunities it came up with were at the perfect level of abstraction to get ideas flowing, not too narrow nor too broad.

• Regenerating the same prompt multiple times leads to a different output. As Shavin puts it, “For example, a quote filed under “personal satisfaction” might be grouped under “desire for ownership” on a different, subsequent run. This lack of consistency can make it difficult to use effectively in a team setting.” If you are looking to recreate this use case I’d suggest using the Open AI sandbox rather than the ChatGPT so that you can turn the temperature setting down low to avoid this problem. Temperature corresponds to creativity, and the lower it is the more predictable the output becomes.

• The other issue was with the number of quotes you could feed ChatGPT in one go. Rather than giving it ten thousand quotes to work with, Shavin had to feed the quotes in 40-50 at a time. To overcome this limitation, keep reading because the third use case in the post covers a brilliant workaround for working with lots of text.

• You also have to watch out for when GPT-3 hallucinates information. For example, in the second step, ChatGPT wrote a job story about “not being constrained by the landlord” but there was no mention of a landlord in the quotes or interviews.

• For a full write-up of this case study, along with all the output for each of these prompts, check out Shavin’s guest post on the Buildspace blog.

• If you’d like to know more about Shavin Peiries, you can read about his work on his blog or you can get in touch with him on Twitter.