The best way to screw your research is to do it out of the real context

I frequently hear about doing "guerilla research" by intercepting random strangers in cafes, offering them a drink in exchange for the conversation about your very product. This is wrong on many levels:

  • There is a low probability that the stranger belongs to your target user group (except if you are developing next social network or search engine that will rule the world - which you, probably, are not doing).
  • Assume you are testing a consumer mobile app and need a feedback. The real context of usage is not interrogation in a cafe and simulating some "user tasks", fully concentrated. The more likely usage is with divided attention under suboptimal conditions, with the smudged screen filled with notifications while listening to loud techno music.
  • It's even worse if you are researching B2B app. The real B2B context of use is in the open office, where your fellows are calling you on the phone, someone is tapping you on the shoulder asking for a favor, urgent emails are arriving at the pace of instant messages and you have your laptop docked to two large monitors while one doesn't work.
  • And - let's be real: what is stranger's motivation for a sincere and deliberate response, beside getting a free flat white?

Doing research in cafe

I should put /s at the end of this rant so you got the point. But with or without sarcasm the research IS expensive, and the research outside the real context is BOTH expensive and pointless.

There are better ways to understand your users and their use cases. Shadowing. Mental models. Follow-me-home research technique. Think-aloud. Contextual inquiries. Diary studies. And many more! All of them require effort and direct human to human communication. There is no substitute for it.

No LLM will replace research with real users.

Nor a casual, comfortable interview in a shiny neighbour cafe.

Continue reading