8 Rules for interviews

  1. Adopt a beginner’s mind Listen with a “fresh pair of ears” and avoid interpretation. Explore unexpected jobs, pains, and gains in particular.
  2. Listen more than you talk Your goal is to listen and learn, not to inform, impress, or convince your customer of anything. Avoid wasting time talking about your own beliefs, because it’s at the expense of learning about your customer.
  3. Get facts, not opinions Don’t ask, “Would you…?” Ask, “When is the last time you have…?”
  4. Ask “why” to get real motivations Ask, “Why do you need to do…?” Ask, “Why is___important to you?” Ask, “Why is___such a pain?”
  5. The goal of customer insight interviews is not selling (even if a sale is involved); it’s about learning Don’t ask, “Would you buy our solution?” Ask “what are your decision criteria when you make a purchase of…?”
  6. Don’t mention solutions (i.e., your prototype value proposition) too early Don’t explain, “Our solution does…” Ask, “What are the most important things you are struggling with?”
  7. Follow up Get permission to keep your interviewee’s contact information to come back for more questions and answers or testing prototypes.
  8. Always open doors at the end Ask, “Who else should I talk to?”

Osterwalder 2014 p.112-3

Biased answers in usability testing

Self-reported data is typically three steps removed from the truth:

  • In answering questions (particularly in a focus group), people bend the truth to be closer to what they think you want to hear or what’s socially acceptable.
  • In telling you what they do, people are really telling you what they remember doing. Human memory is very fallible, especially regarding the small details that are crucial for interface design. Users cannot remember some details at all, such as interface elements that they didn’t see.
  • In reporting what they do remember, people rationalize their behavior. Countless times I have heard statements like “I would have seen the button if it had been bigger.” Maybe. All we know is that the user didn’t see the button.

Jakob Nielsen (2001): First Rule of Usability? Don’t Listen to Users

Interviewing users (1)

… in their book User and Task Analysis for Interface Design, Hackos and Redish devote an entire section to the formulation of unbiased questions. They advise interviewers to

  • avoid asking leading questions, to
  • ask questions that are based on a participant’s experience, and
  • to avoid overly complex, lengthy questions.

Michael Hawley in Preparing for User Research Interviews: Seven Things to Remember

Interviewing: Asking users and experts

  1. There are three styles of interviews: structured, semi-structured and unstructured.
  2. Interview questions can be open or closed. Closed questions require the interviewee to select from a limited range of options. Open questions accept a free-range response.
  3. Many interviews are semi-structured. The evaluator has a predetermined agenda but will probe and follow interesting, relevant directions suggested by the interviewee. A few structured questions may also be included, for example to collect demographic information.
  4. Structured and semi-structured interviews are designed to be replicated. Focus groups are a form of group interview.
  5. Questionnaires are a comparatively low-cost, quick way of reaching large numbers of people.
  6. Various rating scales exist including selection boxes, Likert, and semantic scales.
  7. Inspections can be used for evaluating requirements, mockups, functional prototypes, or systems.
  8. Five experts typically find around 75% of the usability problems.
  9. Compared to user testing, heuristic evaluation is less expensive and more flexible.
  10. User testing and heuristic evaluation often reveal different usability problems.
  11. Other types of inspections include pluralistic and cognitive walkthroughs.
  12. Walkthroughs are very focused and so are suitable for evaluating small parts of systems.

From: Preece, J., Rogers, Y., Sharp, H. (2002), Interaction Design: Beyond Human-Computer Interaction, New York: Wiley, p. 424