Biased answers in usability testing

Self-reported data is typically three steps removed from the truth:

  • In answering questions (particularly in a focus group), people bend the truth to be closer to what they think you want to hear or what’s socially acceptable.
  • In telling you what they do, people are really telling you what they remember doing. Human memory is very fallible, especially regarding the small details that are crucial for interface design. Users cannot remember some details at all, such as interface elements that they didn’t see.
  • In reporting what they do remember, people rationalize their behavior. Countless times I have heard statements like “I would have seen the button if it had been bigger.” Maybe. All we know is that the user didn’t see the button.

Jakob Nielsen (2001): First Rule of Usability? Don’t Listen to Users

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s