Biased answers in usability testing

August 1, 2010

Self-reported data is typically three steps removed from the truth:

  • In answering questions (particularly in a focus group), people bend the truth to be closer to what they think you want to hear or what’s socially acceptable.
  • In telling you what they do, people are really telling you what they remember doing. Human memory is very fallible, especially regarding the small details that are crucial for interface design. Users cannot remember some details at all, such as interface elements that they didn’t see.
  • In reporting what they do remember, people rationalize their behavior. Countless times I have heard statements like “I would have seen the button if it had been bigger.” Maybe. All we know is that the user didn’t see the button.

Jakob Nielsen (2001): First Rule of Usability? Don’t Listen to Users

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s