Data-informed design

Our philosophy and approach for every design sprint is to be data-informed, not data-driven. We try to surface every piece of information that will help paint a clear picture of the problem we’re trying to solve. We leverage all of the data we can to understand the core problem, but we don’t blindly build whatever the data may suggest.

Data is an extremely valuable tool and it’s critical to the design process. Designing without data is like flying blind, but purely data-driven design is dangerous and can lead to unintentional and uninspired design. Testing 41 different shades of blue may increase your conversion rate slightly, but if your design is flawed to begin with it will never be able to reach it’s full potential. Relentless A/B testing can only take you so far. Maybe your Google Analytics numbers aren’t quite giving you the whole picture.

Ryan Langlois Data-informed design

 

Advertisements

Measuring performance (e.g. task success rate) and satisfaction (e.g. ‘liking a Website’)

• Performance and satisfaction scores are strongly correlated, so if you make a design that’s easier to use, people will tend to like it more.

• Performance and satisfaction are different usability metrics, so you should consider both in the design process and measure both if you conduct quantitative usability studies.

JAKOB NIELSEN: User Satisfaction vs. Performance Metrics

User experience metrics

Most metrics are marketing oriented, not experience oriented. Unique visitors can tell you whether your marketing campaign worked and social mentions can tell you whether you’ve got a great headline, but these metrics do not reveal much about the experience people have had using a site or application.

(…)

User experience is about more than just ease of use, of course. It is about motivations, attitudes, expectations, behavioral patterns, and constraints. It is about the types of interactions people have, how they feel about an experience, and what actions they expect to take. User experience also comprehends more than just the few moments of a single site visit or one-time use of an application; it is about the cross-channel user journey, too. This is new territory for UX metrics. – See more at: http://www.uxmatters.com/mt/archives/2014/06/choosing-the-right-metrics-for-user-experience.php#sthash.zmdVOE0q.dpuf

Three categories of UX metrics: Usability; Engagement, and Conversion

    Usability
    Usability metrics focus on how easily people can accomplish what they’ve set out to do. This category of metrics includes all of the usability metrics that some UX teams are already tracking—such as time on task, task success rate, and an ease-of-use rating. It may also include more granular metrics such as icon recognition or searching versus navigating. Plus, it could include interaction patterns or event streams that show confusion, frustration, or hesitation.

  • Time on task
  • Task success
  • Confusion moment
  • Perceived success
  • Cue recognition
  • Menu/navigation use
    Engagement
    Engagement is the holy grail for many sites and is a notoriously ambiguous category of metrics. But UX teams could make a real contribution to understanding how much people interact with a site or application, how much attention they give to it, how much time they spend in a flow state, and how good they feel about it. Time might still be a factor in engagement metrics, but in combination with other metrics like pageviews, scrolling at certain intervals, or an event stream. Because this metric is tricky to read, it yields better results in combination with qualitative insights.

  • Attention minutes
  • Happiness rating
  • Flow state
  • Total time reading
  • First impression
  • Categories explored
    Conversion
    Conversion is the metric that everyone cares about most, But its use can mean focusing on a small percentage of users who are ready to commit at the expense of other people who are just becoming aware of your site or thinking about increasing their engagement with it. You can use UX metrics to design solutions for these secondary scenarios, too—for example, by looking at users’ likelihood of taking action on micro-conversions, in addition to considering conversion rate and Net Promoter Score (NPS).

    The metrics in this category can help us to spot trends and get past the So what? question that applies to all data. The big metrics give us the big picture, showing how a site or application changes over time and how it lives in the world or the broader context of other experiences.

  • Micro-conversion count
  • Brand attribute
  • Conversion rate
  • Likelihood to recommend, or NPS
  • Trust rating
  • Likelihood to take action

Pamela Pavliscak: Choosing the Right Metrics for User Experience

Talking with Participants During a Usability Test

3 safe and productive approaches for interrupting or answering users during usability tests and other research studies:

  • Echo – With the echo technique, the facilitator repeats the last phrase or word the user said, while using a slight interrogatory tone.
  • Boomerang – With the boomerang technique, the facilitator formulates a generic, nonthreatening question that she can use to push the user’s question or comment back to him.
  • Colombo – With the Columbo technique, be smart but don’t act that way. (…) One way to do this is to ask just part of a question, and trail off, rather than asking a thorough question.

Kara Pernice: Talking with Participants During a Usability Test

Usability metric

Usability metrics reveal [measure] something about […] some aspect of

  • effectiveness (being able to complete a task)
  • efficiency (the amount of effort required to complete a task)
  • or satisfaction (the degree to which the user was happy with his or her experience while performing the task)

Some examples: task success, user satisfaction; errors

Tulls and Albert (2008), p.7-8