How the SUS Will Turn You Into a UX Superhero

I have been a huge fan of the SUS (System Usability Scale) for a number of years now and found it to be a powerful tool when communicating feedback on the usability of an interface. The SUS can be the tool that gives you superpowers.

I’m sure all UXers and product owners have had challenges convincing stakeholders of the need to fix usability issues. The SUS can play a role in convincing these stakeholders of the need to make changes to eliminate usability issues. It provides stakeholders with an objective understanding of how usable an interface is (or is not), and how it compares to other interfaces.

SUS - unleashing the UX superhero in all of us!

It provides an additional piece of information about an interface to build a bigger picture of the state of the interface. With this objective view, UXers can use the greater context to help stakeholders better understand the need to address usability issues. For example, if you have undertaken a round of usability testing and found lots of small issues but are not convinced that the overall usability is poor, a high or low SUS may provide greater clarity to the cumulative impact of these issues, and help build your argument either way.

The basics - what is the SUS?

The system usability scale (SUS) measures the usability of products and services, including hardware, software, mobile devices, websites and applications. It was developed by John Brooke in 1986 and has become recognized as an industry standard with references in over 1300 articles and publications.

Applications

SUS can be used for:

• Gathering an objective measure of usability.

• Comparing the usability between 2 products (i.e. product with competitors).

• Comparing the usability between different elements within a product (i.e. registration vs search).

• Comparing the usability between iterations or versions of a product.

Why use the SUS?

It is a highly valuable tool for a variety of reasons:

• Allows the usability of a product to be compared to industry standards of usability.

• Provides stakeholders with a solid metric to compliment usability testing.

• Relatively quick and easy to administer.

• Small sample sizes typical of lab-based usability testing can produce reliable results.

• Senior management typically responds well to this type of metric.

What?

The survey contains 10 questions answered using 5 options from strongly disagree to strongly agree.

1. I think that I would like to use this system frequently.

2. I found the system unnecessarily complex.

3. I thought the system was easy to use.

4. I think that I would need the support of a technical person to be able to use this system.

5. I found the various functions in this system were well integrated.

6. I thought there was too much inconsistency in this system.

7. I would imagine that most people would learn to use this system very quickly.

8. I found the system very cumbersome to use.

9. I felt very confident using the system.

10. I needed to learn a lot of things before I could get going with this system.

Note that the word ‘cumbersome’ in question 8 can be problematic for some participants, especially non-English speakers. The term ‘awkward’ can provide a good alternative which has been explored in some detail. 

How?

The SUS is administered following the use of a product and can be collected in a number of simple ways:

• The simplest means is via hardcopy however this does increase the time to calculate the score. Templates of the questionnaire can be readily found online for printing.

• Online survey tools such as Survey Monkey can be used, which offers flexible means of collecting the data on a variety of devices and locations. For example, I have used a tablet to collect the SUS in-store and sent via Skype during remote sessions.

• The usability testing software Morae includes a survey function with a template for the SUS questions plus a means for calculating the metric. This is my preferred method, where appropriate because it is super quick to set-up and analyse the data.

Reporting the SUS

There are a few things to consider when presenting and communicating the SUS:

• Provide some context: a couple of great options include an industry average, competitors and previous iterations of the design. Note the industry average is 68 which represents an average score. Note this is not a good score just an average one.

• The SUS is a measure of perceived ease of use. While typically there is a correlation between task completion rates in usability testing there can be a discrepancy in task completion rates and user perceptions of usability.

• I’m a fan of reporting confidence intervals which helps when dealing with small sample sizes.

• It is not a percentage and should not be reported as such.

In terms of interpreting the SUS, it is helpful to understand the distribution of scores (see table below). The table below also shows a grading system that can be used to provide context for the scores.

Source: http://www.measuringusability.com/sus.php

To wrap up

The SUS can be a powerful tool to communicate the usability of an interface and adds weight to usability assessments. It can help take your UX skills to superhuman levels.

^ https://digital.gov/2014/08/29/system-usability-scale-improving-products-since-1986/