Usability Testing: Does Independence Matter?

I caught up with a former client recently and got onto the topic of independence in usability testing a design. He was lamenting the fact that there are very few companies who focus on usability testing without also providing their own design services. Having tested my own designs and tested the designs of my colleagues, as well as working as a third party brought in to assess an agency’s design, I have strong views on the role of independence. I feel that it is impossible to be truly objective with your own designs however this is not to say that you shouldn't test them. Confused? Let me explain.

Generally speaking, there are two questions that usability testing answers: 1) How usable is an interface? 2) How can an interface be improved? Often these questions are explored together, however, identifying which is the most important will dictate whether independence is required.

When the goal of testing a design is to gain an accurate measure of how usable it is, the testing requires an independent practitioner with no vested interest in a positive or negative outcome. The person or organisation which created the design has an interest in a positive outcome which will impact their assessment of the interface's usability. This is not to suggest that a designer intentionally goes out of their way to present the results in a misleading manner, although I am sure this has happened. Put simply, the assumptions which were made during the design phase make it extremely difficult to objectively assess behaviour during a usability test. The outcome is more likely to be favourable because the practitioner is more likely to acknowledge the behaviour which supports their assumptions (the salience effect). Furthermore, an agency that has designed an interface is also compromised in its assessment of usability, even in cases where separate teams design and evaluate an interface. Again this is because they have a vested interest in a positive outcome. This is never more evident than during difficult projects and where tight budgets exist.

There are a number of scenarios where I think independence matters:

- A project where significant investment has been made to design an interface

- Interfaces that will have a significant impact on the bottom line i.e. a new eCommerce platform

- Projects which are highly strategic for an organisation and numerous stakeholders are involved

- Projects where the team must report to a steering committee or senior management

- Interfaces that depart from industry or design conventions

In all of the above scenarios, being able to state that an interface has been independently tested and is usable is of great value to the designers and project owners. It also represents a risk mitigation strategy for organisations in identifying readiness for deployment.

So having argued all the pros of independence, when does independence become less important? When the goal of usability testing is purely focused on enhancing the design. Why? Because a good designer should be able to see the flaws in their execution and ultimately have an interest in creating the best possible design. Hopefully, they are motivated to identify opportunities for improvement.

Ironically it seems the UX marketplace is moving away from independent usability testing despite the value that it offers to organisations as well as the integrity it affords the design process itself. Focusing on the objectives of usability testing will help to clarify whether independence is necessary. I think that if understanding usability is the goal, one agency or designer cannot objectively test their own design.