Why review a products usability
Theres a number of reasons why you might want to review a product for its usability flaws (or potential). It’s becoming more common to do so and tends to be a typical starting point for any user experience design project that is attempting to build on top of an existing product.
A user experience (UX) audit or review can tell you:
- How a product is performing
- What problem areas exist
- Where users are having difficulty completing actions
- Where bottle necks in user journeys might exist
- Where to concentrate for usability improvement gains
In order to run a successful UX audit, there are a number of steps to consider.
Planning your audit
This is a really important step (perhaps the most important) as a UX review can quickly spiral into different directions and get out of hand if left un checked. Its possible to go really deep into usability issues and you might suddenly find you’ve spent all your hours critiquing 1 feature with 90% of the product still to go over.
These are the steps I like to take when planning a UX audit:
- Review the business and user objectives
- Understand the audit objectives
- Set a time limit
- Decide who will be involved
- Understand what resources are available
- Decide if interviewing an option
Review the business and user objectives
Firstly, speak with the project stakeholders about the bigger picture. If its a website or mobile app you are reviewing, those are typically customer facing products to a organisation. So, understanding what that organisation is trying to achieve is a great place to start from. Learn about how the product fits in with overall objectives, what are the main intended objectives for users when using the site, what are the primary and secondary user journeys in place and are there any key understandings already known by the project team?
Understand the audit objectives
Next, its a really good idea to try and understand what the objectives of the audit are. If you’ve been asked to perform a review by a specific team member or department, understanding their motives can be useful. A design led thinker may be interested in if all features are easy to understand and use where as marketing may be more concerned with conversion rate optimisation (CRO). Knowing if there are any areas the audit should focus on and why will help ensure you deliver a report thats going to be useful for the person who’s requested it.
Set a time limit
Your time limit for the work will largely be based on the budget but will let you plan out how long to spend on each task. There’s no golden rule to this and really experience is your best guide. Have a quick run through the site-map or feature list to try and gauge the walkthrough workload. For larger sites, you might need to make some decisions on where to prioritise your time.
Decide who will be involved
Wherever possible a UX audit should be completed by someone not attached to the project. Its hard for people who are too close to see issues from an average users perspective. Smaller audits will likely be done by one person. If possible, get the opinions of a cross section of people to provide additional valuable insight.
- Designers
- Developers
- Average users
- Business owners
If possible, include a decision maker in the process in order that they see the issues first hand. This will foster buy in and support for change recommendations later on.
Take a look at available resources
Find out if there is any data available to view that might give you some insights. Google analytics is a great place to start but there are an increasing number of tools available to aid in understanding user journeys such as Hotjar. Other sources of data might be: conversion rates, downloads & sales figures.
Is interviewing an option?
For larger audits, decide if interviews can be arranged. There’s no better place to gain valuable usability insights than from actual living breathing users. Beware though, doing user interviews wrong can result in tainted or misleading insights. If this is a serious consideration for you then a great book to read on the subject is ‘Interviewing users’ By Steve Portugal. As well as user interviews, you also might want to consider interviewing various project stakeholders such as business owners, designers and developers to get their take on where the product is at and where it might be heading.
Conducting the audit
There’s no right way to complete a UX audit and no set order you should follow. I like to start with a cognitive walkthrough of the product from a customers perspective though. This heuristic product evaluation allows you to experience first hand the users perspective, ideally without knowing too much ahead of time (to gain that fresh faced knowledge).
Follow each of the established users journeys (as set out by your understanding of the user objectives) and make notes against any potential problem areas. Consider each journey a user might want to take and walkthrough the process of completing each action. In addition, make sure to visit each additional page (for larger sites, visit each page type). Take notes on anything that does or doesn’t work as expected, anything that stands out as broken or anything you feel might be worth highlighting as a plus point.
If multiple devices are likely used to access the product, ensure walkthroughs are completed with each device type (ideally a real device rather than an emulator). Make notes where the experience differs between devices.
This cognitive walkthrough alone should give you enough usable insights to start forming opinions on the usability of the product.
Usability guideline checks
Where a walkthrough will give you a personal experience of the product to draw conclusions from, a common set of usability guidelines will provide confidence in consistency across your audits. Again, there is no right set of guidelines to use but a number of commonly agreed UX principles exist that are easily found online. I like this one for web and mobile projects. You should find a set that works for you and build on them as you need to.
Review the product against your set of guidelines. Provide a Pass, Fail or N/A for each of them and any accompanying notes to backup your result.
Review the data
If you have data available, decide how best to review it. You might find it easier work alongside a data analyst here or at least someone who can help interpret the stats. Decide where to focus to get the most out of the data and look for trends that stick out.
Conduct interviews
If you have gone down the interview route, you will need to ensure you conduct them in such a way as to not influence your interviewee’s answers. Things like avoiding asking leading questions and ensuring real world situations are reflected are important. Again, I recommend starting with Steve Portugal’s book for this (there really is a wealth of information in there).
Stakeholder interviews, are useful for gaining a more in-depth understanding of the product, its design and how it benefits the business. User interviews, are useful for gaining a more in-depth understanding of how people actually use the product, which may be different from the product owners intentions.
Report the findings
Collate your notes
Once you have completed your review, the first thing you’re going to want to do is collate all your notes. Taking the notes made during your heuristic product evaluation, organise them into suitable groupings, for instance:
- Styling/layout issues
- Content issues
- Customer journeys
- Device or feature specific
If there is a specific device/environment/operating system that creates a lot of issues, consider placing these in their own group. Similarly, if there is a specific feature that plays an important part in the audit, consider placing issues relating to this in their own group.
For each issue provide the following:
- Issue title
- Description of issue
- A suggestion to resolve the issue
- An example of where the issue can be seen
Consider reporting positive notes as well as negative. Clients can get quite upset when you start pulling apart their product. It can be a good idea to highlight some positives too so its not all bad. Be conscious about delivering bad news.
Look for trends
Look for trends in both the data and the walkthrough notes.
- Are there any obvious problem areas that can be identified?
- Are there any bottlenecks in place?
- Are there any major areas of drop-offs in user flow?
- Are there any clear paths that are working well?
- Are there any unexpected or unexplained issues that need further investigation?
Identify key learnings
Provide some key learnings that can be taken from the study. What are the major problems that would provide the biggest results if corrected? Are there any quick wins that would generate some instant success? I like to report these up front with a short paragraph on each to provide anyone quickly scanning the document some good takeaways.
Create recommendations
Provide some in-depth recommendations around the key learnings.
- Emphasise the positive
- Avoid jargon
- Be specific where possible
- Be tactful in reporting frustrations
Report template
Your report template will differ from the next reviewers but there are a few items to make sure you include:
- Title & date - make sure readers know when the review was done so they can assess at what point the product was at.
- About this audit - detail the approach you took and what the audit does and does not include.
- Key learnings - put these upfront so quick readers can easily find the good stuff.
- Issues identified - list these in detail so in-depth readers can identify specific problems and your suggested solutions (if there are lots, consider putting these in a appendix).
- Usability guideline checks - show you are basing your opinions on commonly agreed guidelines.
- Interview transcripts - consider including these if appropriate.
- Data sources & references.
A well produced UX audit can be an invaluable document to put together ahead of any new development projects starting. It gives a great starting point for designers to work from in tackling usability issues and gives project stakeholders confidence they are focusing on the right areas to provide maximum gains. I hope this article is helpful for you in developing your own audits.
