This week we looked at a new technique for assessing our designs if usability testing was not possible or we just wanted to take a step back and assess a design. This approach is called a Heuristic Evaluation or a UX Audit and involves going through a design and seeing if it meets Jakob Nielsen’s 10 Usability Heuristics for User Interface Design. Any issues you find you give a severity rating and this gives you a list of issues and a ordered list in which to solve them. A UX Audit can also be carried out as part of a competitor analysis, before tackling a redesign of a product or while assessing early iterations of a product.
As we covered Nielsen’s 10 usability heuristics in class I felt rather than cover them again I would look for articles talking about best practices for completing a UX Audit and the benefits for a design of completing one.
While looking for information on the benefits of completing heuristic evaluations and some tips on how to complete them effectively I found this piece https://www.interaction-design.org/literature/topics/heuristic-evaluation on the Interaction Design Foundation website.
While I am not going to go into each of the heuristics individually I did think this illustration of the ten heuristics was useful as a visual reminder of what you would be looking for during an heuristic evaluation.
Source: https://www.interaction-design.org/literature/topics/heuristic-evaluation (Last Accessed 31/12/23
The article goes on to list some pros and cons of completing a heuristic evaluation with the main pros being the fact that you can carry out the evaluation early in the design process spotting potential issues early and solving them. It is also much quicker and less expensive than carrying out user testing with real users.
However, while these are appealing benefits to the heuristic evaluation process there were definitely some concerning downsides to the process. The requirement for trained and skilled heuristic evaluators can add costs to the process and as the process is based on assumptions (albeit heavily researched and validated assumptions) about good usability it can still miss things that real users would spot. This was clarified with some statistics that said that heuristic evaluation could only identify 21% of usability issues compared to a user test. This was even more concerning as the same study suggested that 43% of issues flagged during a heuristic evaluation where not really issues to users.
This is concerning and does reiterate the fact that while a heuristic evaluation is an incredibly useful exercise and can offer some real insights into the usability of a product it can never been seen as a replacement for a usability test with a sample of real users.
To look deeper into more advantages and disadvantages of the heuristic evaluation process I found this article https://ux247.com/the-pros-and-cons-of-heuristic-evaluation/ on a website called UX24/7.com.
While it covered the same pros and cons as above it also added a few more. For advantages it said by using multiple evaluators you could get a range of views leading to picking up more issues. It also mentioned how the process of setting up a heuristic evaluation focused the minds of the design team onto the core elements of the design and this therefore led to more focused results from the evaluation. Heuristic evaluations also have less ethical and practical issues as the evaluators are all professionals and are aware of the role they are taking on. They are also familiar with the process and what is required from them.
The main further disadvantage that was suggested was the room for the evaluator’s own bias to creep into their results. As designers we all have slightly different views on what is a problem and what level of severity a problem represents. These biases could allow certain problems to be missed or to be wrongly labelled and therefore not fixed.
Another key point I saw across these articles and others I looked at was the need to have multiple experts to conduct a heuristic evaluation and the expense that this could incur. With the need for usability testing to still be completed in an ideal scenario it does question the use of a heuristic evaluation.
I now wanted to look at the process of conducting a heuristic evaluation and helpfully the Interaction Design Foundation’s article provided an eight-step guide. The first step much like when I looked at design critiques is to decide what elements of the design you want your evaluators to concentrate on, is it one key process for users or the overall design. The more specific you can make the task the more in-depth feedback you can receive.
It also suggests defining the heuristics you want to use in your test it suggests using Nielsen’s heuristics with some modifications if required. Much like defining what part of your design you want evaluated, defining the heuristics you wish to use and selecting the correct ones will hugely benefit the process as well as the results you will get from the evaluation. A heuristic evaluation is something that requires time to be spent before the evaluation to truly understand what needs to be tested and which heuristics to test them against.
From this look into heuristic evaluations, I can see there is lots of benefits to carrying one out on a design the speed and being relatively inexpensive compared to a usability test alongside the use of tried and tested heuristics makes them an appealing prospect. Although from looking onto some of the disadvantages associated with this technique it is clear they are not a replacement for user testing and should not be seen as such.
It is clear that a Heuristic Evaluation is a great tool when assessing the usability of a design. It can offer the opportunity to spot issues early in the design process or offer a clear starting point for areas to improve in a redesign or when compared to a competitor. The key thing is not to mistake a heuristic evaluation as a substitute for usability testing with genuine users, the two techniques can work well when used together however heuristic evaluations do have limitations that mean a usability test will offer the best results.