Of all forms of testing, user acceptance testing is often the most essential to get right. Why? Because it saves you money. Too often I find organisations underprepared for the task of doing UAT properly and I believe this is largely due to the organisation not knowing the true impact of NOT doing UAT properly.
In 2017, IBM found the cost to fix a bug found after a build was 4 to 5 times higher1 than if the bug had been found in the design phase. More alarmingly, it was 100 times higher if the bug was found after deployment to production.
4 Steps to getting UAT right
- Define your UAT team
It’s important to avoid restricting your UAT team to project team members. The most important group to include in UAT testing is the “real” end users of your solution. And the key word here is ‘users’. This is crucial because they’re the people who will use the solution daily. Every persona and stakeholder group should be included, which means that people from each group should be selected to join the UAT team. Prepare that UAT team early and enable focused UAT to happen by booking in the UAT resources ahead of time. Consider booking time in your UAT team’s schedule to enable them to adequately perform UAT. - Confirm test cases and acceptance criteria
Agree upon user stories and their acceptance criteria during the design phase of your project. Each user story should cover a specific use case or scenario of the solution and therefore lends itself to being turned into test cases for your UAT. The test cases are normally a set of actions which the UAT team member can carry out to verify if the solution has worked as intended. The acceptance criteria define what is considered to be “working” in the solution. - Develop a UAT plan Consider the following when writing your plan:
- When will UAT start and finish?
– It is important that a finite date is applied to the UAT period to avoid perpetual testing. - How will the test results be collected?
– Write down how the communication between your project team and your UAT team will take place while testing. Doing this prevents the project team from the nightmare of receiving emails, word documents, spreadsheets, screenshots (or no screenshots) and endless discussions through email and other channels.
– Collecting and managing bugs in a central repository will also mitigate the risk of duplication of bugs, as well as assist with the management of bugs through the correction and retesting process. - How will you label any bugs found (bugs, feature-requests, usability, training, etc.)
- Who will triage the results?
– Remember: the triage process can bury you. Plan ahead and appoint someone to manage the triage and confirmation of defects.
- When will UAT start and finish?
- Conduct UAT
Provide the UAT team with the list of test cases (as demonstrated in table1 below) as well as for instructions about how to log a bug well (i.e. the bugs should include the account used for testing, the device used for testing, screenshots, URLs, description of how to reproduce, etc.). Consider holding a UAT ‘briefing session’ to take the UAT team through what is expected of them throughout the UAT period.
With the UAT plan in place and the UAT team readied for the task, the UAT can begin with the tests being performed and the results recorded. Were the tests successful, or did defects result? Any bugs raised then need to be triaged, confirmed as defects, corrected and re-tested.Table1 – Sample UAT test caseSample test case:
As a user, I can see news on the home pageID Acceptance criteria description Pass/Fail/other Comments 1234 Verify that you can see a news carousel on the homepage 1234 Verify that the news carousel auto rotates 1234 Verify that you can click on a news image and be taken to the news page 1234 Verify that when you click on the navigation arrows in the carousel, the image displayed changes. - Close and sign off UAT
Once all testing is complete and all bugs are corrected, the project team should conduct a UAT closure meeting. During the meeting, you should look to tie up any loose ends, and formally close the testing period. Signing off or closing the UAT period means that you can move your solution into production.
Many development teams, including Engage Squared, apply an agile approach to projects with continuous integration and continuous testing. Agile’s incremental approach allows more feedback, flexibility, and of course testing, so that every time a feature, fix, or function is added or changed in the code it’s checked for bugs. In turn, we have found this helps avoid preventable bugs – ultimately saving time and money.