Whether you are a backend developer or a frontend one – you will know the pains of testing, user acceptance testing in particular. On one hand, it means that the bulk of the work is done – a nearly finished product is ready to be presented. On the other hand, that sometimes means that the worst is yet to come. Errors that are found during UAT testing can be pesky to get rid of if they managed to sneak through previous stages of testing.
What is user acceptance testing? Well, it doesn’t matter if the project in question is, for example, a booking app, or a browser-based game, the steps are very much the same. To take a step back, the idea of UAT testing is to simulate and test as many ‘real’ user scenarios as is possible given the time, staff and resources available for testing. As it is the last stage of testing, it is a very crucial one, since anything that slips past this stage will be included in the final product. This includes bugs as well as problems with usability.
UAT testing realities
In a perfect universe, there would be plenty of time and resources for this step, and all mistakes are found and rectified before launch. If you’ve done UAT testing before, you probably just chuckled. The reality is quite different. Usually, a couple deadlines have already been missed, and the lost time needs to be made up for somehow – this time is taken out of UAT testing. It is easily mistaken as one of the less important steps of testing and development in general.
After all, extensive testing has already taken place, so what is the point of having another testing stage? Investors and clients are often in a hurry and want to wrap things up quickly. Add to that the likely overworked programmers, or, if some were hired, overworked QA testers, usually also a lack of funds…it can become a recipe for disaster far too easily.
It can be avoided, though. How? By eliminating as many potential issues as possible. A fixed budget allocation can take care of most of these issues. It makes sure that enough funds are available to cover all the necessary work – no sloppy mistakes because of hurried work, and no fear of not getting paid for contractors. If the client allows it, it is also a good idea to draw in people unfamiliar with the project – the neighbors, your partner or the janitor. They will give an outside perspective, and their lack of ‘insider-knowledge’ (plus the fact that they don’t need to be paid) will come in handy.
Have a user acceptance testing plan
Now, unfortunately, the budget is very often extremely hard to control, and if you are a developer, possibly impossible to influence at all. Instead, there are other elements you can influence. The most efficient way to ensure smooth running is to pre-plan as many elements as possible, while also setting up a detailed user acceptance testing plan.
‘Elements’ like this could include a staff list – deciding in advance who will work on the UAT testing can ensure that everybody is prepared, aware of their role, and can effectively utilize their time. While it may be necessary to make some last minute amends or additions to the team in order to cover the workload, having a user acceptance testing plan in place will at least give you a steady foundation. And when the storm hits, that is exactly what you will need.
Come up with a detailed list of test cases
The next step is to set up a detailed list of user acceptance test cases. Here, the complexity of the project plays a major role – the bigger, the harder it will be to cover every eventuality. The idea here is to come up with as many natural user scenarios as possible. For an e-commerce website, this would involve things like adding items to the cart, checkout, selecting favorites, creating a personal account and so on and so forth. For an app-based game, this could include account creation, in-app purchases, device permissions and of course, gameplay. The earlier in the project this list is made, the easier it will be to include as many possibilities as possible.
What you want to do here is not to come up with fringe cases, but with real-life scenarios. So, while yes, the app may crash if it is run with 11 other apps open in the background, or if it is run in a (badly) emulated environment, but that is not what UAT testing is for. Instead, it is interested in whether or not it runs smoothly if Susan from next door uses it while taking a break from her Candy Crush binge.
During UAT testing, keep in mind the test-environment
A somewhat simplified example, this is still the idea behind of UAT testing. Another aspect that needs to be considered here is the test-environment. For a website or web application testing, that would be different OS and browsers, for a phone, make and model. You’ll be hard-pressed to find a web project that does not heavily feature on responsiveness and cross-browser availability.
Now, there are thousands, if not tens of thousands of possible combinations of device and browser. It isn’t possible to cover them all, and quite frankly if in 2017 someone wants to use an old Nokia 2660 (flip)phone to look at a website, that is their choice. It is however not part of the target audience, so fringe cases like this are pretty safe to ignore. In the utopian case that there are time and resources left over after regular testing is completed, this is certainly a good thing to look into, but primarily, the ‘likely’ cases need checking.
That generally includes popular phone models from the past 3-5 years and all the ‘big’ browsers as well as the default ones on the devices that do not already come with Chrome/Safari as a pre-set. There are lots of statistics available on what the most popular combinations are, so try to extrapolate from that, as far as (once again) time and resources allow.
Another thing to keep in mind here is the bigger picture. Always consider why a user will be using the product in question. If it is a game, to pass time. If it is an e-commerce type project, the user will want to (depending on the type), either locate product X and purchase it, or shop around, compare, and spend more time. You should tailor your test cases to the expected behavior of your typical users.
UAT testing: Good documentation is everything
All of that covers the preparation and planning of the UAT testing – but what about the UAT testing itself? Here, it’s all about the documentation. It can be dangerously tempting to not keep exact tabs on what happens in what case, or to only report encountered issues verbally – especially in smaller sized teams – but do not let yourself be led astray. Good record-keeping can make or break this least popular stage of development. If your product is web based, DebugMe could prove to be a great addition during the documentation phase of UAT testing.
Now, what is good documentation? Glad you asked. It needs to contain a minimum of information – what happened, what device/OS/browser did it happen on, is it unique or recurring, how serious is it (consider setting up stages like Low, Medium, High, and Show-stopper) and potentially proof/evidence like a screenshot, screengrab or the like. The more info, the easier it will be to resolve, and the less time is lost there.
Also included should be a reference to similar issues. So, say, for example, Colleague 1 has an issue displaying a logo on his PC using Chrome and Colleague 2 has an issue with the logo on his Galaxy phone using Chrome – the two are likely related and should be marked as such. The bigger the team, the harder it is to keep track so this would be a good task for a project manager. The devs will thank you for it (or if you are the dev, you will pat your own shoulder for it later!).
There is only one more aspect that we have not yet covered: the user ‘acceptance’ part of this testing. In addition to functional issues, UAT testing is where you find out whether users actually like what you built. What may seem perfectly obvious to a developer, could be extremely complicated to a user. The most flawless piece of code won’t take off if it’s hard to use. So, this is a good time to ask friends, family and the accounting and HR departments to pitch in on the UAT testing. Simple comments like ‘This button should be bigger’ or ‘The checkout process isn’t clear’ can turn your work from good to great. And when it comes down to it, that’s actually the easiest part to fix up. All the hard work is done, the last changes are probably more cosmetic than anything else. At this point, you will also find out if users feel that your project takes too long to load or is unpleasant to look at.
You will get some outlying results, but with a big enough group of testers, you will have more than enough to work with – and more than enough input to get a sign-off on your work, and you’re home free!