UAT organisation and management

Introduction

User Acceptance Testing (UAT) is one of the most critical phases for any software project, since its ultimate goal is to establish that the business users, and therefore the business, are happy with what has been delivered. Our team of QAs have extensive experience of preparing, managing and supporting the UAT cycle, aiming to make sure that its goals are achieved, while the risks are managed as much as possible. This blog entry aims to distil some of that experience in ways we hope you’ll find helpful in managing your own stress free UAT cycles.

What is UAT?

The User Acceptance Testing (UAT) cycle is a testing phase where the final users, or their representatives, are able to test a release candidate of the application and approve it before it is deployed in a production environment. The objective is to reduce the risks of problems with the final release candidate, before it’s released into a production environment. During this phase the users might also provide feedback, which will be prioritized for future releases.
Each project is different, and has different requirements – but we’ve found there are some general recommendations when preparing for and running a UAT phase.

How to manage the risks

As with any Quality Assurance activity, UAT is about the management of risk – most notably that the release candidate does not meet the client’s requirements. For any type of testing it is better to “shift effort to the left”, and reduce these risks proactively and in a lower environment, rather than wait until UAT. You can reduce these risks by:

Involving stakeholders from the start

At Adaptive, we offer our clients a dedicated design phase at the start of any engagement, incorporating collaborative workshops with functional experts, technical architects, delivery specialists, UX, and QA. We find this to be invaluable – we get a lot of input from the stakeholders, and, in some cases, the final users. Any UX designs or high-level functional backlog produced will give the project a real head start.

Having regular demos

It is common in agile projects to have a demonstration, at the end of each sprint, where the whole team is present. Ideally stakeholders should be involved so that they can provide early feedback, or minor course corrections. We find this is really important, and clarifies everyone’s understanding of what’s been delivered.

Defining test scenarios early

Creating test scenarios early in the process will help craft agile user stories in a way that makes the QA process meaningful, maximizing the chance that the stakeholder gets what they want, while minimizing the possibility of regression.

Aiming for continuous UAT

Where possible, don’t wait until the final release candidate is deployed, to make it available to the user who will be performing the UAT. Breaking down the UAT into smaller increments, always defining the scope for each one, can amortize the risk across the entire sprint. Also, aim to have the UAT environment available as often as possible, so that users can test whenever they get the opportunity.

Avoiding assumptions

It’s often best not to assume that your stakeholders understand the software development lifecycle in depth, and that includes UAT. Having transparent conversations about the process as soon as possible will set expectations, eliminate misunderstandings, and make the whole process smoother for everyone.

Getting to know your client

Having QA as an ongoing client-facing role can really help clarify what is important for the client, what they want to achieve, and focus the whole team around the delivery of value.

Have a plan for your UAT cycle

The mechanics of the UAT process for a given project are sometimes prescribed by our clients, but we find having a plan for a UAT cycle focuses everyone around five key points:

• What is the scope? Are the test scenarios formalized? Any known issues?
• Who will be responsible for testing – is there a list of test users? Do they have access?
• Where is the environment for UAT? Is the data representative of production?
• When is UAT happening? Are the necessary stakeholders available?
• How will the feedback be accumulated and processed?

It’s also a good idea to create a timeline / checklist for UAT. We’d suggest something like the following:

As soon as the project starts and/or test strategy is defined

• Establish who will be responsible for running UAT
• Define what environment UAT will run in, and who will support it
• Request first drafts of end-to-end test cases from BAs / Product Owner

As soon as release dates / scope are agreed

• Request a finalized list of UAT cases
• Request list of users to participate in UAT
• Make sure the UAT environment will be available
• Create a shared document or wiki page listing everything you know about the release

Just before UAT starts

• Run some smoke tests on the UAT environment
• Set up the test users and verify their access

Risk management while running UAT

Below are some of the main risks that you could encounter during the execution of UAT, along with how we’d suggest you mitigate them.

Risk: There are issues in the system that fail UAT.

Mitigation: Perform all your planned testing before UAT, in a separate QA environment, to make sure to catch any major issues. Do an initial test run in the UAT environment before you hand it over to the user. Report any outstanding known issues before the UAT starts, along with a planned resolution date and advice on areas to concentrate on while these are being fixed, if applicable.

Risk: the users responsible for UAT are disengaged.

Mitigation: plan a training session for the users before the start of UAT, and be clear about what is expected of them during the UAT process. This situation is more likely to happen in a larger organization, so also see our tips for large scale projects below. Also, provide support, and work closely with the business users, resolving any queries as soon as possible.

Risk: the users are too busy for UAT – or start too late in the process.

Mitigation: communicate with the user as much as possible, and – if you can – do a test run together. Make sure that the users or their representatives report on the progress (automated if possible).

Risk: the users are raising a lot of non-critical issues.

Mitigation: provide guidelines on how to report issues, and how to categorize them by severity. Have them reviewed by a single person – preferably the Product Owner.

Risk: The user is blocking acceptance due to low priority items.

Mitigation: communicate the scope of the release clearly to the user, and agree the priority and type of issue that should block acceptance. Anything outside the scope should be prioritized for other releases. Working with the client upfront to define test cases can also give them a better idea of what to expect.

What about larger projects?

In the world of financial technology, projects often involve a large amount of interconnected systems, and consequently, UAT involves several disparate groups of users. In this environment, UAT can sometimes become significantly more difficult to manage. We recommend having daily status reports/meetings, with all the right representatives present, from any systems the product needs to integrate with. It’s also beneficial to provide anyone responsible for UAT with a handbook that includes everything they need – including test data, access links/file locations, instructions, what areas to concentrate on, how to report an issue, and any contacts they might need.

Conclusion

UAT is an important stage of almost every project, and keeping it in the forefront of your planning can really yield dividends, particularly in managing risk. At Adaptive we have extensive experience in running and supporting our clients during the UAT phase, and we hope the advice above will help make it a positive and successful experience for you, too. We’d love to hear your comments and ideas on this article – feel free to get in touch with us at info@weareadaptive.com.