A guide to acceptance test-driven development (ATDD)

When you go to develop a software application, there are a number of different methodologies that you can use to structure and streamline your development process. One of these methodologies is acceptance test-driven development (ATDD). ATDD fosters a collaborative environment ensuring all project stakeholders are aligned on the product requirements from the beginning.

In this article, you’ll learn more about ATDD, including its benefits, common pitfalls, and how to go about implementing it within agile teams.

What is acceptance test-driven development (ATDD)?

ATDD is an agile methodology involving collaboration among customers, developers and testers to define acceptance criteria before starting any development. This helps in facilitating alignment on the product vision across stakeholders. By focusing on the end product in the initial stage, the ATDD strives towards streamlining the development process while enhancing the quality and relevance of the software.

The discussion that happens among the customer, developer, and tester to create acceptance criteria is called three amigos. Three amigos represents the following interests:

• Business/customer — What problem is being solved?
• Development — How will the problem be solved?
• Testing — What could the possible outcomes be?

The goal of this discussion is to agree on what will be built and when it’ll be considered acceptable. Using this approach, the work should be reviewed incrementally. ATDD brings collaboration among different teams from the very beginning till the launch of the software.

The acceptance criteria defined together by three amigos acts as a guiding principle for developers to build the product and for testers to validate the way it’s supposed to function.

Alternatives to ATDD

ATDD is also known as story test-driven development (SDD), specification by example, or behavior-driven development (BDD). Test-driven development (TDD) is another approach similar to ATDD used in building software products.

Let us understand the differences between these methodologies:

Test-driven development

Test-driven development involves writing unit tests before coding. Focus is placed on developing small units of work in isolation. In this approach, a failing test is written first, enabling developers to write a minimum code to pass the test. It’s an iterative process that combines programing, unit test creation, and refactoring.

The TDD process comprises of red, green, and refactor phases:

• Red stage — In this stage, a failing test case is written on the feature to be implemented
• Green stage — Make the failing test pass as soon as possible
• Refactor stage — Improve the existing solution on top of the passing test case

Acceptance test-driven development

This approach leverages collaboration among the business, developers, and testers to describe the acceptance test cases from the end user’s perspective (user stories) and build a product that meets the needs of users. Here are the steps involved:

1. Discuss — Identify the problem to be solved
2. Distill — Write the acceptance criteria
3. Develop — Write the code to pass the test. If test cases fail, the code is refactored until it passes the defined criteria
4. Demo — After a small unit of work is done, developers demo the software application to stakeholders. This helps in getting their feedback timely and integrating any changes needed

Behavior-driven development

Behavior driven development is a combination of TDD and ATDD focused on end user requirements and their interaction with the product. The key difference between TDD and BDD is that the BDD focuses on testing the behavior of the application and user scenarios whereas the TDD approach tests the code implemented. BDD inculcates writing the test cases in plain English and hence eliminates any language barrier between technical and non-technical teams. Here are the steps in BDD approach:

1. Describe the behavior in the user story using Gherkin format
2. Identify the test scenarios based on the acceptance criteria
3. Develop the code based on the acceptance criteria
4. Test the implemented code
5. Refactor until the code passes the acceptance criteria

Benefits of acceptance test-driven development

The acceptance test-driven development approach has various benefits to enhance the quality of software and improve the team efficiency. Some of the key benefits are listed below:

• Improved communication and collaboration — ATDD creates an environment where developers, testers and stakeholders work closely from the beginning. This helps in shared understanding of the product requirements
• Clear product requirements — With defined acceptance criteria upfront, developers and testers are clear about what needs to be built and tested
• Early feedback — As soon as implantation starts, feedback is shared based on the smallest unit of work implemented. This enables you to provide early feedback and detect any issues
• Increased quality — Meeting the identified acceptance criteria helps in achieving a higher quality product with a clean and maintainable code
• Enhanced customer satisfaction — As the acceptance criteria is defined by the business, validating the implemented code against the set criteria ensures a product that meets the customer needs. This leads to increased customer satisfaction
• Reduced rework and lower costs — Refactoring the code based on early feedback helps in reducing the rework that needs to be done. This helps in saving costs as fixing issues early in development is less expensive than making changes at later stages of development or after deployment
• Continuous integration and continuous delivery (CI/CD) — Automated acceptance tests allow rapid integration of changes and continuous delivery of software. This automation ensures that new features can be deployed quickly with confidence
• Promotes test automation — ATDD encourages the automation of testing processes, which leads to high development pace while ensuring the software remains stable and reliable

Common Pitfalls in ATDD

As every methodology comes with its own challenges, ATDD has some common pitfalls too. Addressing these challenges early can help in implementing ATDD effectively to reap its benefits. Let us take a look at the most common pitfalls encountered with ATDD:

Complicated acceptance criteria

Complex acceptance criteria can be difficult to understand. If the testers and developers don’t understand the acceptance criteria clearly then it can lead to building a product that doesn’t fully meet the criteria. This can lead to delay in meeting the timelines and wastage of resources. \

Mitigation — Keep the acceptance criteria simple to understand. Clarify the need by providing additional information such as examples in the acceptance criteria

Insufficient collaboration

ATDD is based on collaboration among team members and stakeholders. If any of the team members or stakeholders don’t participate and express their thoughts then it can lead to misunderstandings about product requirements. Ultimately the features built could not be what was initially expected.

Mitigation — An open communication culture and frequent discussions with the key stakeholders will help bring in more participation and collaboration

Lacking test automation expertise

ATDD needs automated testing while building a complex software to ensure acceptance criteria is met as the software evolves. Teams without expertise in automation testing can struggle to implement ATDD.

Mitigation — Invest time in training the resources on automated testing or hire external experts to help set up an effective automation framework

Inadequate maintenance of test cases

As the software grows, and complexity increases, it’s crucial to adjust the test cases as per the changing requirements. Failure to maintain the test cases can lead to regression defects.

Mitigation — Regularly review the acceptance test cases and update if required as per changing requirements

Refactoring multiple times

Until the code meets the requirements identified in acceptance test cases, the developers need to refactor the code. However, refactoring the code multiple times to meet the exact acceptance criteria might turn out to be expensive and lead to delays in completing the projects.

Mitigation — There should be a trade-off between meeting the exact design and functional specifications per acceptance criteria and the effort involved to meet it. Issues/defects with minimal impact on users can be scoped to be fixed at a later date to speed up the time to market

Implementing ATDD in agile teams

Acceptance test-driven development within agile teams encourages collaboration, brings clarity in requirements, and ensures the end product is aligned with the stakeholder’s expectations. Let’s now understand how to implement ATDD in agile teams:

1. Understand product requirements and define acceptance criteria — Understand the problem that needs to be solved and what would be the solution. Define the acceptance criteria that outlines the solution to the problem
2. Write acceptance tests — Start writing the acceptance criteria that defines the solution of the problem from the user’s perspective. These are generally referred to as user acceptance criteria in user stories which acts as a guiding principle to build and test the feature
3. Implement code — Ensure the developers and testers understand the acceptance criteria defined in user stories before implementation starts. Once all the teams are aligned on what needs to be done and how it will be accepted, then implementation begins
4. Run acceptance tests — After a small chunk of work is implemented, the code is tested to validate if the acceptance tests are met or not. If the tests are not met, the code is refactored until the acceptance criteria is met
5. Review, iterate, and deliver — The software is continuously reviewed with the stakeholders to address any feedback while development is still in progress. Once a satisfaction level is achieved, the software is ready to be launched. However, even after launch, based on customer feedback the product requirements are reviewed and scoped for further enhancement

Tools and resources for ATDD

Automated testing tools can help run automated tests and reduce the manual effort needed leading to reduced time to market. Here’s a list of useful tools for implementing ATDD:

• Cucumber — A popular tool for behavior-driven development (BDD) and ATDD that allows you to write acceptance criteria in plain language (Gherkin). It then automates these acceptance criteria as tests that can be run against an application
• SpecFlow — Similar to Cucumber but tailored for .NET environments, it allows the definition of acceptance tests using Gherkin format
• FitNesse — An open-source automated framework to enable customers to create test cases in a wiki format. It allows you to create web pages with test tables that have test data
• Robot Framework — Another open source, generic test automation framework for ATDD. Its keyword-driven approach provides a simple syntax for acceptance test cases creation. Robot Framework works well for testing web applications and supports various technologies
• Selenium — Primarily an open-source tool for automating web browsers, Selenium is often used in conjunction with ATDD to automate acceptance tests for web applications. It cannot be used for desktop and mobile applications

Final thoughts

ATDD bridges the gap between technical and non-technical teams, fostering a shared understanding of product requirements and business goals. The software industry is evolving continuously, however the principles of ATDD — clarity, collaboration, and customer focus — remain relevant and enable teams to deliver a successful product.

When it comes to customer needs, ATDD provides a robust framework to deliver high-quality software. Adopting the ATDD methodology is a step towards achieving excellence in building software products with reduced costs and quicker time to market.

Featured image source: IconScout