Should Acceptance Criteria be mapped to unit tests?

Question

My organisation is adapting a new SDLC process, and a part of this is to force engineers to map acceptance criteria (AC) to unit tests written for each user story in the pull request (PR).

To satisfy the enforcers of this SDLC process, our team wants engineers to manually create a table in PR description to map acceptance criteria to unit tests. This is how the table looks like in our PRs:

As an engineer, I feel they have completely lost their mind. Imagine creating this table each time while raising a pull request, especially when the amount of unit tests are more. Moreover, I don't think this is the purpose of unit tests. Unit test is just a method that instantiates a small portion of our application and verifies its behaviour independently from other parts. Mapping them to acceptance criteria is illogical in my developer's mind.

It would have made sense if acceptance criteria are mapped to test cases from QA perspective, but that's just my opinion.

So I wanted to know if it is standard practice to map ACs to unit tests? Also, does it make sense to create such a table in our pull requests?

Answer 1

I've said it before and I'll say it again:

the most important thing to consider is a test's audience

You're grating at this request precisely because this busy work could force the tests audience to change which changes the purpose of the test. That can have a profound impact. If that's where this is headed, stop it now.

However, while these tests are not identical, they are linked.

That is, this working

AC1 Verify that on the page load customers list is displayed  

depends on this working

GetCustomers_WhenCalled_ReturnsCustomersList

so long as the page load uses GetCustomers.

Now that is something you can map. You can map it with a comment in the test: //Supports AC1

Standardize that comment and you can automate the mapping. But all this documents is that, at one point at least, AC1 depended on the method tested here working. That can change for reasons that have nothing to do with this methods implementation or its unit test. That dependency is controlled in the page.

And that's the rub when you start documenting things like that. You're actually making it harder to make changes by doing this. Because now you have to track down everywhere this dependency is expressed any time you make a change to it. You're going to hate that every code change means changing everything it's no longer using to keep this report updated.

The payoff is less manual testing. If people can trust the automated test suite (which is more than just unit tests) you don't need to wait through 6 months of manual testing before each release.

So I'll emphasize this again. These are different tests. They serve different audiences. They mean different things. Each can pass and fail independent of the other. But if no one's changed how the page works the page will fail to load when the method fails. This can be a useful thing to report. But it is not a change friendly report. This report locks you down. Therefor automate the report as best you can. Make it as easy to update as you can. Because otherwise you're casting your code in cement.

^{Now if you had a tool that could automate mapping from AC1 to the page load, to the page load methods, to those methods tests, to the results of those tests, all without that stupid standardized comment, well then you'd really have something nifty.}

Answer 2

I feel that part of this is common, BDD/Gherkin/Cucumber style testing where you write tests as acceptance tests and have them print out whether they work or not in that human readable format.

Adding a hand written table to a PR... I mean you can see the motivation but surely there is a better way.

• Tag the tests with the acceptance requirement Id and then just check they passed and the requirement Id is present?
• A custom test report?

You can think of a dozen ways but they would all need programming work, where as this is a quick human thing.

You probably have someone who wants a report and needs help generating it from the test results.

Answer 3

People often abuse the term "unit test" for any kind of automated test which is executed by some unit testing tool. So first thing you should do here is make a clear distinction between

• acceptance tests - which only test acceptance criteria

• unit tests - which are exactly for what you wrote (testing a small portion of your application etc.)

That should not be too hard to explain to your "new SDLC" promoters, as well as the fact that acceptance tests are far more often end-to-end tests, of which you will usually have orders of magnitudes less than unit tests.

Regarding the "table requirement": this can be easily resolved by first having all user stories with their acceptance criteria inside some ticket system or other database with some id, and annotate your acceptance tests with that ID in code, just as Ewan already wrote. Then it should be simple to generate the table automatically for anyone who wants to see this table.

However, you should also check if there is really someone in your organization who uses this kind of table or reference. Otherwise, you are just adding formal overhead for no benefit.