I prefer to test the interface, not the implementation.
If your consumers have access to your code, tests are a form of documentation that is guaranteed to be up to date. If you only test for 200, consumers may come to rely on that, and changing to 204 (always or sometimes) is more likely to be a breaking change.
From a more selfish point of view, tests ...
It is not only acceptable: it's recommended! Your tests should be according to the contract that your API specifications promises.
Separation of concerns: tests and implementation can evolve independently but still tell reliably if the implementation is valid or not;
Maintainability: You will not have to remember some unnecessary coupling that ...
The term QA (or Quality Assurance) may not be used often in open-source projects, but a large number of projects do have QA practices.
The term QA does not refer to a team of testers, but rather it refers to practices that are designed to assure the quality of the product remains high. These practices include, but are not limited to
code reviews (possibly ...
So if you get a status 200, that is correct and your test must pass. And if you get a status 204, that is also correct and your test must pass.
Let’s think hard about this...
Yes, your test must pass in both cases.
(This is ignoring the fact that a server must always be expected to return all kinds of errors due to not functioning correctly right now, ...
I think you should have two different tests.
You return a different status code based on what the value is in the database. You have defined two different behaviours: when the database is in one state, you return 200. When the database is in another state, you return 204.
But this is a unit test of your web-service - so the test should be decoupled from ...
How does the new employee know whether Joe's changes broke feature 2345 if he doesn't know how feature 2345 is intended to function?
That is not necessary in most cases. Usually it is sufficient to check if Joe's changes did not change the previous behaviour of feature 2345, not if it works as its described in some documentation. So if in doubt, one needs ...
Regardless of if you are a half-man team or a 200 man team you need to be clear about what your goals are. Which mean yes there must be project management.
Now you don't need to draft a two hundred page word document with signatures, and graphs, and links to further reading. In fact please don't do that.
But you do need to be clear about:
Is it acceptable to test my implementation like this, just because the interface allows both values for the same case (and it's not really important)?
So this is a good example of your tests leading development. It is a design smell that the response from the server is not important and 200 and 204 could be returned by the same call to the server.
In the ...
The challenge that you express is the following:
Faster alone, further together
- African proverb
First of all, you shall not see testing as a luxury for bigger team. Writing some unit tests along with the software helps you:
to avoid loosing time on finding bugs later;
to do more tests and thus deliver higher quality despite your few resources,
How to deliver fast and avoid growth challenges with a codebase?
These are commonly at odds with each other. Fast delivery can mean cutting corners and accruing technical debt, which leads to growth challenges when the codebase needs to adapt to new requirements.
The answer is a balance between the two, and it is contextual.
In very short
Yes, the synergy with a light user documentation can be a good solution for your problem.
All the details
The ideal approach, from a quality assurance point of view, would of course to have fully automated test that you can run systematically after each change, to check the regression. If you can go into that direction, go for it.
Having a dedicated QA team in the way that proprietary software often have makes less sense when you have an open development process and openly accessible bug tracker, as a good chunk of your most important users will be watching activities in your repository, testing prereleases (nightly/alpha/beta builds), filing bug reports, or even be involved in the ...