First, the most obvious grouse someone has against this I can think of is the intricacies of an actual method. It's not enough to merely ensure no errors are thrown. Functions usually contain alternate paths borne of if/else constructs. For this, assume I use a declarative syntax that causes the generated/in-memory tests to follow all those paths. Also assume test framework can detect valid input from the endpoint's validation rules.
Another common test expectation is during unit tests, that certain methods return specific values. Or, during integration tests, that interacting with elements on a page matches asserted values.
Here are my questions:
- why isn't it sufficient to know method executed successfully, or that the return type conforms to method's type-hint?
- Is it really scalable to edit our tests each time an expected specific in the implementation is replaced?
- Why doesn't integration tests involve the above described procedure replicated for each of the other endpoints part of that feature?
Apparently, the inventor of PHPUnit itself embarked on a similar undertaking some years back, but unfortunately deserted it. What challenges could be behind his resignation of defeat? This SO answer hints thus
it's also an endless long-term project
I don't have enough forum rep to ask what could possibly protract its completion. Why is this seen as such a dangerous form of securing a codebase?
I disagree with the comments by @jonrsharpe and @user253751, and this is my defense. I am only interested in testing action methods tied to all defined routes. I have a test case for
/is-prime-number/5. The handler calls an internal/private method that decides whether or not a number is a prime number. Why do I have to redefine another method within the test case that determines whether or not its main counterpart is correct? If I got the SUT calculation confirming a number is a prime number wrong, what's the guarantee that the test case calculation will be correct?
The only way this can be avoided is if it's another developer writing the tests for someone else's code, which in my experience, is rarely the case.
The code already knows what it wants to do. Tests ought to confirm it successfully executes its desires both during runtime and at compile time. Such specific domain specific semantics as whether the number is a prime number in real life should exist within the scope of the business logic itself.
Users on this board are downvoting me because they incorrectly think I'm referring to functional testing. I want tests that insulate updates from creating regression bugs. The contents of the response have been manually tested via the browser/console/postman/network tab etc. and are certainly satisfactory to me before I'm ready to commit/run tests