A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.
38

End-to-end tests are also necessary. How else can you know you hooked up all the units together correctly? On very simple code, it is possible to test all the paths through the code with only end-to-end tests, but as you get more layers, it becomes prohibitively more costly to do so. For example, say you have three layers, each with five possible paths. ...


36

There was no test-driven development process during the development due to very tight deadlines This statement is very concerning. Not because it means you developed without TDD or because you aren't testing everything. This is concerning, because it shows you think TDD will slow you down and make you miss a deadline. As long as you see it this way you ...


30

A stub method is a method that just returns a simple but valid (though not necessarily correct) result. They are typically made when building the infrastructure and you don't want to spend time on every method needed right now. Instead you create stubs so everything compiles and your IDE's auto-complete knows about the methods you are planning to use. ...


22

Is it still okay or good idea to stop most of the development and start writing whole possible test cases from the beginning [...] ? Given legacy1 code, write unit tests in these situations: when fixing bugs when refactoring when adding new functionality to the existing code As useful as unit tests are, creating a complete unit test suite for an ...


20

Yes, end-to-end tests (or integration tests) make a lot of sense, but so do unit tests, too. Ideally, you have both of them, since both typically catch different kind of bugs. Thus, having end-to-end tests should never be an excuse for not having unit tests.


19

Am I doing it is right? If not what exactly I have to change It's hard to say just from that short description, but I suspect that, no, you are not doing it right. Note: I am not saying that what you are doing doesn't work or is in some way bad, but you are not doing TDD. The middle "D" means "Driven", the tests drive everything, the development process, ...


14

I think the answer is you both should be thinking about your own set of edge cases. He as the dev should handle edge cases that are data specific such as does the app crash from any given user input, 5 / 0 certainly falls into this part of the spectrum. The dev should ask about you what you think would be an appropriate error message when the input given as ...


12

Acceptance tests act at a very different level than, say, unit tests. A unit test is very precise: it deals with one method, sometimes a part of a method. This makes it a perfect choice for regression testing. You make a change. A test fails while it passed during the previous commit. Great, you can easily pinpoint the source of the regression both in time (...


12

In my experience, tests do not need total coverage to be helpful. Instead, you start reaping different kinds of benefits as coverage increases: more than 30% coverage (aka a couple of integration tests): if your tests fail, something is extremely broken (or your tests are flaky). Thankfully the tests alerted you quickly! But releases will still require ...


11

Is it true to say that the product manager and the QA should be responsible for taking the user-story and breaking it down to acceptance tests? Mostly. They may not actually write the actual acceptance test. They may approve something you wrote. But they approve the acceptance tests. Yes. the acceptance tests need to be formal, so they can be used as ...


11

Acceptance tests access the application through a special purpose API. You presented this use case: Given Michael has just been created in the application, his status should be left to non-activated. The API implied from this use case is something like: CreateUser(String name); enum UserStatus {non-activated}; UserStatus GetUserStatus(String name); So ...


11

we don't want to modify code that could be shared, because that will cause a big regression test impact Above sounds about right to me. The more important is the code, the more it is shared, the higher are quality requirements, the more quality assurance should be involved when it changes. Since your system is implemented in Java, you can see an example ...


11

The team needs to work together as opposed to having a "Not my job, not my responsibility" type of attitude/mantra. Acceptance criteria comes in the form of: Business Acceptance Quality Assurance Acceptance Typically the business acceptance is usually answering the question: Does the feature that has been implemented do what I want it to do? The feature ...


11

The way I've always seen it done is to have a single code base capable of serving both pages/views/forms. ie. Its Feature flagged and deployed with two or more configs, or the 'does the user get A or B' method itself is part of the app. In this case you only have the one version of the code, so your source control doesn't come into play. This is far more ...


10

I worked in the car navigation field over a decade ago. Step A) Use a reference package and select a large sample set, run A/B tests. Not looking for exactness, looking for outliers - The reference set showed Reroute 1234 as 10.34km, and we calculated 123.5km. Step B) - Refine our software and the reference software - Add more samples and reduce the ...


10

Modern games are actually a ton of creative art content developed using an in-house or proprietary game engine. The engine itself is unit testable for most part (rendering, geometry, physics, AI modules etc). Similarly, simple tests can also be attached to individual parts of developed content. This means unit and white-box testing is indeed feasible and ...


9

It personally sounds as if your Acceptance tests have encompassed properties of Integration tests and that you are trying to "kill two birds with one stone" as the saying goes. In the traditional Waterfall model a single Acceptance test should determine if a single requirement has been met. If developing based on a strict SRS document, you may find that ...


9

The acceptance criteria defines when the application is finished. Or to put it another way, when you can ship it. It includes list of requirements that it has to fulfill. This means that some requirements (usually "nice to have" requirements) may fall off, and be implemented in next version. To expand it even further (taken from here) : Microsoft Press ...


9

I don't think there exist any metrics to calculate "cost for regression tests / LOC of reused code built". And I don't think anyone had ever invested so much time and money to build the same "big" system twice, one version with a lot of resuable components, and one without, to make any serious research on that. But I have seen problems caused by reuse like ...


9

If you are developing an online system for your company, that would imply that somebody besides you must have an idea of how it supposed to work at least at the UI level. So that would be the level at which your work can be reviewed by others within the company. Unfortunately, that means that there is nobody to review your code. One possibility is that you ...


9

What you are mentioning is a classic problem that many companies have introducing agile development. Unfortunately, there's no easy solution. For the specific problem of acceptance, the "standard" workaround is having someone in the team acting as a proxy for the product owner. This is not great but it can work as long as this person takes care to demo ...


9

Yes. It tests that the roles are taken into account when checking for permission. It also tests that the permission for the role are setup correctly. Note that these are unit-testing the permission system, not whatever the operation with the permission is. So the tests will be creating a role and changing the permissions and testing a (mocked) operation to ...


8

On the one hand, the developers want bugs that are in the code that need to be fixed. On the other hand, how is QA supposed to know whether it's code, data, scripts, or human error? QA knows that it doesn't work. QA reports that it doesn't work. What happens if it appears to be because the delayed job wasn't restarted, but it was in fact a bug in the ...


7

When there is a dispute between the users and the program about how the program should work. The solution isn't to fix the users. That goes double when you're presenting data to domain experts. In this case if the devs and the users have different ideas about what data should be presented, then correct response is to make the data presented match the user's ...


7

Let's suppose that it works for email, but not name. Now your acceptance test will fail half the time. Just re-running the test will potentially change the result. That'll make it harder to notice errors and track down the problem with it occurs. It'll be way easier if you just write two tests, one for each login method.


7

In general no. For toy programs (say solutions to Project Euler problems), sure. This is before getting into a religious discussion of what are unit tests (if it uses a file system object, is it a unit test? Let the inquisition begin!) For N-Tier applications, where N > 1, integration tests are needed, and even they may not actually be suitable for ...


6

yes, there can be duplication. Often with line of business type websites, you find the acceptance tests can actually substitute for developer oriented tests, because the depth of code needed to implement features is not too great. As a comparison, image recognition software may have huge depth, but at an acceptance level it might be simple as needs to ...


6

After a few more years of coding and working on projects I'll provide an answer to my own question. Yes, you should write unit tests. End to end tests are harder to write and brittle especially if they're relying on UI components. If you're using a framework like Django or Rails (or your own custom classes) you should have a form class that will handle ...


6

...shouldn't it be the role of TDD to emerge the design and therefore these components? No. This is a common misconception about TDD. The purpose of TDD is not to "grow a design." The purpose of TDD is to insure that a program stays "well-designed." TDD will force you to create an API that's testable, and specific functional requirements will emerge ...


6

In practice, I tend to leave the design a bit rough when first starting out with an acceptance test, just introducing simplistic versions of concepts to make progress. The trick is recognise when you have enough information about the implicit design to refactor. Actually, I think you have a bigger problem with your first test. You're starting with the ...


Only top voted, non community-wiki answers of a minimum length are eligible