New answers tagged

1

If you are concerned about an integration test that interfaces with an unreliable external system, then one thing you can do is have a separate testing phase on your automated tests for the external test. This should not fail the whole build, if this phases fails (it may indicate a bug or it may indicate the downstream system has gone down), but indicates ...


3

Testing with stubs/mocks/fakes is good, but it can never fully replace system and integration tests where the real thing gets used. If you don't have enough control over the external service to reliably use it in an automated integration test, then you should perform a manual integration test before you merge your work to the 'master' branch to prevent just ...


2

When you are writing mocks for the upstream services you depend on, you are matching the service's behavior for a limited number of cases. If you get a 4xx error from the service in production, it means your mocks are not accurate to the service's actual behavior or the upstream service introduced a breaking change. The best way around this is to checkout ...


2

Not having separate QA is a very bad thing. At the most basic level, the software developer wants the software to work to demonstrate they did a good job, and the QA person wants the software to not work to demonstrate they did a good job. You can't do both at the same time.


0

Create new branch off Develop (assuming develop is all GOOD code and the main trunk. master is inline with develop.) Dev works on this branch, code completes, raises PR against develop. publishes PR to team inclusive of QE and Devs Once approved PR build passes, QE can take this up for feature testing on lower test environment. Assuming meanwhile develop ...


0

It is the same metric as when you're testing your product manually. Practically, it's easy to identify these low-confidence zones: assuming that you are shipping the product, I suppose you have some post-pipeline manual steps that improves your confidence of being shippable. These are the areas you should automate to improve the confidence in the automatic ...


2

Is trackUserPerformance within the data access layer? If no, then you should consider adding the layer. A common—although not the only one—approach is to separate applications into presentation layer, which handles all the UI logic, business layer, which contains the actual logic of the application, and the data access layer which provides persistence. If ...


0

I think it's worth keeping in mind that, if you are running tests as part of your development/design loop, it doesn't matter very much how to describe a test failure. You don't need a lot of information emitted by the test when you know the root cause is the line of code you just changed. Where it tends to matter is when a bunch of new code gets merged in ...


7

Ask yourself what unit tests are good for. The main purpose is that after changing your code, you run all the unit tests, and if they all pass, you have a bit more confidence that your code is fine. For this purpose, what matters is the number if independent assertions that you passed, so for this purpose a single unit test with 100 assertions is no problem. ...


4

Each unit test should assert a single requirement. Now, you may not always have formal requirements for every method. Methods are written in pursuit of fulfilling such formal requirements. But given that a method should do only one thing, should have a single purpose, you should already have a fair idea of what to expect from the method in terms of ...


0

You are agonising over a technique, rather than the goal. The goal is: How will I know that the code has done something wrong? Sometimes the answer will be: I wrote a unit test, and it is failing. But other answers include: I've analysed the algorithm and traced out every outcome, those outcomes precisely meet my understanding of the problem. Therefore ...


0

I think you're going to find many different answers on this topic, since there is no one, clear cut way to approach testing. Unit tests are generally low level tests that are used to test specific methods or bits of functionality in your code. I agree with the other answer that mentions testing public interfaces as this will help prevent regression. I ...


1

Exercise the public interface of all classes and all utility functions. Doesn’t matter how “small” the feature is, you can still get regressions.


1

The trick here isn't to worry about about complete coverage but in managing the risk of your changes. Let's say you're using your pipeline to deploy the exact same version as is already in Production - what's the risk of regression error? Zero (because there's no change). Now, let's say I want to change a piece of text on one of the screens. I've added the ...


2

When do you have enough automatic testing to be confident in your continuous integration pipeline? In most economic environment you will not have the budget to implement enough confidence (> 99%) but you have to manage a limited budget: It is all about the cost/benefit ratio. Some automated tests are cheap to implement some are extremly costly. ...


16

In General When do you have enough automatic testing to be confident in your continuous integration pipeline? The answer probably becomes clear if you think about what you want to be confident about. Ultimately, it maps 1-1; every test makes you confident about the one thing it tests: Unit testing gives you confidence that a class (or module) does what ...


4

There is no metric you can calculate that will give you the confidence you are looking for. Confidence is built by doing something, and then succeeding at it or failing and learning something from it. The only "metrics" I've found that gives me confidence in our test coverage is: Number of defects found in production Can you refactor the code base and rely ...


3

Is this use of terminology something unique to this person? I have no idea if this specific use of terminology is unique or not, but it is not uncommon to find teams or persons in the software industry using terms in a quite unusual, non-standard, or imprecise way. For example, due to the popularity of tools like JUnit or NUnit, I have very often heard ...


2

A unit test is an (almost always) automated test which verifies the behaviour of a small, isolated unit of code. This unit of code is usually a single method or function. Any more than that and you start getting lots of scenarios. There are normally multiple tests verifying a single piece of code with each testing a specific scenario. Generally a good unit ...


2

They are completely different. Unit testing is concerned with verifying that small chunks (functions/methods) of code work in isolation. They should test cases of common usage, edge cases and any case in which an error can occur. For example, if you have a function that adds two numbers, then your unit test must test whether or not the function calculates ...


1

You are looking at a combinatorial problem: You have 3 columns, so you need all combinations of 3 columns: [[],[1],[2],[1,2],[3],[1,3],[2,3],[1,2,3]] where [ and ] enclose a list of included columns. And yes, [] is a valid combination. You have 2 filters available for each column, plus an unfiltered state. You also have ensure that you have data that ...


0

Your 5 columns and 6 filters example is simply a nested loop. for (int column = 0; column < numberOfColumns; column++) { for (int filter = 0; filter < numberOfFilters; filter++) { RunTestForColumnFilterCombination(column, filter); } } To use all bit combinations in a range: int getBit(int bits, int position) { return ((bits &...


Top 50 recent answers are included