20
votes
Accepted
How to get flaky tests fixed after having mitigated their flakiness
I've been in a similar situation a few years ago: the original external team had been asked to create tests, and had effectively only created acceptance-level ones, running the whole application and ...
- 11.3k
15
votes
How to get flaky tests fixed after having mitigated their flakiness
I'm adding this answer to address a different point that I think is more important than the specific on how you could've written the codebase (which is important too, but it feels like you're capable ...
- 47.7k
7
votes
How to get flaky tests fixed after having mitigated their flakiness
you could run the test as many times as you liked and it would continue to fail.
That’s called deterministic.
Sometimes one test would change state of the application and another test might rely on ...
- 105k
6
votes
How to get flaky tests fixed after having mitigated their flakiness
Anyway, my immediate observation was that lots of these tests unreliable in the sense of would sometimes fail for all sorts of spurious reasons, test issues, like not waiting for something to be ...
- 203k
3
votes
How to get flaky tests fixed after having mitigated their flakiness
Interesting problem, but one detail caught my eye: There are genuine failures which do not occur every time you run the test (timing issues, race conditions, memory corruption etc.). In fact, those ...
- 1,974
3
votes
Why are logs not used for testing?
Why logs are not utilized for testing?
Timestamps.
Testing is about verifying expectations. Record a golden log to regression test against and when you run it again in a test all the times are wrong.
...
- 105k
1
vote
How to get flaky tests fixed after having mitigated their flakiness
Flaky tests are a reality, unfortunately.
Very often the source issues of the problem are common to multiple tests, and it's important to attack those first.
I have seen tests fail because they ran at ...
1
vote
How to get flaky tests fixed after having mitigated their flakiness
Consolidating the integration tests that rely on specific states can reduce their resource load, freeing up the other resources for other, later tests.
This may not be solving the entire problem, and ...
1
vote
Load Testing for Large-Scale Architecture Involving Cloud Servers and Switches
However, our tasks must pass through a cloud server that only recognizes real switches. Unfortunately, this cloud server lacks a mechanism to recognize simulated or 'fake' switch signals.
In that ...
- 12.5k
Only top scored, non community-wiki answers of a minimum length are eligible