14
Sometimes you have some control over the timing, and you can intentionally force a race in a test. When you are using a guarantee of an external system like a database, you usually can't control the timing that precisely.
In that situation, we explicitly call out the requirement during our planning as an acceptance criteria. "Must prevent an ...
8
One possibility is to replace the scheduler with one that is under your control.
That is essentially how Microsoft Research's CHESS tool works. It uses static analysis to try and determine all possible data dependencies and interleavings of execution paths of your code, and then runs the code with all the possible interleavings by replacing the standard ...
6
The scenario should be set up in the Given step. This involves mocking parts of the system. However, a mock might be used in the When step when that's part of the action.
When reading your two scenarios, they both look reasonable as natural language:
Given a control panel in its initial state, And an engine that will fail to start, When the start button is ...
6
The main approach I use is to lead by example and pick your battles.
Large projects are almost never in great shape, because of the mix of talent, and because practices and code bases evolve. There's a lot of history involved that you can't see.
For leading by example, a lot of those things on your list are things you can just do. Before a regular change, ...
4
If you really want to automate this kind of test, your best bet is probably to build a specific "test mode" into the code section which handles the race condition. When this test mode is activated, there should be an artificial delay activated inside the critical section.
To make this work robustly and quick, I would not make this "delay" ...
3
If your organization is following the values and principles of Agile Software Development, then there should be regular reflection on how the team is working and the opportunity to improve. This is the only specific practice called out in the Manifesto for Agile Software Development. In Scrum, this general practice comes in the form of the Sprint ...
3
Sometimes code needs to change to be easier to test.
Many standard libraries have abstractions over files that can be used instead. For example, Java has an InputStream and OutputStream with both file and byte array implementations. If you have to make your own, it is a lot simpler, because you don't have to make it flexible enough for every single standard ...
2
Kent Beck actually answers this question himself if you watch a video of him teaching TCR. The answer is yes. You do skip "red" when you are practicing TCR.
I also found this article that directly states you skip the "red" of red, green, refactor when you practice TCR. And this TCR tool can only be used under that assumption.
I think that ...
2
This sounds like an integration test but, since textToPdf is just one function, I think it classifies more as a Unit test (where the unit of code under test is that single function).
The mark of a unit test is not "just one function". The mark of a unit test that there is only one component under test (i.e. real, not mocked, and the focus of the ...
2
I understand starting a vehicle is a representative example, so I will focus more on the Behavior-Driven Development and mocking aspects of the question.
I am unsure where/how to specify the behavior of a mock in a test scenario...
You don't. The behavior of a mock is transparent to the scenario. Remember that BDD focuses on the business process, not the ...
1
E2E Tests should depend upon built, versioned artefacts, not branches or source control.
In the same spirit of manual QA testers being reliant upon built, versioned artefacts, automated E2E tests should ideally follow the same mantra. For one, it's valuable to be able to redeploy those exact same artefacts in another environment if there's a need to ...
1
Although I've never worked in automotive and had to deal with ASPICE, this is very similar to some standards from the aerospace, medical device, and pharmaceutical industries so it's possible to draw some parallels.
I don't think it's safe to assume that the key point of interest is the failed part, although it's probably more important for failures in order ...
1
I you really want to test the details of DB operations in your code, the method is the same as always: you mock the component that receives the operations and assert that the received operations satisfy certain constraints.
For instance, you might assert that all the operations are wrapped into transaction boundaries (explicitly or implicitly). (For bonus ...
1
I hop in, as I have experience with many projects and company cultures. And you are on thin ice.
First you are in a non-scrum environment, the tag agile is so overused, and there are so many variations. Also there exists the attitude "do it simple as I understand it; no overhead." So if you want to see scrum, propose a workshop, best of a ...
Only top voted, non community-wiki answers of a minimum length are eligible