I recently joined a team of developers working in a software project for several years. I am asked to creat a unit testing test process to improve thier output and suggest new test cases and test strategies. I have been working before as a black box tester with a programming backgrounds i am not sure how to start and from where as the software they are working on is quite complex. If you could guide me from where to start.
In the first place you have to find out what the team means by "UnitTests".
I found that most teams do not distinguish between
- automated tests done with a UnitTesting Framework
"real" UnitTests verify a single expectation about the behavior of a unit in isolation.
In this definition a unit is
a small part of the production code with the same reason to change
(Roy Osherove, "The Art of UnitTesting")
UnitTests are a responsibility of the developer and best done following the Test Driven Developement approach.
automated tests done with a UnitTesting Framework
You can use any UnitTesting Framework to create other kinds of test as well, especially module test, acceptance tests, performance tests and alike.
The difference is that you test a part of the application at its "natural borders". In particular you don't use mocks created with a mocking framework as test doubles for parts not being in the tests scope.
This kind of test are the responsibility of a dedicated tester.
A simple thing is
test coverage. There are even tools which will generate the reports automatically, and then you can see which lines of code are well or not at all covered.
More important are the the "contents" of the tests. Do tests include the "happy path scenario" (where everything works as expected) only, or do they care for exceptions, for edge cases etc.? I do not know tools for that (but I won't exclude that they exist).
I'd also check if they are really interested in Unit Tests only, or also in Integration Tests, Load Tests, etc. Such tests can provide additional valuable information to the system under test.
a unit testing test process to improve thier output
- Unit tests should be required
- But judgement must be used when deciding how much is enough and how good the tests are
- Beginning unit testers must learn how to write code, and refactor as needed, that facilitates testing that code. Note I did not say beginning programmers; I mean new to unit testing.
- writes unit tests for code (s)he writes
- writes unit tests as the code itself is written - TDD - test driven development
- I promise you that it is harder to write unit tests after the fact.
If y'all do code reviews also review the unit tests
- Unit tests should always be run when your code is compiled
- Unit tests must be kept current. As your code changes, tests will have to necessarily change too. Do not allow broken tests to simply be turned off or deleted.
- One or more persons needs to be the evangelist to teach, encourage, guide, etc.
That is very basic process. There is also the technical implementation. You'll just have to research for your particular development languages and environment.
suggest new test cases and test strategies.
This does not make sense in the context of unit testing. The developer decides on test cases for the code (s)he is writing. I don't know what you mean by strategies.
Unit Testing has to be driven by the programmers, period. Any attempt to change that is futile exercise. So rest of the points is more focused on test driven development.
The product should provide a way to test api's (testability aspect of software quality metrics) to test functionality.
As a tester, your test strategy should be driven from the requirement perspective than what is written in code (In code there can be omissions which can never be caught from unit testing).
Once the apis(assuming apis are provided, can be selenium) are published, then test suite should be created in parallel to development.
If there is a continuous integration pipeline, then add the test suite execution as a job to it so that the developers can see the progress of the project.
Effectiveness can be measured in terms of code coverage/number of tests successful/failed
Your first priority is to get the tools working. Pick a testing framework that works for your combination of language, platform and IDE. Preferably a testing framework that support coverage reports and continuous integration (or find a way to integrate it).
While it is possible to have tests done by custom tools, creating such tool is not your current project. So, do not do that, get something that already exists.
Whoever is responsible for the code should write and review test, or delegate these tasks. It is not a good idea to have others write tests for the developers, have other review the tests may or may not work.
What tests to write?
Considering that we are talking about introducing testing to an existing project, you will not start with real unit tests. The reason being that we are not sure if the code is in good shape to do so. We do not know yet how easy it will be to isolate or mock components. The code may not have interfaces or dependency injection in place to ease testing.
Start by writing tests for your requirements or documentation. That is, if the documentation or requirements say that when you do X then Y happens, make a test of that. Do the same with contracts. If possible, convert these contracts into tests (not always possible).
Any defect found should translate into new tests. If you have the means to use tests to replicate a bug, do so. These will be your regression tests, which purpose is to make sure that when making changes they do not break what is already fixed.
Once you are done with the documentation / requirements, move to increase test coverage. The idea is to make sure that each odd situation is tested, and not use the code path of regular or intended usage.
Once you have added tests to increase coverage of a component to a point where you consider that the easier way to increase coverage is to refactor the code... then do so. The existing tests should give you additional confidence when refactoring. This is when you start molding your code to ease actual unit testing.
All the above is for legacy code. For new code, write tests before and during writing the code itself. Testing should be a consideration in the design of the code.