Facing quality issues

Question

A workforce management software has complex GUI (for example values in a page depends on the status (closed or open) of other pages). Only latest and near past development has test coverage. During our last release, we received lots of bugs from customer in-spite of 2 weeks of testing Sprint. We don't have dedicated test team. The developers does the unit test and User acceptance test. Every day triggers automated regression test.

I am afraid the developers are not testing the entire workflow because its time consuming also not able to automate it because of its complexity. Any suggestions? The legacy code (15 years of development) has less code coverage. How can I improve quality?

Note: Now not possible to hire testers to have independent test team.

Answer 1

It sounds like the complexity of the legacy system is such that none of the developers really understand yet all the dependencies and coupling. In being assigned user stories they are modifying code that is causing bugs in other areas and functionality because of this unfamiliarity.

This is a problem that in the short term can only be solved by setting aside additional hours for user acceptance testing for every user story. Without a dedicated tester this will significantly slow down development time.

In the long term however, refactoring and lowering coupling along with gradual incorporation of unit tests will help the software become more stable.

You have stated several times already that hiring a dedicated tester is not possible, but it is important. Try to convince management that it is a good idea. Testers cost less money than developers, so if a developer is performing the user acceptance testing then you can argue that the manager is paying too much money for testing work per hour. If you think about it, hiring a tester will save the boss money.

Answer 2

One of the reasons why developers can't effectively test (the way a dedicated QA does, not unit testing which all devs should do) is because they know what they did in the code. They therefore won't push the edges of the system. They won't do the things a user might do because they know it wouldn't work. Therefore they don't find the bugs, the unit tests don't catch. Further, I have never seen application developers effectively do integration testing or regression testing. All they care about is their one small piece not how it fits together with everything else. You need this kind of testing and you clearly aren't getting it.

You really need professional qa to catch this stuff before going to production. It saves the company money and time and embarassment where the users find the bugs.

Since you don't have professional qa, you need to at least have different developer do the testing than the one who wrote the code. Someone whose goal is to break the other person's code. If you can borrow some users to do user testing, that is even better.

You also might try assigning one developer at a time to be the QA for a Sprint (you can change each sprint if you have to) that person will develop the other kinds of tests beyond unit tests and will write test cases for user testers to perform. You might find someone who is really good at it and who enjoys it and suddenly you have the start of your QA team.

Answer 3

There is no scientific way with which you can guarantee quality without allocating resources in your plan, preparing comprehensive test cases and executing those cases.

Face the fact that the system has issues and start a project to fix it. Define test cases that cover the scope of your project's responsibilities accurately. It may be the case that your are not responsible for the legacy system. Make the management aware of this. Construct small teams that could work on entire flows. The individual developer may not be familiar with the entire flow, so your small teams should be constructed with that in mind.

Answer 4

Well you don't strictly need a dedicated test team, you can have your developers also develop the tests. It's not the best, and of course, when they're testing they're not developing. And switching back and forth carries some overhead with it. But you can double up. All it takes is time.

Having daily automated regression testing is good. The fact that it still let a lot of bugs come through means your tests aren't worth much. They need to be further developed. And 2 weeks to develop test scripts for a complex system isn't much. There's a reason people get hired full time for this. How long did it take to develop? It'll take about that much time to fully test it. Otherwise, expect to spend 80% of your time debugging (that's 4 times as long as development).

But fix the incoming bugs first. Log them, and develop test cases into the regression test. When they have time, the developers should work on developing the test cases. Don't be afraid to get them extra hardware to mock input.

Answer 5

You are having( or started having) Unit tests, which is really good to start with.

As you are saying this is a complex project, a separate Quality Assurance team is must. A developer testing their code does not look at the application with the users perspective and they will be able to design and execute the test cases which gives higher level of confidence.

Until the QA team is established , Ask the developers to follow steps to increase the test coverage for your application and increase the confidence on the application.

1- Keep adding unit tests for all new codes and bug fixes.

2- For Functional testing:

a) Write down the test cases and get them reviewed with users. You don't have to write detailed documents but bullet points are enough ( like , Open the timesheet of an employee , approve the timesheet . Change the schedule to crossing midnight, adjust and approve the timesheet etc).

b) execute them and if you find bugs, don't stop and fix the bugs. But document the bugs and fix them at the end and release. Test them in Non dev environments.

c) I have also seen issues coming in prod for some users/ or some specific scenarios.. That is a clear indicator of some of the data is incorrect and other data is fine or the design of the application has issues.

so If you still see many bugs are creeping up, the problem must be at the design level or the data level. Make sure the data is clean and db has enough constraints. For design issues, you will have to refactor the application slowly.

Good luck.