3

I am a team lead on a project that develops specialized engineering software. My developers generally have advanced degrees and the domain knowledge needed to work with the software we develop, but my testers don't have the necessary background knowledge. I don't have much flexibility in determining the composition of my team, so I can't hire new testers with an engineering background. What steps can I take to help my test team understand the software better -- or alternatively, how can I make it easier for them to do "black box" testing for which they won't need that knowledge?

5

If your test team does not have the necessary background knowledge, they are not going to be able to test your product adequately.

(This is a special case of a more general problem: If a team member does not have the necessary background knowledge, he is not going to be able to do his job adequately.)

You're going to have to figure out how to explain to your management and your HR people that your test team members need this background knowledge to do their jobs. This will probably have the highly-undesirable (from management's perspective) side-effect of costing money, in the form of (a) requiring the company to spend more money on their salaries, to be able to hire people with the requisite knowledge, or (b) requiring the company to spend money teaching/training them (and then quite possibly having to pay them more to retain them, now that they have the knowledge).

You would not ever, for example, ask a non-pilot to take a new airplane up for first flight test. He needs that knowledge. You MIGHT ask a non-pilot to try to test a new piece of avionics equipment, but you'd be smart to stick him in a simulator or send him up with an instructor so he gets some idea of what the pilot is up against in the Real World. (Pilots HATE complicated avionics. I didn't really understand why until I flew a sim scenario, and learned firsthand. They have their hands full with the airplane and the mission, and anything that adds workload is THE ENEMY.)

Having your developers write the test procedures is not the answer. Even if you want to argue that the developers are most familiar with the product, and most able to do it, you also need to test the paths that the developers AREN'T familiar with. Example: Certain functions on a certain airplane are required to be done in a very specific order. A certain operational pilot, during a VERY high-profile certification test, deviated from the procedure, and demonstrated a previously-unknown MAJOR issue, that caused the certification test to fail instantly -- and with very good reason. The airplane REALLY should not have done what it did. (I'm being deliberately vague here. I have my reasons.) A good test engineer, having been told "Do it like this", is GOING to deviate, to see what happens if he doesn't "do it like this". That's what test engineers do.

Note: When asked why the pilot deviated from the procedure, the lead answered by asking "Do you know 'pilot'?". Wry smiles all around the room.

  • All good points, thanks, and you definitely get at the crux of the problem. Test scripts alone are always going to be inadequate for this reason. – Ernest Friedman-Hill Mar 3 '15 at 2:31
  • A QA manager I worked with liked to say, riffing on Arkady Renko in "Gorky Park", "QA decides what interests QA". :-) – Ross Patterson Mar 3 '15 at 11:13
4

Your developers are going to have to write very detailed and explicit testing instructions until the testers learn the domain and the app to the point where they can write their own. Make testing training videos from screen recordings just like they would for user training.

You may find some interesting bugs getting discovered because your testers don't think like engineers or programmers.

If this isn't feasible, someone in your firm needs to address the current business testing model and make room for more qualified people.

  • 1
    I like your suggestion about screen recordings. We already do that for training purposes, so it would not be hard to use that same capability to help create testing instructions. – Ernest Friedman-Hill Mar 3 '15 at 2:29
  • Whatever techniques or practices you find work for training users, it will probably work to get your test team up to speed. – JeffO Mar 3 '15 at 3:00
3

Your testers do not need to think like a developer - for black-box testing they need to think like a user of your software, and they should have additional knowledge about how to test software in general. So assumed your testers are smart enough to learn those things in a reasonable amount of time, train testers the same way you train your users, and make them learn testing basics. For the first part, if you are developing a professional product, I guess you already have a standard procedure. For the letter, there are books, courses, and lots of other material available at the web.

And if the usage of your software needs a fully skilled engineer, you will need your testers to have equal skills, at least some basic skills from that field. For example, when you develop a CAD software, you need people trained in technical drawing, if they are users or if they are testers. Probably your testers don't have to be as good as your users in the domain, but some knowledge of the domain will be essential. But maybe you could see this also as a chance to improve your software in a way to lower the entry skill level?

  • Thinking like a user is not enough. In an article about the development of Zork, a very early text adventure game, one of the Four Great Implementors told a tale. One day, he made a modification on the weapons technology module, and made it possible for the player to acquire the axe wielded by a certain Very Bad Dude. (Normally, the axe vanished in a puff of smoke when the VBD croaked.) He told one of their better testers. The tester immediately said "GREAT! I'm gonna go chop down some trees in the forest!" Oops. – John R. Strohm Mar 3 '15 at 7:01
  • 1
    @JohnR.Strohm: having played a lot of those text adventures for myself some decades ago, this tale sounds as if it could have been also an intentional part of the storyline ;-). But I get your point, and you hopefully got mine when I wrote "they should have additional knowledge about how to test software in general". One of the first things you learn in general courses about testing is trying to be destructive, check edge cases, and so on. – Doc Brown Mar 3 '15 at 7:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.