Testing strategy for games

Question

I've inherited a web-based educational game. Over the past year I've worked towards stabilizing the code and adding new features. Most of the logic is in the front-end, so back-end unit tests, while helpful, cover a small percent of the code.

The game has gotten to the point where it's starting to get complex. There are two different modes for each game and the game behaves differently depending on the mode. There are also various flags that affect game play.

I've been an application developer for 10+ years now and this perplexes me. In the enterprise world, an algorithm always functions the same way. I'll write a unit test for an algorithm, I'll expect the value 42 and it'll error if I don't get that value.

When it comes to games, I'm lost. How do I test them? I have testers available to me. I can spend time writing unit tests.

Testers are ... unreliable. They aren't the best at rooting out issues and I haven't given them the best direction. Short of spending a ton of time every release cycle testing every permutation and combination of the game, how should I use them as a resource?

Unit tests seem limited. Since most of the logic is javascript (and I inherited spaghetti code), I can use a front-end suite like Cucumber or selenium to ensure certain features are working.

Is that the best strategy? How do game companies test games?

I have read the question "Test Driven Development for Complex Games" (among others on the site), but it doesn't address what I'm looking for. I'm asking for strategies, not specific examples of how to test.

Answer 1

In the enterprise world, an algorithm always functions the same way. I'll write a unit test for an algorithm, I'll expect the value 42 and it'll error if I don't get that value.

This is not very different in games. The presence of two modes and multiple flags in the game you're working on doesn't change anything: if you take a specific mode with a specific set of flags, you will still get the same value again and again when testing a part of the source code.

With too many modes and flags, the game can quickly become very difficult to test, because of the number of possible variants. In order to mitigate the risk of running into this difficulty early, you should:

• Mock/stub severely. Keep the tested parts small, and mock everything they rely on.

  If they rely on time, mock the time object to always give a specific result or a set of specific results, independently of the actual time.

  If they rely on random(), mock it to provide a constant every time.

• Refactor. If tests start to be overly-complicated or if you see that a class, in order to be tested, requires twelve arguments after you implemented Dependency Injection, while it required only two before, you need to refactor the code.

• Question the actual business rules of the application. I stopped counting the number of projects which were nearly impossible to test because of the number of different variations required by the functional requirements nobody needed nor understood—not even the stakeholders.

  When a small e-commerce website needs ten pages to explain the functional requirements used to determine how shipment prices are computed, one shouldn't start by writing tests or code, but instead, should get back to the stakeholders and work with them to produce sane requirements.

Finally, automate every test. Testers are needed to discover situations where the application fails. Using testers to execute, again and again, the same action at every revision is counter-productive and disrespectful of testers: regression testing should be done by machines, not people. People, on the other hand, are much better at finding possible bugs. "What if I..." is one of the techniques they can use ("What if I enter a few megabytes of binary data containing several \x00 into a field which asks for my age, and see what will happen?"), but more formal approaches can be used as well.