Unit Testing a stateful framework such as Phaser?

Question

TL;DR I need help in identifying techniques to simplify automated unit testing when working within a stateful framework.

---

Background:

I'm currently writing a game in TypeScript and the Phaser framework. Phaser describes itself as an HTML5 game framework that tries as little as possible to restrict the structure of your code. This comes with a few trade-offs, namely that there exists a God-object Phaser.Game which lets you access everything: the cache, physics, game states, and more.

This statefulness makes it really hard to test a lot of functionality, such as my Tilemap. Let's see an example:

Here I am testing whether or not my tile layers correctly and I can identify the walls and creatures within my Tilemap:

export class TilemapTest extends tsUnit.TestClass {
    constructor() {
        super();

        this.map = this.mapLoader.load("maze", this.manifest, this.mazeMapDefinition);

        this.parameterizeUnitTest(this.isWall,
            [
                [{ x: 0, y: 0 }, true],
                [{ x: 1, y: 1 }, false],
                [{ x: 1, y: 0 }, true],
                [{ x: 0, y: 1 }, true],
                [{ x: 2, y: 0 }, false],
                [{ x: 1, y: 3 }, false],
                [{ x: 6, y: 3 }, false]
            ]);

        this.parameterizeUnitTest(this.isCreature,
            [
                [{ x: 0, y: 0 }, false],
                [{ x: 2, y: 0 }, false],
                [{ x: 1, y: 3 }, true],
                [{ x: 4, y: 1 }, false],
                [{ x: 8, y: 1 }, true],
                [{ x: 11, y: 2 }, false],
                [{ x: 6, y: 3 }, false]
            ]);

No matter what I do, as soon as I try to create the map, Phaser internally invokes it's cache, which is only populated during runtime.

I cannot invoke this test without loading the entire game.

A complex solution might be to write an Adapter or Proxy that only builds the map when we need to display it on screen. Or I could populate the game myself by manually loading only the assets I need and then using it only for the specific test class or module.

I chose what I feel is a more pragmatic, but foreign solution to this. Between the loading of my game and the actual playing of it, I shimmed a TestState in that runs the test with all assets and cached data already loaded.

This is cool, because I can test all of the functionality I want, but also uncool, because this is technical an integration test and one wonders whether I couldn't just look at the screen and see if the enemies are displayed. Actually, no, they might have been misidentified as an Item (happened once already) or- later in the tests- they might not have been given events tied to their death.

My question - Is shimming in a test-state like this common? Are there better approaches, especially in the JavaScript environment, that I'm not aware of?

---

Another Example:

Okay, here's a more concrete example to help explain what's happening:

export class Tilemap extends Phaser.Tilemap {
    // layers is already defined in Phaser.Tilemap, so we use tilemapLayers instead.
    private tilemapLayers: TilemapLayers = {};

    // A TileMap can have any number of layers, but
    // we're only concerned about the existence of two.
    // The collidables layer has the information about where
    // a Player or Enemy can move to, and where he cannot.
    private CollidablesLayer = "Collidables";
    // Triggers are map events, anything from loading
    // an item, enemy, or object, to triggers that are activated
    // when the player moves toward it.
    private TriggersLayer    = "Triggers";

    private items: Array<Phaser.Sprite> = [];
    private creatures: Array<Phaser.Sprite> = [];
    private interactables: Array<ActivatableObject> = [];
    private triggers: Array<Trigger> = [];

    constructor(json: TilemapData) {
        // First
        super(json.game, json.key);

        // Second
        json.tilesets.forEach((tileset) => this.addTilesetImage(tileset.name, tileset.key), this);
        json.tileLayers.forEach((layer) => {
            this.tilemapLayers[layer.name] = this.createLayer(layer.name);
        }, this);

        // Third
        this.identifyTriggers();

        this.tilemapLayers[this.CollidablesLayer].resizeWorld();
        this.setCollisionBetween(1, 2, true, this.CollidablesLayer);
    }

I construct my Tilemap from three parts:

• The map's key
• The manifest detailing all assets (tilesheets and spritesheets) required by the map
• A mapDefinition that describes the tilemap's structure and layers.

First, I must call super to construct the Tilemap within Phaser. This is the part that invokes all those calls to cache as it tries to look up the actual assets and not just the keys defined in the manifest.

Second, I associate the tilesheets and tile layers with the Tilemap. It can now render the map.

Third, I iterate through my layers and find any special objects that I want to extrude from the map: Creatures, Items, Interactables and so forth. I create and store these objects for later use.

I currently still have a relatively simple API that lets me find, remove, update these entities:

    wallAt(at: TileCoordinates) {
        var tile = this.getTile(at.x, at.y, this.CollidablesLayer);
        return tile && tile.index != 0;
    }

    itemAt(at: TileCoordinates) {
        return _.find(this.items, (item: Phaser.Sprite) => _.isEqual(this.toTileCoordinates(item), at));
    }

    interactableAt(at: TileCoordinates) {
        return _.find(this.interactables, (object: ActivatableObject) => _.isEqual(this.toTileCoordinates(object), at));
    }

    creatureAt(at: TileCoordinates) {
        return _.find(this.creatures, (creature: Phaser.Sprite) => _.isEqual(this.toTileCoordinates(creature), at));
    }

    triggerAt(at: TileCoordinates) {
        return _.find(this.triggers, (trigger: Trigger) => _.isEqual(this.toTileCoordinates(trigger), at));
    }

    getTrigger(name: string) {
        return _.find(this.triggers, { name: name });
    }

It's this functionality I want to check. If I don't add the Tile Layers or Tilesets, the map wouldn't render, but I might be able to test it. However, even calling super(...) invokes context-specific or stateful logic that I cannot isolate in my tests.

Answer 1

Not knowing Phaser or Typescipt, I still try to give you an answer, because the problems you are facing are problems that are also visible with a lot of other frameworks. The problem is that components are to tightly coupled (everything points to the God object, and the God object owns everything...). This is something that was unlikely to happen if the creators of the framework created unit-tests themselves.

Basically you have four options:

1. Stop unit-testing.
   This options should not be chosen, unless all other options fail.
2. Choose another framework or write your own.
   Choosing another framework that is using unit-testing and has lose coupling, will make life so much easier. But perhaps there is none that you like and therefore you are stuck with the framework you have now. Writing your own can take a lot of time.
3. Contribute to the framework and make it test friendly.
   Probably the easiest to do, but it really depends on how much time you have and how willing the creators of the framework are to accept pull requests.
4. Wrap the framework.
   This option is probably the best option to get started with unit-testing. Wrap certain objects that you really need in the unit-tests and create fake objects for the rest.

Answer 2

Like David, I'm not familiar with Phaser or Typescript, but I recognize your concerns as being common to unit testing with frameworks and libraries.

The short answer is yes, shimming is the correct and common way to handle this with unit testing. I think the disconnect is understanding the difference between isolated unit testing and functional testing.

Unit testing proves that small sections of your code produce correct results. The goal of a unit test does not include testing 3rd party code. The assumption is that code is already tested to work as expected by the 3rd party. When writing a unit test for code that relies on a framework, it's common to shim certain dependencies to prepare what looks like a particular state to the code, or to shim the framework / library entirely. A simple example is session management for a website: maybe the shim always returns valid, consistent state instead of reading from storage. Another common example is shimming data in memory and bypassing any library that would query a database, because the goal isn't to test the database or the library that you're using to connect to it, just that your code processes data correctly.

But good unit testing does not mean the end user will see exactly what you expect. Functional testing takes more of a high level view that a whole feature is working, frameworks and all. Back to the example of a simple website, a functional test might make a web request to your code and check the response for valid results. It spans across all of the code that's required to produce results. The test is for functionality more than for specific code correctness.

So I think you're on the right track with unit testing. To add functional testing of the whole system I would create separate tests that invoke the Phaser runtime and check the results.