Tile 16

Effective unit testing in Unity

Preface

This is a post intended for developers who are experienced in writing testable code and studying effects of decoupling Unity during video game development.

This is not a tutorial on how to make the game in Unity, but a journey through how I went about figuring out how to effectively unit test Unity scripts.

You are expected to be able to:

You are welcome to skip to the conclusion if you just want my formula for testability in Unity.

Introduction

One of the biggest problems of using frameworks is that you are almost always forced into coupling your code to the framework making a large chunk of code untestable. Unit testing becomes especially difficult when the framework deals with physical elements, such as GUI, audio, and animations. These physical elements are usually tested manually, or tested through some kind of (awful) GUI macro software. Both of which are terrible ways to test software because of lack of reproducability and consistency.

A lot of people will tell you unit testing in Unity is impossible. Go ahead look it up… seriously go look it up, and if you do find something let me know because I want to read about it! But really at the time of this post there were almost nothing on this topic. There were some primitive insights on the like this article from Unity. Give it read and you'll quickly realize that this is insufficient because they don’t test anything coupled to Unity, like inputs and animation. You also end up with a ton of interfaces for the sole purpose of testing.

Take the example from the SpaceshipMotor in the Unity article, this is what the FixedUpdate() function looked like:

private void FixedUpdate()
{
    if (Input.GetButton("Horizontal")
        MoveHorizontaly(Input.GetAxis("Horizontal"));
    if (Input.GetButton("Vertical")
        MoveHorizontaly(Input.GetAxis("Vertical"));
    if (Input.GetButton("Fire1")
        Fire();
    if (Input.GetButton("Fire2")
        Reload();
}

There is a good chunk of logic in there and is basically all untested. Usually things analogous to the real world, in this case the controls, are not unit tested because you usually don't interact with the real world in unit tests. The key here is to reduce all code that are difficult to test into 1 liners:

private void FixedUpdate() { SpaceshipInputController.UpdateControls(); }

We, the cool developers, don’t test one liners right? Because they are usually trivial and it’s a waste of time to test them.

So what if all your Unity script classes are composed of nothing but one liners? Hold on to that thought, because this is where we are headed.

Tile 16 is puzzle game built to explore unit testability in Unity

All tests, source code, and project files are available for your taking. They are unlicensed and released into the public domain. Feel free to make millions off them. All I ask is for you to let me know whether this was actually helpful or not.

This game is a crude replica of Ten Gen. I found this game on Newgrounds and it was quite fun. It was simple enough and I could do a remake in less than a week in Unity so I thought would be a good candidate for this case study.

To the creator of Ten Gen, thank you. And please don’t sue me for copyright infringement :)

I’m not going to explain the gameplay. You should just play it yourself, understand the game before continuing, and give them some ad revenue. You can also try my version here.

A Bit on the Game Design

Here was the summary I wrote while I reverse engineered the game:

  • The game randomly generate up to 4 numbers each wave
  • Each wave must have numbers that can generate pairs with all existing numbers on board
  • Each wave is advanced when there are no more possible pairs to combine or the timer clocks
  • The player can move tiles around the tile set given there is a path
  • Tiles can be combined to increase the counter on the tile
  • The player wins when a tile reaches 16. Here are some of the more detailed requirements I wrote down for the game:
  • Grid size, wave timer, number of spawns, min/max spawn tile value should be adjustable
  • Winning conditions should be adjustable
  • Clock wave timer that expires after x unit time
  • Tiles with same value can be combined
  • Tiles can be moved anywhere in the grid provided there is a valid path
  • Each wave generates x number new tiles ; the resulting board must have at least a tile pair that could be combined
  • Start and stop game Basic actors for this game are:
    Player – Gameplay– tile logic (movement, combining etc), start and stop the game
    Difficulty Adjuster – Difficulty Adjustment – grid size, wave timer, number of spawns
    Winning Judge – Winning Conditions – value to win, or else? I designed and implemented the game around these requirements and actors. As a result the game would be highly flexible for adjustment and extension.

Wave 1: Don’t use Unity

Defer Unity as long as possible, because Unity is evil…for testing. The first task was simple: implement as much of the game as possible isolated from Unity.

The good news is given that this is a tile game, I could create majority of the gameplay on some kind of 2D array data structure. I ended up with the following architecture. Here’s the high level UML for your convenience (the code is available in the tile16_decoupled folder you can get near the end of the article):

I’m not going to go over the code. It is developed using TDD. Here are some highlights:

Wave 2: Use Unity

My first thought was just to make it work and manually test everything, and in meantime keep the code very clean. My hope was that I could refactor the code at the end and perhaps extract some sort of layer to decoupling Unity.

Here is the UML:

I’ll go over a little bit on the new code:

Here are the highlights:

The intermediate code is available here for your amusement.

Why Unity Scripting is Restrictive

  • MonoBehavior is the worst class. It handles EVERYTHING. Every single game component is available for access by calling GetComponent<ComponentName>. This decouples the components from MonoBehavior internally but externally any class that uses MonoBehavior are essentially coupled to every single Unity component. My TileController class uses: sprite renderer, animator, sound, box collider, text mesh, and rigid body.
  • Unity wants the developer to handle each GameObject independently of one another. This is made clear by the fact that you can only attach scripts to specific game objects rather than some sort of governing agent. You can create hierarchical tree of GameObjects, but that’s really meant for relating to their physical rather than logical children.
  • For example, a sword being a children of a hero GameObject, that works great because the sword will move with the hero
  • Where it doesn’t work: GameController uses BoardUpdater and Tile GameObjects. However GameController is a logical entity and it doesn’t make much sense to create an empty GameObject in Unity just to bind GameController. The only way for GameController to communicate with board and tiles is through internal components (such as a collision), or global variables. This works, but it puts a lot of restrictions on architectural decision of your game.
  • The “classical” OO dependency tree you create for your game architecture simply breaks down under Unity. So you really have to be careful on how your game is designed to minimize architectural friction.
  • My guess is that since Unity went for the GUI driven approach, it was meant for a non-technical audience. But in my personal opinion, software is software. If you are using OO language and not following OO principles then there’s something smelly about your framework. Unity is definitely successful in what it is trying to achieve, but I really question whether or not it is a viable framework for scalable video game projects. In a sense, Unity actually encourages making a mess. Please let me know if you have a better view of the situation as I’m eager to learn.

Wave 3 – Decoupling Unity

This is where I converted untestable to testable code. One of the tasks that I actively drove while I was at AMD working with software close to hardware level was converting legacy code into testable code. It’s not the easiest thing to do with drivers, especially when pretty much everyone writes code like it was the 1980s and doesn’t believe in unit testing.

So where could I start extracting something testable? The simplest class was the BoardUpdater.

BoardUpdater had two responsibilities

  1. It creates the physical tile and sets its position
  2. Convert logical tiles to physical tiles when called (triggered by TileController and GameController)

One obvious abstraction I could make is to separate CreateTile() method to its own class, which would fix the SRP violation.

Let’s take a look at what Unity coupling is involved here:

The first thing came to mind was to abstract the above into its own class called TileCreator. There needs to be some sort of medium in between the TileCreator and the BoardUpdater to help communicate the Unity specific messages. I know, a mediator! If you are not familiar with the mediator pattern, it’s simply a mechanism to decouple two classes by mediating the messages between them rather than having them call each other directly. BoardUpdater no longer creates the tiles but instead goes through TileCreator and TileCreatorMediator: 

void Start ()
{
    tileCreatorMediatorMediator = new TileCreatorBehaviorMediator(GameBoard);
    tileCreator = new TileCreator(tileCreatorMediatorMediator); 
}

public void UpdateBoard()
{
    var tileList = GameBoard.PopNewlyCreatedTiles();
    foreach (var tilePair in tileList)
    {
        tileCreatorMediatorMediator.SetNewTile(Instantiate(tileGO));
        tileCreator.Create(tilePair.Key, tilePair.Value);
    }
}

Look at how much simpler BoardUpdater became! Furthermore the base mediator is an interface which means I can create a mock for the TileCreatorMediator and actually test tile creation!

[Test]
public void TestTileCreatorMediator()
{
    var mockTileCreatorMediator = new MockTileCreatorMediator();
    var tileCreator = new TileCreator(mockTileCreatorMediator);
    Point point = new Point(0,0);
    Tile tile = new Tile(7);
    tileCreator.Create(point, tile);
    Assert.That(mockTileCreatorMediator.IsInstantiateTileCalled, Is.True);
    Assert.That(mockTileCreatorMediator.IsAddToTileControllerDictionaryCalled, Is.True);
    Assert.That(mockTileCreatorMediator.Position, Is.EqualTo(new Vector2(50.0f, 50.0f)));
    Assert.That(mockTileCreatorMediator.ValueText, Is.EqualTo("7"));
    Assert.That(mockTileCreatorMediator.TileColor, Is.EqualTo(new Color32(0, 245, 255, 255)));

}

In the test case I verified the following:

This was good. I could even make the test more aggressive by checking order of calls (i.e. tile must be instantiated first before position can be set). I was well aware of where I could take this in terms of aggressiveness, but…I was lazy.

I proceeded to do the same with TileController. I named the extracted class TileInputController (crappy name I know, it’ll be fixed later). Here was TileInputControllerMediator at the time:

public interface TileInputControllerMediator
{
    //Transform
    Vector2 GetTilePosition();
    Vector2 GetVelocity();
    void SetVelocity(Vector2 velocity);

    //Inputs
    void MouseDown();
    void MoveMouse(Vector2 to);

    //Sound and Animation
    void PlayPickUpAnimation();
    void PlayPickUpSfx();
    void PlayDropSfx();
    void PlayDropAnimation();
    void PlayCombineSfx();

    //Logics
    void PrepareTilesOfSameValue();
    void SnapToPosition();
    Point MoveBoardPoints();
    void UpdateBoard();
    void DestroyCollisionObjects();
    void RestoreColliderSize();
    void SetColliderSize(Vector2 vector2);
}

I saw a pattern here. So I organized them somewhat according to Unity components. There were some methods in here are simply one line wrappers for Unity. It almost looks like facade… This is actually significant and I’ll touch on this later.

Notice the //Logics section. These are abstracted methods that do something more than just assigning Unity component fields. Most of my tests at this time were simply checking whether a certain function was called. The abstractions were actually untested, because they ended up in the concrete mediator still coupled to Unity. It was the same as the SpaceshipMotor example, only by a hair better.

By the time I was somewhat finished with the TileInputControllerMediator, GameControllerMediator was somewhat defined as well. I had a suite of tests that verified majority of TileController, and things like animation, sound, and input were all tested to a certain extent. At this point I’ve written enough tests that I developed an idea on how to further testing the //Logic section. In addition, it would allow me to do TDD by writing tests first.

Wave 4 – Low level Unity decoupling

Mediators coupled to Unity aren’t testable. But what if the mediator implementation was really, really simple? What if they were just one-liners that simply just wrapped Unity components in some way such that they can be tested via my eyeballs? Eye Driven-Development. EDD, I called it.

Take a look at the ShrinkCollidersOfSameValueExceptSelf() method from the wave 2, it is called when a tile is picked up:

private void ShrinkCollidersOfSameValueExceptSelf()
{
    foreach (var tileController in TileControllerDictionary[curTile])
        tileController.boxCollider.size = new Vector2(0.75f,0.75f);
    boxCollider.size = new Vector2(1.0f, 1.0f);
}

PrepareTilesOfSameValue() from the TileInputControllerMediator uses this functions. PrepareTilesOfSameValue() prepares all tiles of same value ready for potential collision. It sets collider of all tiles of same value other than itself to 70% scale. It also sets the rigid body to kinematic so they can pass through each other to provide that combine feel. This was the test-last test case for picking up a tile:

[Test]
public void PickUpTileTest()
{
    tileInputController.PickUp();
    Assert.That(tileInputController.IsPickedUp, Is.True);
    Assert.That(mockMediator.PrepareTileOfSameValueIsCalled, Is.True);
}

This is a crappy test. Sure I checked if tiles were prepared, but was it prepared correctly?

What if I write this test first?

[Test]
public void OnPickupShrinkOtherTilesOfSameValueColliderSmaller()
{
    tileInputController.PickUp();
    Assert.That(mockMediator.IsKinematicUnset, Is.True);
    Assert.That(mockMediator.LastSetColliderSize, Is.EqualTo(new Vector2(1f, 1f)));
    Assert.That(mockTargetMediator.LastSetColliderSize, Is.EqualTo(new Vector2(0.7f, 0.7f)));
}

That was much better! I was actually verifying states. With this test I could no longer keep PrepareTilesOfSameValue() and required a lower level of abstraction. I needed the mediator to be able to directly set collider size as well as kinematics for the rigid body.

Here’s the resulting new TileControllerMediator:

public interface TileControllerMediator
{
    //Transform
    Vector2 GetPosition();
    void SetPosition(Vector2 vector2);
    float GetScaleX();

    //Renderer
    float GetBoundSizeX();
    void SetRendererInactive();
    //Inputs - not apart of the interface, but should be implemented in the subclasses

    //Sound and Animation
    void PlayPickUpAnimation();
    void PlayPickUpSfx();
    void PlayDropSfx();
    void PlayDropAnimation();
    void PlayCombineSfx();
    void PlaySpawnAnimation();

    //Collider
    void SetColliderSize(Vector2 vector2);

    //RigidBody
    Vector2 GetVelocity();
    void SetVelocity(Vector2 velocity);
    void KinematicUnset();
    void KinematicSet();

    //Logics
    void UpdateBoard();
    void DestroyCollisionObjects();
    void DestroySelf();
}

Methods like MouseDown() were also removed because they can be directly triggered by the TileController class. If a callback is required for some reason I could always add it back or use a delegate.

The mediator implementations ended up being one to two liners methods, and EDD was sufficient to verify their correctness. Think about that for a moment. I hope you see where I was going with this.

I started out test-last in wave 3. Little by little, I started doing test-first. I managed to refactor and rewrite the game logic into testable code that tested everything Unity handled, but did not actually touch any Unity components. I ended up catching a few new bugs that weren’t handled, rewrote a few parts completely test first. Also notice interfaces for WaveController and WinCondition were extracted for testability as well.

Eventually I didn’t even need Unity editor. I was only working in Visual Studio and churning out code that was indirectly testing Unity component logic as well! My tests were sufficient to test pretty much every single aspect of the game!

For the sake of completeness, here is the UML for GameController:

Here’s the complete packaging diagram:

Everything in the bottom layer is testable. And all light blue classes are mostly composed of 1-2 liner methods (aside from initialization and object instantiation) and can be verified very quickly through inspection.

The complete project and source code is available from here.

Wave 5: Improving ease of testability

I was fairly satisfied at this point, but I knew there was something I could do to make testability much easier.

Every class that I wrote in Unity would require a corresponding unique mediator. This could result in a lot of very similar looking code. There was also a major ISP violation with using mediators. It still had too many coupled responsibilities.

There was an obvious pattern among the mediators, almost like facades that I mentioned earlier. Perhaps I could extract some kind of common mediator interfaces? Followed by some logically abstract classes? Followed by structuring them into a reusable middle layer that is easy to maintain, write, and tested? I figured it out, and I think you can figure it out as well. This will segregate the interfaces as well as make the mediators reusable. I’m not going to post the code and I’ll leave it as homework for you. But do let me know if you do actually implement something like this and release a “Unity Mediation Testing Framework”, because I want to be credited for that :)

Conclusion

The more I write this post the more I realize how much I dislike Unity’s scripting architecture. It is a bit too architecturally restrictive for my taste. Lacking in testability is a big restriction, especially if you intend to develop scalable games that can one day turn into a high developer count project.

But all’s not lost, if you are a good developer you just have to make some small efforts to make your code testable again. Although your code will still be ultimately coupled to Unity, they can all still be automated in the form of unit tests outside of Unity.

Here are the basic steps you need to write a fully testable script in Unity:

  1. Small design session of your module – understand the requirements and what you are about to implement
  2. Create a blank mediator interface for that module
  3. Write a test
  4. Anytime you reach a call that requires a Unity component, create the appropriate mediation method in the mediator interface
  5. Mock out the mediator appropriately in the test
  6. Write your code, make the test pass
  7. If you realize the method is too board, keep extracting methods out to your testable module until you reach lowest level of abstraction that is closest to Unity
  8. Modify your production code and tests accordingly
  9. Refactor
  10. Repeat 3-9 until module is complete

If you truly understand the domain of your game you can most likely create a reusable mediation testing framework, so you wouldn’t have to write a unique mediator for every module in the game.

This whole process took me about 70 hours. I enjoyed the experience.I hope you enjoyed my post, but more importantly I hope you learned something new. If not, get bent.

Blog Comments powered by Disqus.