Disclaimer: These views are not the views of Amazon or official Lumberyard policy. Honestly, it’s just a rant.

The value of testing

It is a misconception to think that test are written to determine if you wrote your code properly. Sometimes this is true if you are a true Test Driven Developer (TDD) but in general, we write our test during or after we write the code, and after we know it works.

Sometimes we use test to see if some abstract logic is working, and that is definitely a benefit, but is not the main benefit.

The main benefit of automated test is to ensure that future code doesn’t break past functionality.

In fact, Facebook has developed a snapshotting technology for their Jest testing framework, that can snapshot the state of objects and then compare future test runs to the previous snapshot. This testing method assumes that the test are created after the functionality is verified already. Unfortunately they wrote it for icky icky JavaScript. And so did I for a different testing framework (https://github.com/horvay/jasmine-snapshot don’t judge).

However, from my limited experience, I think automated testing has been underrated for the gaming industry in general. With Lumberyard’s EBus system, testing is super easy.

The difference

Someone once said that integration test will tell you something is wrong and unit test will tell you where it is wrong.

Simply put, unit test try to test one unit at a time. Integration test put together multiple units to see if they interact properly.

Of course, that’s abstract as hell, and doesn’t mean much when it comes to writing test.

For Lumberyard, here are some rules I think apply:

• If you are testing your own code and how it interacts with Lumberyard’s code, in order to see if Lumberyard handles it as expected, then this is an integration test.
• If you are testing your own code, and mock out all of the interactions with Lumberyard (as done here), then it is a unit test.
• If you are combining two radically different sections of your own code, I’d consider this an integration test too. Like how your player object interacts with some custom world entity. It is validating something on both the world entity and the player.

So for instance, if I’m testing one of my components by instantiating it and running some functions on it, then it is probably a Unit test. However, if I’m testing it by creating an entity and adding the component to that entity, and firing off an EBus event to call something on the component… then it’s definitely an integration test.

In general, unit test tend to mock outside connects in order to test the one unit/class/component/etc. In Plato’s world of shadows, all outside connections to the class would be mocked; however, in real life, sometimes you integrate with Lumberyard in order to make writing the test easier. However, the focus in a unit test is testing the internal logic of that class.

Sometimes in integration test, people will pick some connection between two classes/systems/units/etc and mock all other connections. Sometimes they choose to mock nothing. An example is where you may mock everything on gEnv (which is generally old CryEngine stuff) but don’t mock any Lumberyard stuff. So it’s an integration test between your code and Lumberyard.

A last category which should be small, are sanity test or smoke test. These test should be super high level and mock nothing. The goal is to put as much coverage of your code and span across as many layers as possible. However, these test can be painful to debug if you don’t have integration and unit test.

So hopefully the statement, Simply put, unit test try to test one unit at a time. Integration test put together multiple units to see if they interact properly. now makes more sense.

As time goes on, I will put more links to examples from this site.

Test all things

In a world where time doesn’t exist, every line of code you write will be in a test. This is definitely doable, but not very practical.

If I were to give a completely arbitrary percent, I’d say something like 80%-90% should be covered.

Also, this is only for your own code. You should never test Lumberyard’s functionality explicitly. Even though integration test run their code, we are not testing for the accuracy of their code. If we had to test every library we used, we’d die of Carpal Tunnel, if that’s possible.

At google, they have a policy about automated testing. If Person A breaks Person B’s code with some new functionality, it’s actually Person B’s fault, because they did not have a test to catch it. That means Person B has to fix it, not Person A.

I think that’s a good way of looking at testing. Ensuring that your functionality works is on you to make automated test so that future updates can be validated against those test.

Written by Greg Horvay