Twas the first usability test of the season...

At Volition, usability tests were designed to be an impersonal affair. Usability tests were, by design, an impersonal affair. The person running the test did so from a different room, communicating with the testers through a speaker, and observing them via cameras.

The reason for this elaborate setup is that usability tests are supposed to measure what a normal player would do, and it’s hard for developers to keep a poker face while testers are bumbling through their game and “playing it all wrong”. Even if you can fight the urge to explicitly tell the player what he’s doing wrong, you still send signals when you take notes, when you sigh, and when you invariably facepalm.

(Figure 1: Subtle body language from a developer indicating you’re not playing the game as he imagined.)

So when we had a recent usability test with family and friends to get some early feedback on combat, I did my best to stoically observe them.

… for about thirty seconds. Then I began to grimace and cringe as they got lost in the first combat menu, and after one tester died for the third time because he was neglecting to actually attack enemies, I gave up and started helping them through the level.

Working on a game every day, you build up a lot of intuition for how things “should” work. You know how to use all the features because you implemented every detail of their functionality. And while you doubtlessly try to build in hints to guide the player along, you’ll probably dismiss something as “obvious”, only to have players puzzle over it as if it were the Riemann hypothesis.

So as it happened, the most important data we ended up getting from our “combat” usability test was how confusing our interface was.

Here were some of the key take home points:
-The player wasn’t informed he needed to move all his units before being able to select “end turn”
-While the player is required to move all his units before selecting “end turn”, the game doesn’t remind the player to select an attack or ability for them, so it’s easy to simply forget to issue those orders
-The first level had a ruthless difficulty and would become impossible after only 1 or 2 mistakes. Hardly appropriate for someone just learning the game.

There’s more, of course, menus with vague wording and poorly laid out buttons. The good news is that none of these features are particularly hard to fix, and identifying issues like this early means we’ll have more than enough time to iterate on them until we get them right.