Assessing frustration in Cosmic Express playtests

While out at GDC I spent a bit of time chatting to Alan Hazelden about Cosmic Express [official site], his train-themed logic puzzler. I was playing the preview build of the game at the time and was curious to know a bit more about how you go about sorting the levels for a difficulty curve and how you interpret good and bad kinds of frustration when playtesting a puzzle game.

It was an interesting chat, punctuated by the arrival of a little dog partway through, so I'm going to just pull out a few of the observations which inform the difficulty curve discussion. It's nice to have it as an accompaniment to yesterday's the review.

The game itself is about laying a train track across the surface of a grid so that it stretches from the entrance to the exit in an unbroken path without overlapping itself and while delivering little coloured blobby passengers to their correspondingly coloured destinations. Restricted passenger space, awkwardly-placed obstacles and a whole host of other limiting factors mean that while some levels yield their solutions more or less instantly, others involve a lot of swearing and vowing to give up on gaming forever (after one more attempt).

The broad approach as Hazelden explains it to me involves a lot of playtesting. I'd assumed it would, but in that hazy "it is probably a playtesting thing" without really delving into the logistics of that. You make a level, you aren't sure how hard it is in the scheme of things because you're the developer and not the average player, and so you playtest to see whether the player has the skills they need to solve the problem or whether you need to add some more teaching moments in the previous sections to make for an easier uptick in complexity.

"I mostly group them around a theme," says Hazelden. "Each constellation has a different mechanic to it – or most of them anyway. Then within a constellation I'll arrange them roughly by difficulty. It's hard to get that precise but that's why you've got branching levels so even though one [level] is slightly off there will be something you can work on."

So when you're looking at difficulty via playtesting and you know that someone is getting frustrated, is that always bad? I know that when I play puzzle games a frustrating level can be the worst thing AND the best thing because of the hours of banging your head against the idea and the rush of glee when you do finally see the solution.

"If people are getting frustrated that might be a good thing or a bad thing depending on why they're getting frustrated. Are they just not getting the puzzle even though they have all the information or is there something the game is doing a poor job of teaching?" says Hazelden. "In [the latter] case it's my fault and I need to fix it. If it's that it's a hard puzzle it might be that it shouldn't be level two of this level pack."

He adds that often players will confound his expectations. This is a recurring theme when I talk to any dev who is involved on the playtesting side of things. When attending the Dishonored 2 AI talk at GDC I remember them talking about the way you have to account for extremes of behaviour because players can and will do things you never thought of.

"What happens in that situation if I stop time, while swapping position with my clone in front of a teleporting witch and then possess a bloodfly that's carrying a stunmine?" was Arkane's example.

Hazelden's experience involves fewer teleporting witches and no discernible bloodfly chicanery but there's still that element of people not doing something you specifically thought they would.

"The thing that I like to think I'd be good at but I'm not always is predicting what the things I'm teaching are going to be. Like, I've got this level and it's going to teach this thing, and that level will teach that thing and this level will combine the two and everyone will get it.

"Then you watch people playing and you think, Ohhhhhhh, this doesn't do that at all…!"

One of the notable sticking points is the set of levels which have a kind of mystery box element. Up until then you've learned that you are guiding a coloured blob to a box of the same colour so it can unpack its possessions and make a little home on that tile. The mystery boxes make it so that any coloured blob can set up a home there. I find those levels relatively straightforward in some respects because you look at the boxes which require a specific colour blob and then you know that any remaining blobs go in the mystery boxes.

It's a bit less comfy in terms of mental bandwidth because you need to keep that information in your head rather than having it explicitly on the screen. It also generally means there are a few extra permutations of blob and box to consider so it can be a bit tricky to hold those ideas in your noggin simultaneously.

"I think the concept is fine – it's fairly intuitive to pick up – but the levels in that pack tended to be quite hard," says Hazelden, "Not because the mechanic was hard to teach but because the levels that mechanic lent itself to were just hard."

And then a dog arrived so that is all my news on Cosmic Express and difficulty curves!