Useabilitosity: Jason Schklar Interview
Jason Schklar and I started talking after I linked to his piece on Left 4 Dead as a proof-of-concept for a DungeonMaster game in a Sunday Papers. We ended up deciding that since what Jason does isn't exactly what gets brought before the general gamers' eye that much, an interview would be interesting. Because after working at Big Huge Games and Microsoft Game studios, he's become a freelance usability and playability specialist. As in, he looks at the absolute fundamentals which makes games easier to get into and harder to put down. That's unusual, thinks I. Let's see what that actually means...
RPS: I suppose the starting question has to be... well, what attracted you to the area in games development? While of useability is totally central, it's not a specialty which you see people fantasising about doing. What was the turn on?
Jason Schklar: The "gee I wish I could work in the games industry" moment for me came in 1999. I picked up both Halflife and Planescape Torment based on review scores. I hadn't played videogames in a few years -- and what a welcome back! Those games changed my life and helped motivate me to find some way to get into the games industry.
The exact moment I knew I was on to something was when a boss of mine at a statistical software company I was working at scoffed at my suggestion that we could apply learnings from game user interfaces to statistical software. Six months later I was a games user-testing specialist at Microsoft Game Studios. Like most people in the games industry, I've loved and played games for most of my life -- though I was never a hard core gamer. I was too busy in highschool, too busy partying in college, and too depressed in grad school.
I've always been a student of human behavior. People never cease to amaze me. I received some more formal training in graduate school where I did research on judge and jury decision making and taught undergrad statistics. I did a lot of observational work as well as survey research and worked with some awesome research scientists. Grad school allowed me to travel to cool places to present my research -- but it also left me broke, burned out, and depressed.
RPS: You're now doing it in a freelance consultancy basis - what made you decide to make the leap there?
Jason Schklar: I've worked for some great companies and with some great people. Microsoft Game Studios, Big Huge Games, Amazon.com. All of them played an important role in my professional development. The problem is that I like to do all kinds of work and get burned out if I'm only doing the same thing over and over again. Running my own business allows me to do all kinds of different things with all sorts of interesting people. Sure, I have to spend more time networking and doing business development than I'd like, but it's worth it. I've met a ton of interesting people and been exposed to some really neat projects.
RPS: Do you think having a more casual background with games helps you questioning that sort of basic useability assumptions? As in, someone who's been native in games so long may not even realise anything was wrong with - say - Dwarf Fortress. You're less likely to do that.
Jason Schklar: That and my terrible memory are probably two of my greatest "strengths" in this regard ;) Actually, I think that the ability to approach each new game as a naive, unsuspecting user would is more governed by my academic training. No assumption is sacred, no question is too embarrassing to ask, and no game is ever perfect. Of course, you can't make games better by just asking people who haven't played many games what they think. Remember that at least for traditional games within established genres, you need to make them appealing to folks who have spent hundreds (if not thousands) of hours playing these games. They have certain expectations that need to be met -- or if not met, then these players need to be allowed to fail gracefully and learn "the new paradigm" in fun ways. Moreover, my background in various aspects of game development (production, design, and usability) mean that I have a lot of experience breaking down user experience problems into their specific game play components. It's easy for someone who hasn't played many games to say "this game sucks". But it might be exceptionally hard for that person to explain why in ways that you could act on.
RPS: At least to my novice eye, Usability is one of those areas of games which is about avoidance of negative impressions. As in, we note its absence rather than its presence. Would you agree with that?
Jason Schklar: You are absolutely correct. There are other weird things about usability when it comes to games, as well. A usable game isn't necessarily fun. And some fun games are not at all usable. The approach of my company -- Initial Experience Consulting -- is to work with game developers to identify the core mechanics and systems first and iterate on them until they are fun. Then we work backwards to figure out how to make these core elements discoverable through play (and not reading or boring tutorials). The idea is to get people into the core fun experience as quickly as possible (like the first few minutes) and keep then interested and wanting more.
RPS: And if so, could you mention some key moments where you've added something to a game you worked on which made you feel... fucking hell! This bloody works. I'm a Genius!
Jason Schklar: Most of my "brilliant" successes are really shared moments with the development team. I run usability in a very communal and participatory manner and try to surround myself with team members who are smarter than me in their various crafts (design, programming, art). My goal is to help the team identify the true causes of problems so that we can think about how to solve them given existing vision, scope, and resources. Or, if the results are dire enough and the problems are fundamental enough, we step back and think about existing vision, scope, and resources.
One of the most interesting challenges we faced as a team was on Catan for Xbox Live Arcade. The game centers around trade -- I offer a sheep and would like to receive a brick in return. It seems like a simple UI problem to solve, but keeping in mind that we wanted to keep text to a minimum and had limited screen real estate (on a standard def tv) it actually took a number of iterations to get it right.
The key to our success was not our "brilliance" -- if we were brilliant we would have designed it correctly the first time. Instead, we adopted a process that allowed us to try out several different solutions until we were able to validate that one of them worked. We did what some folks call RITE usability. Rapid Iterative Testing and Evaluation. Each time a participant failed to figure out how the "offer/receive" UI worked, we would try a new solution and test it with the next participant. Once we had a solution that worked and was validated with multiple participants we knew that the fix was "good enough".
The important thing to realize was that we were iterating on an almost hourly basis. The designers and artists would create fixes in real time as we observed them and we would get them into a build for another test later on that day. This allows for an incredible rate of finding and fixing usability issues -- and a game that is much more polished after only a few intense days of work.
RPS: The death of the manual in gaming. Discuss.
Jason Schklar: One of the speakers on a SxSW panel I was on talked about how user researchers should read FAQs and roll that feedback back into the game. I couldn't agree more. If usability work is done early and often enough there should be very little need for a manual. That said, sometimes issues are identified too late to fix before shipping. And I do believe there is some value in having a manual: Some gamers prefer to do a little reading and research before they play.
Most importantly to me -- and something that I stress when I work with the folks who write manuals -- is that the manual should, itself, be usable. To me this means: Centerfold or back of manual should have a controls layout and/or "quick start" hints; concepts or mechanics that have been found to be confusing or difficult to learn (via usability) should be clearly explained; and there should be a list of advanced tips and tricks that readers can refer to if the game is too challenging for them.
RPS: The weird thing about manuals which I've learned... well, I've heard repeatedly about the manuals being written months in advance due to printing and design, I'll guess, but it's got to be suicidal, yes?
Jason Schklar: Processes are getting better for this. I've got some friends who work on game manuals that I could introduce you to if you're interested. Yes, you need to lock the manuals down before the game ships (especially if you're going to localize and sim-ship world wide). But if core controls, mechanics, and game play needs to change between manual lockdown and going gold, there are probably bigger problems with the game than just having a vague/inaccurate manual.
RPS: Jim recently did an interview with Chris Delay of Introversion. One anecdote stood out - basically, Valve had them over while they were working on Defcon. They sat down a complete random on the sofa and let them play. By the end of it, they were in distraught tears over how wrong they were getting it. Which sounds like the sort of hard lesson in Usability you'd approve of. You'd encourage people going through this, yes?
Jason Schklar: Yep. There's no better lesson than the first time the key creatives on a team get to sit and watch and listen to strangers play their game completely unassisted. It teaches you to think about your game in ways that you assumed you were doing all along. Once people get past the initial defensiveness that is inevitable ("oh, those aren't real gamers" or "oh, only one person got confused, so it's not that bad") they truly start to appreciate the benefits of doing user-testing early and often. Why? Because when they actually fix the issues that are preventing people from enjoying the game, then they get the positive reinforcement of watching people play the game and have fun.
All of the great designers I've worked with crave this kind of feedback. Why? Because the game is still in production and they have a chance to fix it. No one wants to make a game that truly sucks. Fail early, fail often, and fail in front of usability participants who are under NDA as opposed to failing in front of folks who paid $50 to buy your game.
RPS: What's the most common problems you see. What stuff, when you see in a game, is like nails down a blackboard?
Jason Schklar: I work on a lot of games across a number of genres, so there aren't a lot of specific commonalities. The most fundamental learning is that as we move from "developer" to "outside usability consultant" to "actual player" you learn progressively more about how your game either meets or deviates from your game design vision. When a project starts, it is important for developers to be nimble and test out their ideas quickly until they get some features or a prototype working that they are excited about. At some point it becomes useful to bring in and iterate with outside user experience professionals: Someone who understands the developer's vision and who can point out various potential clashes between design intention and user experience reality. This is important so that key user-experience risks can be called out so that they can be implemented and tested (and iterated on) early. Once there is a version of the game that is appropriate for non-professional gamers to play, it's time to run some usability sessions. This means watching people play the game, making fixes ASAP, and testing again. It is at this point where the game starts to become more polished, approachable, and enjoyable.
In terms of "nails down a blackboard", there are probably lots of things. One of the most annoying things that I'll mention is the tendency of some games to try and solve usability issues through pop-up text billboards. To me, this usually signals that the developers left enough time to assess usability issues but did not leave enough time to fix them through game play. Even worse: Games that (seemingly) randomly pop-up text billboards that are not context specific to what the player is currently doing. For example, if I'm trying to pitch in a baseball game and I receive a "hint" pop up that talks about how to steal bases. In the best case, that information will simply be forgotten. Worst case, the player will then think that he's being told to try and steal a base -- which isn't currently possible because he's pitching.
RPS: And there you lose 3/4s of the terribly british RPS with that bat-to-ball metaphor. Thanks for your time, Jason
You can follow Jason Schklar at his blog.