Useabilitosity: Jason Schklar Interview

Photoshop all my own work. No, I'm not interested in doing freelance work. I do it purely for the art.
Jason Schklar and I started talking after I linked to his piece on Left 4 Dead as a proof-of-concept for a DungeonMaster game in a Sunday Papers. We ended up deciding that since what Jason does isn’t exactly what gets brought before the general gamers’ eye that much, an interview would be interesting. Because after working at Big Huge Games and Microsoft Game studios, he’s become a freelance usability and playability specialist. As in, he looks at the absolute fundamentals which makes games easier to get into and harder to put down. That’s unusual, thinks I. Let’s see what that actually means…

RPS: I suppose the starting question has to be… well, what attracted you to the area in games development? While of useability is totally central, it’s not a specialty which you see people fantasising about doing. What was the turn on?

Jason Schklar: The “gee I wish I could work in the games industry” moment for me came in 1999. I picked up both Halflife and Planescape Torment based on review scores. I hadn’t played videogames in a few years — and what a welcome back! Those games changed my life and helped motivate me to find some way to get into the games industry.

The exact moment I knew I was on to something was when a boss of mine at a statistical software company I was working at scoffed at my suggestion that we could apply learnings from game user interfaces to statistical software. Six months later I was a games user-testing specialist at Microsoft Game Studios. Like most people in the games industry, I’ve loved and played games for most of my life — though I was never a hard core gamer. I was too busy in highschool, too busy partying in college, and too depressed in grad school.

I’ve always been a student of human behavior. People never cease to amaze me. I received some more formal training in graduate school where I did research on judge and jury decision making and taught undergrad statistics. I did a lot of observational work as well as survey research and worked with some awesome research scientists. Grad school allowed me to travel to cool places to present my research — but it also left me broke, burned out, and depressed.

Rise of Legends still looks pretty nifty, doesn't it? I should play it again.

RPS: You’re now doing it in a freelance consultancy basis – what made you decide to make the leap there?

Jason Schklar: I’ve worked for some great companies and with some great people. Microsoft Game Studios, Big Huge Games, Amazon.com. All of them played an important role in my professional development. The problem is that I like to do all kinds of work and get burned out if I’m only doing the same thing over and over again. Running my own business allows me to do all kinds of different things with all sorts of interesting people. Sure, I have to spend more time networking and doing business development than I’d like, but it’s worth it. I’ve met a ton of interesting people and been exposed to some really neat projects.

RPS: Do you think having a more casual background with games helps you questioning that sort of basic useability assumptions? As in, someone who’s been native in games so long may not even realise anything was wrong with – say – Dwarf Fortress. You’re less likely to do that.

Jason Schklar: That and my terrible memory are probably two of my greatest “strengths” in this regard ;) Actually, I think that the ability to approach each new game as a naive, unsuspecting user would is more governed by my academic training. No assumption is sacred, no question is too embarrassing to ask, and no game is ever perfect. Of course, you can’t make games better by just asking people who haven’t played many games what they think. Remember that at least for traditional games within established genres, you need to make them appealing to folks who have spent hundreds (if not thousands) of hours playing these games. They have certain expectations that need to be met — or if not met, then these players need to be allowed to fail gracefully and learn “the new paradigm” in fun ways. Moreover, my background in various aspects of game development (production, design, and usability) mean that I have a lot of experience breaking down user experience problems into their specific game play components. It’s easy for someone who hasn’t played many games to say “this game sucks”. But it might be exceptionally hard for that person to explain why in ways that you could act on.

RPS: At least to my novice eye, Usability is one of those areas of games which is about avoidance of negative impressions. As in, we note its absence rather than its presence. Would you agree with that?

Jason Schklar: You are absolutely correct. There are other weird things about usability when it comes to games, as well. A usable game isn’t necessarily fun. And some fun games are not at all usable. The approach of my company — Initial Experience Consulting — is to work with game developers to identify the core mechanics and systems first and iterate on them until they are fun. Then we work backwards to figure out how to make these core elements discoverable through play (and not reading or boring tutorials). The idea is to get people into the core fun experience as quickly as possible (like the first few minutes) and keep then interested and wanting more.

If Alec is looking at this screenshot, he's probably stroking the screen, purring, trying to reach the little pieces.

RPS: And if so, could you mention some key moments where you’ve added something to a game you worked on which made you feel… fucking hell! This bloody works. I’m a Genius!

Jason Schklar: Most of my “brilliant” successes are really shared moments with the development team. I run usability in a very communal and participatory manner and try to surround myself with team members who are smarter than me in their various crafts (design, programming, art). My goal is to help the team identify the true causes of problems so that we can think about how to solve them given existing vision, scope, and resources. Or, if the results are dire enough and the problems are fundamental enough, we step back and think about existing vision, scope, and resources.

One of the most interesting challenges we faced as a team was on Catan for Xbox Live Arcade. The game centers around trade — I offer a sheep and would like to receive a brick in return. It seems like a simple UI problem to solve, but keeping in mind that we wanted to keep text to a minimum and had limited screen real estate (on a standard def tv) it actually took a number of iterations to get it right.

The key to our success was not our “brilliance” — if we were brilliant we would have designed it correctly the first time. Instead, we adopted a process that allowed us to try out several different solutions until we were able to validate that one of them worked. We did what some folks call RITE usability. Rapid Iterative Testing and Evaluation. Each time a participant failed to figure out how the “offer/receive” UI worked, we would try a new solution and test it with the next participant. Once we had a solution that worked and was validated with multiple participants we knew that the fix was “good enough”.

The important thing to realize was that we were iterating on an almost hourly basis. The designers and artists would create fixes in real time as we observed them and we would get them into a build for another test later on that day. This allows for an incredible rate of finding and fixing usability issues — and a game that is much more polished after only a few intense days of work.

RPS: The death of the manual in gaming. Discuss.

Jason Schklar: One of the speakers on a SxSW panel I was on talked about how user researchers should read FAQs and roll that feedback back into the game. I couldn’t agree more. If usability work is done early and often enough there should be very little need for a manual. That said, sometimes issues are identified too late to fix before shipping. And I do believe there is some value in having a manual: Some gamers prefer to do a little reading and research before they play.

Most importantly to me — and something that I stress when I work with the folks who write manuals — is that the manual should, itself, be usable. To me this means: Centerfold or back of manual should have a controls layout and/or “quick start” hints; concepts or mechanics that have been found to be confusing or difficult to learn (via usability) should be clearly explained; and there should be a list of advanced tips and tricks that readers can refer to if the game is too challenging for them.

RPS: The weird thing about manuals which I’ve learned… well, I’ve heard repeatedly about the manuals being written months in advance due to printing and design, I’ll guess, but it’s got to be suicidal, yes?

Jason Schklar: Processes are getting better for this. I’ve got some friends who work on game manuals that I could introduce you to if you’re interested. Yes, you need to lock the manuals down before the game ships (especially if you’re going to localize and sim-ship world wide). But if core controls, mechanics, and game play needs to change between manual lockdown and going gold, there are probably bigger problems with the game than just having a vague/inaccurate manual.

Jade Empire. The Bioware one no-one seems to have played. Man!

RPS: Jim recently did an interview with Chris Delay of Introversion. One anecdote stood out – basically, Valve had them over while they were working on Defcon. They sat down a complete random on the sofa and let them play. By the end of it, they were in distraught tears over how wrong they were getting it. Which sounds like the sort of hard lesson in Usability you’d approve of. You’d encourage people going through this, yes?

Jason Schklar: Yep. There’s no better lesson than the first time the key creatives on a team get to sit and watch and listen to strangers play their game completely unassisted. It teaches you to think about your game in ways that you assumed you were doing all along. Once people get past the initial defensiveness that is inevitable (“oh, those aren’t real gamers” or “oh, only one person got confused, so it’s not that bad”) they truly start to appreciate the benefits of doing user-testing early and often. Why? Because when they actually fix the issues that are preventing people from enjoying the game, then they get the positive reinforcement of watching people play the game and have fun.

All of the great designers I’ve worked with crave this kind of feedback. Why? Because the game is still in production and they have a chance to fix it. No one wants to make a game that truly sucks. Fail early, fail often, and fail in front of usability participants who are under NDA as opposed to failing in front of folks who paid $50 to buy your game.

RPS: What’s the most common problems you see. What stuff, when you see in a game, is like nails down a blackboard?

Jason Schklar: I work on a lot of games across a number of genres, so there aren’t a lot of specific commonalities. The most fundamental learning is that as we move from “developer” to “outside usability consultant” to “actual player” you learn progressively more about how your game either meets or deviates from your game design vision. When a project starts, it is important for developers to be nimble and test out their ideas quickly until they get some features or a prototype working that they are excited about. At some point it becomes useful to bring in and iterate with outside user experience professionals: Someone who understands the developer’s vision and who can point out various potential clashes between design intention and user experience reality. This is important so that key user-experience risks can be called out so that they can be implemented and tested (and iterated on) early. Once there is a version of the game that is appropriate for non-professional gamers to play, it’s time to run some usability sessions. This means watching people play the game, making fixes ASAP, and testing again. It is at this point where the game starts to become more polished, approachable, and enjoyable.

In terms of “nails down a blackboard”, there are probably lots of things. One of the most annoying things that I’ll mention is the tendency of some games to try and solve usability issues through pop-up text billboards. To me, this usually signals that the developers left enough time to assess usability issues but did not leave enough time to fix them through game play. Even worse: Games that (seemingly) randomly pop-up text billboards that are not context specific to what the player is currently doing. For example, if I’m trying to pitch in a baseball game and I receive a “hint” pop up that talks about how to steal bases. In the best case, that information will simply be forgotten. Worst case, the player will then think that he’s being told to try and steal a base — which isn’t currently possible because he’s pitching.

RPS: And there you lose 3/4s of the terribly british RPS with that bat-to-ball metaphor. Thanks for your time, Jason

You can follow Jason Schklar at his blog.

41 Comments

  1. Dr_demento says:

    I like the ironicaly-rude ending.

    More studios should emply people like this! For instance, Lionhead should replace the guy who designs their menus with him. Then shoot the old guy. You should not need to open three menus and scroll past all your food in order to take a standard health potion during a fight because the supposedly helpful auto-potion-selector thinks what your slight injury needs is that resurrection potion you’ve been saving.

  2. Ian says:

    Hopefully Walker (RPS’ base-to-ball (nope, still doesn’t work) correspondent) can come in and tell us crumpet-munchers what that eccentric fellow was talking about.

  3. Feet says:

    Mmmm.

    Crumpet.

  4. YourFriendlyPedant says:

    Second answer, last line.

    some really need projects.

  5. Concerned Reader says:

    I didn’t understand any of that. Do it again using simple language and remove the references that I’m not familiar with and replace them with references to Star Wars. Thanks.

  6. Kieron Gillen says:

    Pedant: Fixed, you pedant.

    KG

  7. Larington says:

    Nice article, thanks.

  8. TwistyMcNoggins says:

    That was quite interesting actually.

    Take him up on his offer to get in touch with the Manual guys! Would be quite nice to get a view on another side of the games industry we usually take for granted.

  9. H says:

    I’m especially interested in hearing/reading more about the manuals; as someone who learned how to write for users, ie information simplification, it’s almost bread and butter. I’m fascinated by that shit.

  10. The Fanciest of Pants says:

    Very cool. Cheers.

  11. The Hammer says:

    Nice insight to that kinda testing, there.

    Manuals should be fun to read too, ala the GTA games.

    The original Gran Turismo had a pretty grand manual, as well. Chock to the brim with all this information about car specifications and the correct way to drive in very technical language.

  12. DMJ says:

    Manuals are part of the game “experience”. I heartily agree that most manuals are badly designed and clumsy and sometimes cryptic. Some are simply wrong. But occasionally a manual gets it right, and it can be a joy to read.

    I’ve kept the manual for Independence War long after the disc has been scratched to oblivion and the game itself only runs on an operating system I no longer use, simply because I find it attractive and the content is interesting.

  13. futage says:

    “While of useability is totally central”

    Just saying, like.

  14. futage says:

    @Hammer

    The original Gran Turismo manual was great, you’re right.

    I also loved the Civ II manual.

  15. Gap Gen says:

    I rarely read manuals if it’s just a list of instructions for how to start a multiplayer match (unless that’s really unintuitive). Some manuals had neat little extras, though. I do understand the need for limiting the amount a player has to read outside the game to a minimum, though. This includes backstory – it annoys me when a developer creates a whole universe of trashy fantasy/sci-fi and then expects me to understand it all in order to follow the story.

  16. Richard Beer says:

    That was a really interesting read, thanks Kieron. I spend a lot of time banging my head against the Usability wall in my industry (digital comms.) as it can be hard to justify as an expense when the results are so difficult to measure.

  17. Dreamhacker says:

    Good article, but does anyone have a clue to how his name is pronounced? Shlar? Czekler? Shar?

  18. Schmung says:

    This is great. More slightly leftfield stuff like this please.

    I did a ton of usability type stuff at Uni and it’s an interesting field. Nice to see that games companies are starting to pick it up as well, though as Richard Beer says it’s a hard thing to justify all that testing and iteration as a cost because the results are intangible.

  19. Rudolfo says:

    I love how people make a living out of offering common sense to others. Go you !

  20. clovus says:

    I think the rise of online distribution is making manuals obsolete. At best you get a PDF file. Bleh. Also, Americans cannot read and we are a huge market.

    I used to really love manuals. I would open up a game box and actually feel a rush of joy if the manual was bound and not just stapled. I would read the entire manual and then be annoyed at any in-game tutorial wasting my time.

    A big change for me though was the civilopedia. Why bother with the paper manual for Civ IV when there is a quicker and more accurate option in the game. Well, I guess you can’t read the civilopedia during a bathroom break (no laptop). I’d given up on manuals for action games years before that. Now I get annoyed not having a slyly integrated tutorial built into the game, and really annoyed if I actually have to touch a physical manual or google a FAQ.

    It was opening day in America yesterday for “bat-to-ball”, so maybe I can help with the metaphor. When “pitching” you attempt to throw a ball at the opposing players head. The opposing players are attempting to steal bases. You get a “run” when you completely remove a base from the field. If you attempt to steal a base when you should be pitching, you are “out”. When a player is concussed from balls to the head they are “knocked out”. If an umpire feels bad for someone taking too many balls to the head then they are TKO’d. Bats are used to protect one’s head, of course.

  21. Otacon9001 says:

    @Rudolfo: Sure, it is common sense, but you’d be suprised how developers of any software system get usuabilty wrong. I design medical software for my company as well as work with outsourcers. And I’m constantly tweaking the UI to make it easier to learn. The basic design principle is if some needs to read a manual to use your software, you’ve failed. The problem is, that developers work so much on their software, and know it so well, it’s hard to know where a user could need help.

    Left 4 dead is a great example of in game teaching. Team fortress 2 was a bit harder. I remember reading those “tip” thingys on loading screen and everytime I died to figure out what the heck I’m meant to do.

  22. mrrobsa says:

    @ Otacon9001:

    Re: Team Fortress 2. I remember I spent my first few hours as Spy running around, oblivious to his cloak ability. I never even considered hitting right mouse button. :S

  23. Gap Gen says:

    I hate the phrase “common sense”. It implies that the correct course of action is always the immediately obvious one, which is why only idiots bemoan the loss of “common sense”. “Gut instinct” is a much better word for what common sense describes.

  24. Dinger says:

    Pedant time: Common Sense refers to the faculty of the sensitive soul that unites the five particular senses to produce a picture of an object. So when I see, touch, smell and hear a (barking) dog, for example, the particular senses bring that data back to the common sense, which unites them into one being.
    “Common Sense” as a notion of an immediately obvious solution derives from this original meaning, as common sense doesn’t require reason. So, by hyperbole, an “obvious answer” is common sense (and not “Gut Instinct” which is an inexplicable and irrational tendency to the right answer).

    Yet, as the Philosopher tells us, the things most known in themselves are those least known to us. It’s the basic interface problem of reality, and one that occurs in simulated reality as well. Building the bridge from the simulation to the user, so that the interface is evident (known in itself, intuitive, “common sensical”) is hard stuff.

  25. Markoff Chaney says:

    Excellent read. Thank You! I also pine for my lost manuals… Something about making them inherent to the whole “I’m taking a dump or installing the game but I still want to be in the game world” that’s been missing since they’ve been electromagnetically pdfed.

  26. PsyW says:

    I was playing Sniper Elite through for the first time recently, and that irritated me with its random tutorial popups.

    I remember that I was trying to outmanoeuvre a tank, get out of its line of sight before it blasted me. A non-contextual popup appeared – it paused the game, but I was in fight-or-flight mode and instantly dismissed it. I realised shortly afterwards that it was telling me something pretty important, like how my camouflage meter or scope worked and there was no way of reviewing that information after it popped up. Even when I read the stuff, I wouldn’t remember most of it because I was in the middle of doing something unrelated.

    It didn’t really occur to me until I read the article what a horrible design principle it was. Pretty interesting, thanks.

  27. Dave says:

    I’ve been playing Silverfall Complete lately, and though the UI is about 100x improved vs. the original Silverfall release, it has its share of annoyances.

    Lack of granularity in controlling the camera’s pitch is one. In over-the-shoulder games I’m used to being able to drag the mouse to change the camera angle, but here it’s keyboard-only and you have a choice between “too high” and “too low”. There’s just too much camera fiddling.

    I only realized after playing it for a week that the “stances” only activate if you drag them to a slot, replacing one of the standard attack types. Suddenly my goblin rifleman is killing the lesser enemies in one or two shots, and I realize I’ve wasted several levels worth of skill points on my lycanthropic troll puglist. There were too few interface clues on this.

    When you want to enchant an item you have to un-demonize it first — but a popup simply tells you this, forcing you to close one interface, open another, close that one and reopen the first one, switch inventory tabs two or three times, then re-demonize the item and close another window. It could have, you know, just worked.

    It’d be nice to show DPS on weapons rather than making me break out the calculator when trying to decide if I want to make it lighter or heavier.

    This isn’t even going into gameplay issues re: unfriendly mazelike (without looking like a maze on the map) level design and scarcity of teleport points… I hope those improved in the expansion along with the UI.

  28. Gap Gen says:

    I did some 3D engine design for a company as part of a summer project a few years ago. My boss was actually the best UI tester I’ve ever seen, because he comes up with such insane ways to do things. There are bugs you never found while testing because you didn’t even think to try what he thinks is the obvious way to do something. Even his computer had his way of thinking, and crashed the code every time you cast a value close to zero to zero for some reason.

  29. Adam T says:

    Alas, I didn’t see the Sunday link to JS’s blog – I don’t read much intarweb on the weekends. And the blog requires login to comment. So I comment on his blog post here. Oh well.

    Dude, some day someone is going to catch on to My Big Idea. Schklar is getting close. If we Meta back from his player / DM concept you can view the play universe as separate from the game engine. Then you can have the L4D player engine setup, and a separate L4D undead engine setup taking place in the same game world. This is pretty cool, but there are a bunch of unnecessary similarities in L4D between playing a zombie and playing a survivor. Why not make the undead half of the game a RTS style game? Well, b/c it means you have to develop 2 games instead of one. So why not sell it as 2 games?

    I was all set for City of Heros to be totally awsome w/ City of Villains. CoV could have taken a completely different tack than CoH. By the time it came out it was clear that it would just be a re-skin of CoH. Damnit!

    Demand waits for a game that sets a DM who works with players like a RTS general who directs FPS soldiers. Keep your base together or your re-spawn time will grow! Tack together a ratings system and then you can have players who have a good rating as soldier or as general. Then all sorts of interesting things can start to happen in an xbox-live type environment. Griefer control would be non-trivial.

    From FPS-FPS interaction you can back out to RTS-FPS. Can you mix other genre’s? Sure, as long as time flow is the same. And one needn’t limit oneself to only 2 roles / engines.

    One day someone is going to execute on this, and it’ll be cool. I’ll have to buy it when it happens, ’cause I’m not in the games industry. =\

  30. Rei Onryou says:

    @Adam T: Sounds like you need to play the Natural Selection mod for Half Life. The marines are FPS with a commander playing an RTS, building structures, giving orders, purchasing weapons for the soldiers.

    That was a good read. Shame usability doesn’t come naturally to me. It tends to be a roadblock in my design, but that’s why having usability testers is so important. It’s definitely right what he said about not being able to see good design, but bad design sticks out like a sore thumb.

  31. Radiant says:

    I LOVE a good manual.
    From cover to contents a good manual always gets me into a game more; when it’s missing or cursory I always feel that something is not really worth my time.

    Don’t get me started on those hateful strategy book.
    20 quid for something that should be half a quarter of it’s size and in the box already. [Sorry piggyback publishing I still love you!]

  32. aquazombie says:

    @clovus

    I wish baseball was that awesome…

  33. Radiant says:

    I would love to have read Schklar thoughts on casual games.

    For me; something as ‘simple’ as wasd controls on cas. games that end up in front of me drive me up the bloody wall.
    Do other devs not realise how complex that is for normal people?

  34. Jason Schklar says:

    Hey all:

    Thanks to Kieron for such a kind write up — and great questions. I’ll finish with a cricket analogy the next time ;)

    Apologies for the disabled comments on my blog (I required OpenId). I was having trouble with Chinese Spider Bots posting spam on my blog, so… I obviously over-corrected.

    Comments are now open! Hopefully I won’t get too much spiderspam.

    I’m dashing between meetings right now, but will post some responses to the thoughtful comments and questions above later on today.

    Cheers!

    J

  35. Tei says:

    “All of the great designers I’ve worked with crave this kind of feedback. Why? Because the game is still in production and they have a chance to fix it. No one wants to make a game that truly sucks. Fail early, fail often, and fail in front of usability participants who are under NDA as opposed to failing in front of folks who paid $50 to buy your game.”

    I am the only one here that think is absolutelly obvius?

  36. IanAetch says:

    @Adam T
    You also might want to play DICE’s Battlfield 2. The game has the option of having a commander for your team, who can view the map from top down and see where everybody is, and give orders to the players (if their organized in squads). The players see those orders as waypoints on thier huds.
    You can also order up artillery strikes and UAV passes and have vehicles and supplies dropped to your team mates/
    The commander is selected from among those who want the job and is picked from those based on past performance.

  37. Jason Schklar says:

    Whew. I’ve had a chance to re-read the post and here are some responses:

    @Radiant: I do talk a bunch about casual games on my blog. Your point illustrates my call for allowing the user to “fail gracefully” if they’re having trouble learning the controls scheme. Many casual games either drop you into the game with little or no instruction and you have to guess which keys to press — or they have boring and tedious tutorials. The key (in my experience) is to layer on controls and complexity as the game progresses in ways that are well paced and fun.

    @Dreamhacker: It’s pronounced “sklar”. The “ch” is silent ;)

    @Richard Beer: There are qualitative and quantitative ways to cost-justify usability and playability work. In the entertainment industry when you’re selling directly to the customer (gamer), the game must be fun to be successful. Qualitatively, the only way to know whether it’s fun or not is to test with users who are not professionals. If it’s not fun, then you need to make changes and then test again. Yeah, this costs time and money — but nothing’s worse than shipping a product that no one wants. Quantitatively, you can set up metrics that need to be passed or exceeded before the game is released in the wild.

    @Markoff Chaney: I use Gamefly (a video game rent-by-mail service), purchase games digitally, and play a lot of online casual games… This means fewer manuals lying around. But, I agree with your basic assertion that manuals make great bathroom reading material ;)

    Cheers,

    J

  38. Radiant says:

    @Jason Schklar hey thanks for replying!
    I find that layering in a control scheme is difficult for casual games because of the length of time each one is played.
    With something like Clown Killer 2 [a wasd flash shooter] it’s difficult to bring in the complex controls over time and still retain the ‘quick blast’ of fun the game is trying to achieve.
    Have you ever played any of the Hoshi’s Story series?
    I think you’d enjoy it; there’s no instructions bar “find the star” and it uses the players confusion to introduce quite complex interactions.
    They’re absolutely awesome to play and I think you’d find them interesting.

  39. Radiant says:

    Sorry I meant Hoshi Saga.

  40. Scandalon says:

    Clovus, you mad me laugh. That can be difficult to do.

    (But you forgot to explain “pass interference” and “free throws”)

  41. jsutcliffe says:

    An excellent read. Thanks for the article. Usability is a very interesting subject, and I don’t think I’d ever given much thought to how usability issues are solved in games.