Of Mice And Gamepads: The Future Of Controllers

If I’m going to be dull and reductive about it, playing videogames works like this: we tell a game something through an input device – say, a gamepad, motion contoller, touch screen or keyboard – and get a response back in the form of images or sound. It’s like a conversation, but it’s shaped by the devices we use to talk. Without the Wiimote, there is no Wii Sports. Without the touch screen, there is no Fingle or Bloop.

If I don’t own the relevant controller, then I can’t play these games. But what if the controller doesn’t even exist? Many games are impossible to conceive of because we don’t have the hardware to act as muse. Are we living on a junk diet of gamepads and mice – or a rich land of controller plenty?

Let’s have a chat with a few developers and see wot what they think.

Tale of Tales are known for their contemplative PC works such as The Path and The Endless Forest, but last year they released sensual stimulation simulation Luxuria Superbia. The player caresses the screen to ‘give pleasure’ – slow and gentle movements are prized over thrusts, endurance over speed.

“We had made a prototype for gamepad on PC but we wanted to make a simpler game to release,” Tale of Tales tells me. “We chose to make it for iPad to force us to keep the engine lean. The idea of touching plays a big role in Luxuria Superbia. So we thought it would make sense to work with a touch screen, in order to remove the need for an avatar to touch the virtual environment. We were also curious if the App Store would help us reach a new audience since we thought Luxuria Superbia would appeal widely.”

In fact, they admit they are usually indifferent regarding hardware. “Our choice of hardware is motivated mostly by the people using that hardware, not by the technology.”

Yet how the game feels depends on the device you are interacting with: with a tablet, it’s like trying to tease the G-spot of the touch screen; on the smaller screen of a smartphone, your fingers are clumsy and fat; on the PC with a gamepad, the player manipulates the analog sticks to arouse the game which feels more standoffish than the intimacy of a touch screen. Tale of Tales confirms: “On a touch screen, Luxuria Superbia feels more like a creature, a partner to play with. While on a big screen with a controller, playing feels more like travelling through an environment.”

This is an example of how the interface between player and machine defines the feel of the videogame. It’s like how playing chess facing an opponent resembles a duel, but if the players were to sit side-by-side some nuance would be lost. Same game, subtle difference.

But new hardware isn’t necessary to create something that feels fresh. Doug Wilson of Die Gute Fabrik has worked on several games that re-appropriate controllers in unusual ways. With Copenhagen Game Collective, he worked on B.U.T.T.O.N., where players race to press gamepad buttons in the manner the game dictates, and Dark Room Sex Game, which is a sort of rhythm game where players shake motion controllers to simulate sex. (I promise I won’t bring up any more sex-themed games. Wait, no, I might.)

More recently, Wilson has been developing the multiplayer game Johann Sebastian Joust, part of the Kickstarter-backed Sportfriends which was just released to console (a PC version is in the works). In Joust, each player holds a Move controller which they have to keep safe while trying to jostle other players’ controllers.

“A lot of these devices are, or were, under-explored,” explains Wilson, “and it’s always exciting to run wild in a rich, untapped design space. I like taking consumer technologies and subverting them to new ends. One of the things that surprised people about Joust was how I was re-appropriating a controller that most people had written off.”

Lest we concentrate too much on the input side of things, developers can also experiment with output. Incus Games have chosen to work on games for the visually impaired and are hard at work on their first game, Three Monkeys. From the build I played at last year’s Eurogamer Expo, I’d describe it as a first-person shooter without any graphics, yet it also feels like participating in a radio play, a delightful contrast to the big budget shooters which take the Hollywood blockbuster as their cue.

Incus director Steve Willey explains the genesis of the idea: “Because we have a number of audio guys in the team, naturally we talk a lot about audio implementation in games. I began thinking about the [Three Monkeys] concept a couple of years ago but it was through months of chatting it over as a team that the game began taking an identity and direction. We had noticed that there weren’t too many audio games out there and most of them relied on fear as the core mechanic.”

So are enough developers taking advantage of the toys already at their disposal? Joust’s Wilson thinks not. “I don’t think the majority of large commercial developers really understand how to utilize motion control to its full potential. That’s why you saw so much shovelware on the Wii, and so few decent games for the Kinect. I think it’s less about using inputs in ‘unconventional’ ways and more about learning to really grapple with the radical possibilities of that hardware.”

But what do you do if the interface you need for your game doesn’t exist? If you’re George Buckenham then you just go ahead and make it.

“It came from a stupid conversation on Twitter,” Buckenham says, “where people were putting together ‘verb the noun’ combinations. This was the funniest!” Buckenham is talking about a game he made called Punch the Custard. It’s pretty much what it says on the custard tin: two players punch custard for points when the computer tells them to.

“I think the idea stuck around until I was asked if I wanted to make something for Hide&Seek‘s Sandpit events. So I took a day and made it.” Now you won’t find ‘custard punching hardware’ in your local game store, so Buckenham had to make the sensors to detect player punches himself. You can’t buy the game, but the instructions on how to build it yourself are available online.

Punch the Custard is not an isolated case and Buckenham should know. He is one of the organisers of The Wild Rumpus, a ‘multiplayer indie game event’ that hosts not just games with traditional controllers such as Joust, Nidhogg and Luftrausers, but many games with custom interfaces. You’re just as likely to see Mega GIRP, Bennet Foddy’s finger-contorting GIRP played on dance pads instead of a keyboard, or the two-player Swordfight where each player straps a customised Atari joystick to the groin and tries to poke their opponent’s trigger with the stick. (Okay, now I totally promise not to mention any more sex-themed games.)

There’s a reason why the event gravitates towards the unconventional. “I think what we’re primarily looking for is spectacle,” says Buckenham. “We think a lot about the experience for people who come but don’t play things – it’s surprisingly common. And completely valid! These things provide an experience that you can’t get at home. If we’re just going to show local multiplayer games you could play on your couch, why bother?”

There’s another reason why Wild Rumpus shies away from keyboards and controllers. “Custom controllers lower the barrier to entry for new players. A 360 controller is intimidating if you don’t play games. A 360 controller covered in fur, with a little bumhole made out of a balloon poking out from under the tail, only the 2 analogue sticks exposed – less intimidating.”

Robin Arnott is a sound designer who terrified exhibition crowds with Deep Sea, which is referred to as ‘a sensory deprivation video game’. In the game, players have to survive an encounter with a sea monster in a pitch-black ocean environment. The specialised hardware in this case is a gas mask which blinds the player but there’s more to it than just darkness; the game plays back the sound of a diver’s regulator when the player breathes, which means the player must hold their breath to listen out for the creature.

Arnott says, “The original motivation behind Deep Sea was a dirt simple question: how do I maximize immersion? It was a curiosity drive! I started out knowing from my own experience that fear can short-cut the rational mind and touch players at a pre-cognitive level.”

Arnott has a theory that being able to see so much prevents us from seeing what games are about. “Games that reject visuals are rare because most games are about handling data. I think that’s what a lot of people think a game is. Handling data and making information-based decisions is as much a part of the paradigm of this medium as words are to literature. And humans are visual creatures, which means that we can process a lot of data in images.

“I think until we get past this paradigm of games being about interpreting and managing data, controllers will still be based around the dexterous hands and fingers. There are a ton of biofeedback technologies that are mature in their development. The only reason these haven’t been integrated into controllers yet is that market leaders think players want more of the same – games about navigating data and making decisions. That’s not what players want, that’s just what the edges of this particular skybox look like.”

It’s clear there is no shortage of ideas when it comes to new interfaces. Kiss Controller uses the tongue as input. ShockCube and PainStation are devices that inflict pain on losing players. What are developers looking for?

Tale of Tales are looking for one thing: people. “We are very keen on seeing the audience for videogames grow. We want the medium to reach out and spread. We worry that specialized interfaces sometimes have the opposite effect. The Wiimote was excellent. But Nintendo completely failed to work with the new audience it had managed to access through the Wii.”

They are not alone in feeling let down. “Using consumer technology means I can more easily distribute the game,” Wilson says. “What disappoints me is how a generation of shoddy motion control games have turned core gamers against the very idea of motion control gaming.”

Wilson can only see Microsoft’s recent decision to make Kinect optional for Xbox One as a negative development. By reducing the number of players who own Kinect, the financial risk for developers increases and, inevitably, chokes off the supply of games. “Console-based motion control and physical play was already largely ‘dead,’ but Microsoft dropping the Kinect is a symbolic moment – another nail in the coffin. What Microsoft lacked was developers who knew how to think beyond the immersive fallacy and subvert technological constraints. As I see it, Microsoft could have done a better job supporting and incentivizing Kinect developers.”

Developers rely on big companies to run the controller show because while new interfaces are interesting, the problem is commercial viability. There are exceptions. Some games are sold with custom hardware like Guitar Hero or Atari’s driving controller with the 1977 Atari game cartridge Indy 500, but these are still products driven by corporate behemoths.

Buckenham argues that commercializing custom hardware means making some hard decisions. “It’s hard from a commercial point of view, if you’re operating at a small scale. We faced this with Tenya Wanya Teens – how to commercialize a game when each controller costs £500 and requires being hand-built? We’re still figuring this out, but likely if it does happen, it’ll be reworked for a more standard controller.”

But he’s pragmatic that commercialization isn’t an issue because many installation games only make sense in a public context. “There’s no real way to make money on it, so you have to squeeze it alongside other work, and can’t focus on it properly. Personally, I love this stuff, and I would love to see more things being made for this context. But there is also a lot of really good stuff in this space that is being made. That GDC could have an expo of alternate controller interfaces says something.”

However, expecting players to have enough controllers can be just as risky as developing custom ones. Sportsfriends contains Joust, which requires a Move controller for each player, and Hokra, a four-player game that works best with four gamepads. Could Sportsfriends change our attitude to controllers?

“I think the real question here is: do controllers become more commonplace?” says Wilson. “Like, will it be common for PS4 owners to own two or even four controllers? Will PC users get more comfortable connecting controllers to their computers? Will the Steam controller become a mainstream purchase? Does Apple end up designing a controller to go with Apple TV? As my colleague Bennett Foddy says, the ‘platform’ for local multiplayer isn’t any one console or operating system. It’s controllers in general. So, we’ll have to wait and see what kind of hardware makes its way into the mainstream.”

But there’s some pessimism about the future. “We’re not very excited about the new interfaces that the engineers are coming up with,” say Tale of Tales. “They don’t inspire us. There’s a serious lack of imagination in hardware creation. But that’s probably motivated by the shortsightedness of capitalism that is apparently the sole reason why any widely spread hardware is created these days.”

On the back of the Microsoft Kinect decision, Wilson sees a conservative trend forming. “I think it’s getting less and less diverse these days. It really feels like the latest generation of consoles have all but given up on motion control. Guitar Hero is past its prime, there are hardly any Kinect games, almost nobody is developing for the Move, and Nintendo seems to have backed away from physical play in favour of a more iPad-like experience. It’s starting to feel like the heyday of alternative interfaces – at least on a large-scale commercial level – is behind us.”

Buckenham disagrees. “I don’t think we are lacking in interfaces – there’s endless commercial ones now, and a willingness to experiment with them. Look at the Oculus Rift, the Wiimote, force-feedback driving wheels, TrackIR, touch screens, fightsticks, Skylanders, Panoramical… there’s so many niches where there’s weird new controllers.”

It is Arnott, however, who is the most optimistic. His followup to Deep Sea is a project called Soundself which makes use of the Oculus Rift. He sees VR as breaking the paradigm of games being about data.

“If I were in the console business right now, I’d be looking for a way to get ahead of the curve by integrating heartbeat sensors, breath-tracking, and EEGs into peripherals. The next generation of what we call games will not be about using the body as a means of control. It will be defined by experiences that blur the lines between self and software. This leap is right around the corner.”


  1. Alice O'Connor says:

    I made that weird gross cat controller cover up top link to shoving.biz

    • Ross Angus says:

      And I see it’s loosely based on your avatar.

    • Joel Goodwin says:

      When I went looking for pictures of the controller George described, I had to go on his description alone, because he hadn’t told me what game he was referring to. Google delivered me to your page with searches for furry controllers.

    • KDR_11k says:

      It reminds me of the cover that came with my C64’s mouse. Evidently the previous owner had trouble visualizing why the thing was called a mouse.

    • kwyjibo says:

      Was it made from a Merkin? Which according to Rab, is a front bum wig – link to youtube.com

    • lurkalisk says:

      Cat? Thought it some sort of bog-tribble…

  2. disconnect says:

    Sexy iPad game Fingle is free at the moment fyi link to itunes.apple.com

  3. pupsikaso says:

    What the heck is up with those tags? Spambots somehow managed to edit these?

    • Phasma Felis says:

      They’re all games/creators mentioned in the article. Yes, even “dark room sex game”.

  4. SillyWizard says:

    Oh shit where’s David Cronenberg?!

    • Stupoider says:

      eXistenZ with it’s weird foetus controllers was bloody surreal. Thought the movie was naff but the designs were impeccable.

  5. The Random One says:

    So many sex games. Is it possible that… *dashes forward, grabs Joel’s face and pulls it off* Gasp! Cara Ellison?

  6. Meusli says:

    I must be a Luddite or something, but when I see Devs/Publishers/Game Journalists talking about new controller input methods I just roll my eyes. These types of control input are and always will be rubbish to me and they never once lived up to the potential we are sold because there was none, it is all just hype. All they are capable of is a collection of bland and simple games with no depth to them at all, which is fine if all you have is a phone/tablet but on my PC, F-off. One exception to this was a series that was quickly killed to death with massive over-exposure ala Guitar Hero.
    So what is the next thing they are trying to convince us is the latest in a line of duff ideas?

    “If I were in the console business right now, I’d be looking for a way to get ahead of the curve by integrating heartbeat sensors, breath-tracking, and EEGs into peripherals.”

    Do these guys even live on the same planet as us? Why on earth would we give publishers access to what our bodies are like, which will probably be sold to the highest bidder for that extra profit, just to play a god damned game. And how convenient is this nonsense? Do I have to suit up to begin playing? Sod that I just want to relax and unwind not be monitored by evil companies that always look at amazing new ways to empty our wallet .

    ex-Kinect purchaser and now boycotter of all nonsense control inputs.

    • Shuck says:

      “Please insert rectal thermometer to begin play!”
      Yeah, the move towards heart/EEG/breathing tracking is a bit concerning, given how data could get used, and how limited the benefits are to game control, etc.
      The problem with these new controllers is not that they lack potential, but that the economic realities of game development mean that they’re unlikely to have that potential explored. The Kinect controller didn’t get the exploration it deserved because a) developers are doing cross-platform development, so they’re focusing on common means of control, and b) they’re thinking about games in the context of the controls they’re already familiar with from use. Most game developers can’t afford to spend a lot of time and resources playing with and thinking about the possibilities of a new controller to the point where they’re designing games around it if it’s not owned by their entire target audience. So you either end up with simple, cheap games that use the most obvious features of the controller, or limited use of the controller in more complex games.

    • The Random One says:

      Before you lambast people for dreaming up new controller schemes, remember that a great deal of us PC folk will swear that half of the ultimate control method must be a device built for writing on.

    • Consumatopia says:

      We almost certainly will have a lot more devices monitoring our bodies for medical reasons. I suppose it’s not impossible someone will think to repurpose them for games…

    • Smoky_the_Bear says:

      Give credit to the Wii it was fun as hell when it first came out. Sadly it lead to a lot of bandwagoning from Microsoft and Sony trying to push all sorts of dancing around in front of the TV type nonsense that is nothing more than a gimmick and adds nothing to videogaming outside of some drunken larking around post-pub that gets boring after about an hour.

      While I agree that most of these gimmicky control methods are nonsense, people need to continue conceptualising these things for any forward progress to be made. It’s unrealistic to think we will be using mouse and keyboard for the rest of eternity, something will replace it one day, or it will gradually change into something unrecognisable. Without people attempting to make something new, nothing actually good will ever be made, even if we do have to sift through the shit to find it.

    • xao says:

      I mean, seriously. What kind of fool would use a pointing device like a mouse to play a game when he could be using paddles? I heard someone actually tried to put TWO buttons on a mouse. Heh, I imagine they ran him out of town on a rail. The old ways are the best ways. Change is bad. Innovation must be stifled. We have always been at war with Oceania.

  7. Michael Fogg says:

    I await the standard whiny posts about how adult people find it very hard and generally beneath them to rewire their brains for any new manual skills and therefore reject any notion of controller other than their typewriters and pointing devices.

    • Meusli says:

      It will all end in tears, you mark my words. :D

    • malkav11 says:

      All I need is a convincing demonstration of why the new interface is needed. I was extremely skeptical of the DS, which was a very weird duck among handhelds, but it turned out that there were a number of creative possibilities that they could explore with things like the stylus/touch controls, the microphone, the dual screen setup, etc. Things which really did not translate to other devices. I kept waiting for anyone to demonstrate similar potential in the Wii’s motion controls, in Kinect, and these days in the pure touch interfaces of tablets and other mobile devices. I’m still waiting. I think there are probably a few games out there that do things with those interfaces that could not be done with conventional gamepad or M&K controls (it seems like the big killer app for Kinect was dance games), but I was certainly not able to find any that delivered gameplay that actually interested me, and with all of the above interfaces there’s a dramatic loss in fidelity, responsiveness, and accuracy of input for more conventional gameplay. Supposedly it’s immersive, but when my motion does not actually translate into the intended motion onscreen I don’t find that very immersive.

    • Consumatopia says:

      Actually, it *IS* beneath me to learn new, inferior ways of doing what I can already do. This is a standard complaint because it’s absolutely true, yet interface designers keep insisting on making things new for the sake of new. (Games aren’t even the worst offenders.)

  8. Shuck says:

    I’ve always been skeptical of the Kinect, etc. – the problem was that you had sufficiently different types of motion controllers for each console, making it impossible to have cross-platform games that use the controllers in anything but the most superficial ways, with support tacked on as an afterthought. Motion controls work best for platform-exclusive games., which is why the Wii motion controller got used – making Wii games was relatively cheap. The economic realities of game-making mean that you’re normally going for the largest possible audience; a game of any sized budget can’t afford to be built around non-standard controllers. That leaves indie developers as being in the best position to use non-standard controllers, but they’re also the least likely to actually spur sales of those controllers. Radically new controllers end up limited to special game events and art performances.

  9. hypercrisis says:

    Naff and gimmicky arcade games in an era when arcades are dead, what could possibly go wrong?

    • Hypocee says:

      Now now, B.U.T.T.O.N. isn’t an arcade game, it’s a party ‘game’. Draw a card! Do the silly thing the card says! It’s funny if you’re drunk! Sometimes!

    • RobF says:

      Dunno. Someone might go and have some fun by accident and we couldn’t be having that.

  10. Hypocee says:

    Games whose essence is their physical prop seem generally cool – Analog Defender, Steel Battalion, that caravan spaceship sim – but I’m skeptical of attempts to expand general-purpose control ‘methods’. The last big change we saw succeed was touch, which flowed out of general device usability rather than any game concerns and only works for a few genres. Attempts to drive new methods rather than accepting what comes along strike me as similar to attempting to reinvent Newtonian mechanics; in any reasonably mature field of endeavour, chances are the low-hanging fruit was found first. Existing methods probably cover the majority of the available phase space.

  11. Tusque D'Ivoire says:

    Welcome Joel!

    • Joel Goodwin says:

      Thanks! Not sure how often I’ll be contributing as most of my time goes into my own site. But I’ll definitely be back again with more words of some sort.

  12. Tiltowait says:

    Give me a Logitech G13 with a track ball on it or maybe something like the Kings Assembly with a track ball on it. link to kickstarter.com
    I don’t want to wave my hands around or have parts of my anatomy read as input.

  13. Heimmrich says:

    We, for some reason, tend to think about new technologies in simple terms of “progress”. From my perspective, theres no need for new input devices to be “better” than the standard, they just need to be different. Its about what we can convey in this particular medium and pushing boundaries and breaking paradigms.

    I think this is very hard for some people to understand because traditionally games are built around the input devices people have, not the other way around. Games are usually designed however the technology at hand allows developers to.

    To say that we should stop thinking about new ways to play seems ridiculous to me. In this sense I think that we should take a long and hard look on boardgames and take notes (think of Vlaata’s Space Alert).

    I think Nintendo DS goes a great lenght to show how much creative developers can be in using an input method that was somewhat new at the time (if the economic enviroment allows them to).

    A breathtracker or a heart beat sensor sounds like a great idea to me. I, at least, trust that people can be inventive enough with those devices to produce something that uses these “gimmicks” in a significant way.

    Also, fearing “giving access to our bodily functions” because Big Brother and Global Domination is irrational to me. We are okay with VR technology and Google Glass and being sold shit to eat and drink on every corner, but are worried on the ethics behind someone knowing how fast our heart is beating? First we have to conceptualize new technology, then we discuss the ethical problems that arise (which is as necessary and as important as pushing technology forward)