Skip to main content

Of Mice And Gamepads: The Future Of Controllers

Beyond The Joystick

If I’m going to be dull and reductive about it, playing videogames works like this: we tell a game something through an input device – say, a gamepad, motion contoller, touch screen or keyboard – and get a response back in the form of images or sound. It’s like a conversation, but it’s shaped by the devices we use to talk. Without the Wiimote, there is no Wii Sports. Without the touch screen, there is no Fingle or Bloop.

If I don’t own the relevant controller, then I can’t play these games. But what if the controller doesn’t even exist? Many games are impossible to conceive of because we don’t have the hardware to act as muse. Are we living on a junk diet of gamepads and mice – or a rich land of controller plenty?

Let’s have a chat with a few developers and see wot what they think.

Tale of Tales are known for their contemplative PC works such as The Path and The Endless Forest, but last year they released sensual stimulation simulation Luxuria Superbia. The player caresses the screen to ‘give pleasure’ – slow and gentle movements are prized over thrusts, endurance over speed.

“We had made a prototype for gamepad on PC but we wanted to make a simpler game to release,” Tale of Tales tells me. “We chose to make it for iPad to force us to keep the engine lean. The idea of touching plays a big role in Luxuria Superbia. So we thought it would make sense to work with a touch screen, in order to remove the need for an avatar to touch the virtual environment. We were also curious if the App Store would help us reach a new audience since we thought Luxuria Superbia would appeal widely.”

In fact, they admit they are usually indifferent regarding hardware. “Our choice of hardware is motivated mostly by the people using that hardware, not by the technology.”

Yet how the game feels depends on the device you are interacting with: with a tablet, it’s like trying to tease the G-spot of the touch screen; on the smaller screen of a smartphone, your fingers are clumsy and fat; on the PC with a gamepad, the player manipulates the analog sticks to arouse the game which feels more standoffish than the intimacy of a touch screen. Tale of Tales confirms: “On a touch screen, Luxuria Superbia feels more like a creature, a partner to play with. While on a big screen with a controller, playing feels more like travelling through an environment.”

This is an example of how the interface between player and machine defines the feel of the videogame. It’s like how playing chess facing an opponent resembles a duel, but if the players were to sit side-by-side some nuance would be lost. Same game, subtle difference.

But new hardware isn’t necessary to create something that feels fresh. Doug Wilson of Die Gute Fabrik has worked on several games that re-appropriate controllers in unusual ways. With Copenhagen Game Collective, he worked on B.U.T.T.O.N., where players race to press gamepad buttons in the manner the game dictates, and Dark Room Sex Game, which is a sort of rhythm game where players shake motion controllers to simulate sex. (I promise I won’t bring up any more sex-themed games. Wait, no, I might.)

More recently, Wilson has been developing the multiplayer game Johann Sebastian Joust, part of the Kickstarter-backed Sportfriends which was just released to console (a PC version is in the works). In Joust, each player holds a Move controller which they have to keep safe while trying to jostle other players’ controllers.

“A lot of these devices are, or were, under-explored,” explains Wilson, “and it's always exciting to run wild in a rich, untapped design space. I like taking consumer technologies and subverting them to new ends. One of the things that surprised people about Joust was how I was re-appropriating a controller that most people had written off.”

Lest we concentrate too much on the input side of things, developers can also experiment with output. Incus Games have chosen to work on games for the visually impaired and are hard at work on their first game, Three Monkeys. From the build I played at last year’s Eurogamer Expo, I’d describe it as a first-person shooter without any graphics, yet it also feels like participating in a radio play, a delightful contrast to the big budget shooters which take the Hollywood blockbuster as their cue.

Incus director Steve Willey explains the genesis of the idea: “Because we have a number of audio guys in the team, naturally we talk a lot about audio implementation in games. I began thinking about the [Three Monkeys] concept a couple of years ago but it was through months of chatting it over as a team that the game began taking an identity and direction. We had noticed that there weren’t too many audio games out there and most of them relied on fear as the core mechanic.”

So are enough developers taking advantage of the toys already at their disposal? Joust’s Wilson thinks not. “I don't think the majority of large commercial developers really understand how to utilize motion control to its full potential. That's why you saw so much shovelware on the Wii, and so few decent games for the Kinect. I think it's less about using inputs in ‘unconventional’ ways and more about learning to really grapple with the radical possibilities of that hardware.”

But what do you do if the interface you need for your game doesn’t exist? If you’re George Buckenham then you just go ahead and make it.

“It came from a stupid conversation on Twitter,” Buckenham says, “where people were putting together ‘verb the noun’ combinations. This was the funniest!” Buckenham is talking about a game he made called Punch the Custard. It’s pretty much what it says on the custard tin: two players punch custard for points when the computer tells them to.

“I think the idea stuck around until I was asked if I wanted to make something for Hide&Seek's Sandpit events. So I took a day and made it.” Now you won’t find ‘custard punching hardware’ in your local game store, so Buckenham had to make the sensors to detect player punches himself. You can’t buy the game, but the instructions on how to build it yourself are available online.

Punch the Custard is not an isolated case and Buckenham should know. He is one of the organisers of The Wild Rumpus, a ‘multiplayer indie game event’ that hosts not just games with traditional controllers such as Joust, Nidhogg and Luftrausers, but many games with custom interfaces. You’re just as likely to see Mega GIRP, Bennet Foddy’s finger-contorting GIRP played on dance pads instead of a keyboard, or the two-player Swordfight where each player straps a customised Atari joystick to the groin and tries to poke their opponent’s trigger with the stick. (Okay, now I totally promise not to mention any more sex-themed games.)

There’s a reason why the event gravitates towards the unconventional. “I think what we're primarily looking for is spectacle,” says Buckenham. “We think a lot about the experience for people who come but don't play things – it's surprisingly common. And completely valid! These things provide an experience that you can't get at home. If we're just going to show local multiplayer games you could play on your couch, why bother?”

There’s another reason why Wild Rumpus shies away from keyboards and controllers. “Custom controllers lower the barrier to entry for new players. A 360 controller is intimidating if you don't play games. A 360 controller covered in fur, with a little bumhole made out of a balloon poking out from under the tail, only the 2 analogue sticks exposed – less intimidating.”

Robin Arnott is a sound designer who terrified exhibition crowds with Deep Sea, which is referred to as ‘a sensory deprivation video game’. In the game, players have to survive an encounter with a sea monster in a pitch-black ocean environment. The specialised hardware in this case is a gas mask which blinds the player but there’s more to it than just darkness; the game plays back the sound of a diver’s regulator when the player breathes, which means the player must hold their breath to listen out for the creature.

Arnott says, “The original motivation behind Deep Sea was a dirt simple question: how do I maximize immersion? It was a curiosity drive! I started out knowing from my own experience that fear can short-cut the rational mind and touch players at a pre-cognitive level.”

Arnott has a theory that being able to see so much prevents us from seeing what games are about. “Games that reject visuals are rare because most games are about handling data. I think that's what a lot of people think a game is. Handling data and making information-based decisions is as much a part of the paradigm of this medium as words are to literature. And humans are visual creatures, which means that we can process a lot of data in images.

“I think until we get past this paradigm of games being about interpreting and managing data, controllers will still be based around the dexterous hands and fingers. There are a ton of biofeedback technologies that are mature in their development. The only reason these haven't been integrated into controllers yet is that market leaders think players want more of the same – games about navigating data and making decisions. That's not what players want, that's just what the edges of this particular skybox look like.”

It’s clear there is no shortage of ideas when it comes to new interfaces. Kiss Controller uses the tongue as input. ShockCube and PainStation are devices that inflict pain on losing players. What are developers looking for?

Tale of Tales are looking for one thing: people. “We are very keen on seeing the audience for videogames grow. We want the medium to reach out and spread. We worry that specialized interfaces sometimes have the opposite effect. The Wiimote was excellent. But Nintendo completely failed to work with the new audience it had managed to access through the Wii.”

They are not alone in feeling let down. “Using consumer technology means I can more easily distribute the game,” Wilson says. “What disappoints me is how a generation of shoddy motion control games have turned core gamers against the very idea of motion control gaming.”

Wilson can only see Microsoft’s recent decision to make Kinect optional for Xbox One as a negative development. By reducing the number of players who own Kinect, the financial risk for developers increases and, inevitably, chokes off the supply of games. “Console-based motion control and physical play was already largely ‘dead,’ but Microsoft dropping the Kinect is a symbolic moment – another nail in the coffin. What Microsoft lacked was developers who knew how to think beyond the immersive fallacy and subvert technological constraints. As I see it, Microsoft could have done a better job supporting and incentivizing Kinect developers.”

Developers rely on big companies to run the controller show because while new interfaces are interesting, the problem is commercial viability. There are exceptions. Some games are sold with custom hardware like Guitar Hero or Atari’s driving controller with the 1977 Atari game cartridge Indy 500, but these are still products driven by corporate behemoths.

Buckenham argues that commercializing custom hardware means making some hard decisions. “It's hard from a commercial point of view, if you're operating at a small scale. We faced this with Tenya Wanya Teens – how to commercialize a game when each controller costs £500 and requires being hand-built? We're still figuring this out, but likely if it does happen, it'll be reworked for a more standard controller.”

But he’s pragmatic that commercialization isn’t an issue because many installation games only make sense in a public context. “There's no real way to make money on it, so you have to squeeze it alongside other work, and can't focus on it properly. Personally, I love this stuff, and I would love to see more things being made for this context. But there is also a lot of really good stuff in this space that is being made. That GDC could have an expo of alternate controller interfaces says something.”

However, expecting players to have enough controllers can be just as risky as developing custom ones. Sportsfriends contains Joust, which requires a Move controller for each player, and Hokra, a four-player game that works best with four gamepads. Could Sportsfriends change our attitude to controllers?

“I think the real question here is: do controllers become more commonplace?” says Wilson. “Like, will it be common for PS4 owners to own two or even four controllers? Will PC users get more comfortable connecting controllers to their computers? Will the Steam controller become a mainstream purchase? Does Apple end up designing a controller to go with Apple TV? As my colleague Bennett Foddy says, the 'platform' for local multiplayer isn't any one console or operating system. It's controllers in general. So, we'll have to wait and see what kind of hardware makes its way into the mainstream.”

But there’s some pessimism about the future. “We're not very excited about the new interfaces that the engineers are coming up with,” say Tale of Tales. “They don't inspire us. There's a serious lack of imagination in hardware creation. But that's probably motivated by the shortsightedness of capitalism that is apparently the sole reason why any widely spread hardware is created these days.”

On the back of the Microsoft Kinect decision, Wilson sees a conservative trend forming. “I think it's getting less and less diverse these days. It really feels like the latest generation of consoles have all but given up on motion control. Guitar Hero is past its prime, there are hardly any Kinect games, almost nobody is developing for the Move, and Nintendo seems to have backed away from physical play in favour of a more iPad-like experience. It's starting to feel like the heyday of alternative interfaces – at least on a large-scale commercial level – is behind us.”

Buckenham disagrees. “I don't think we are lacking in interfaces – there's endless commercial ones now, and a willingness to experiment with them. Look at the Oculus Rift, the Wiimote, force-feedback driving wheels, TrackIR, touch screens, fightsticks, Skylanders, Panoramical... there's so many niches where there's weird new controllers.”

It is Arnott, however, who is the most optimistic. His followup to Deep Sea is a project called Soundself which makes use of the Oculus Rift. He sees VR as breaking the paradigm of games being about data.

“If I were in the console business right now, I'd be looking for a way to get ahead of the curve by integrating heartbeat sensors, breath-tracking, and EEGs into peripherals. The next generation of what we call games will not be about using the body as a means of control. It will be defined by experiences that blur the lines between self and software. This leap is right around the corner.”

Read this next