RoomAlive Converts Any Room Into A Video Game

They're coming through the walls!

RoomAlive! That’s the name of a Microsoft research project “that transforms any room into an immersive, augmented, magical gaming experience”. Translated, that means it uses a combination of projectors and depth cameras to map a room and add in a bunch of augmented reality for your entertainment.

As you might expect there’s been a certain amount of “FINALLY MAYBE WE’RE GETTING A HOLODECK!” bandied about in response. But I’d like to say to that, why would you want a holodeck? All it ever did was go wrong and try to kill everyone. RoomAlive sounds far more existence-friendly.

It relies on magic boxes packing a projector and a depth camera, arranged to completely cover a room from different angles. Using depth camera scans it creates a 3D model of the space and then, via the projectors, adds augmented content like robot enemies. It’s also capable of picking up the player’s movements and their interactions with said AR content.

I’m particularly interested in these projection-depth camera units because each of them contains a Kinect for Windows v1 sensor. I’ve long maintained that the Kinect is the single best thing Microsoft has created in recent memory. You can use it in gaming, sure, but also to create a robotic petting zoo, or to get cockroaches to map disaster zones or to help with sexual assault response training in the US Navy.

Want to read the example Microsoft give in the research paper [PDF]?

When the game starts, the room magically transforms into an ancient castle, the walls turn to stone, and flaming torches emerge from the walls casting flickering shadows onto the furniture. Out of the corner of your eye, you see a glowing idol appear on your couch. You walk towards the idol when suddenly, a trap opens on the wall next to you, exposing blow darts ready to fire. You leap out of the way, only to land on the floor face-to-face with a giant cockroach. You quickly get up and jump on the roach. You reach the idol successfully, and a scoreboard drops down showing that you have just scored the best time for the adventure course.

Hooray! I’ve never even played with this prototype and I’m already the best in the world at it! Also, does this mean Microsoft want to make an Indiana Jones game?

Anyway, for those of you reading this and wondering about the game dev side of things, RoomAlive uses Unity3D’s modular plugin framework and scripting interface. According to the paper, the developer then just adds a game object to their scene to load the plugin and thus has access to the RoomAlive API.

The biggest issue in all of this would be scaling the game and its contents to fit all kinds of different spaces. The research team don’t claim to have a complete solution to that but there are several techniques which help deal with the issue. The first is random positioning of targets. Another is mapping which takes account of which surfaces are appropriate to which content/textures. Third is placing content in relation to the user and fourth is allowing the user to position the content themselves – a kind of real-time level editing.

Next on the research list is how to deal with multiple users in the space. For, like, when you have friends over or when your mum decides to let the damn cat into the room or something.

19 Comments

  1. LionsPhil says:

    You walk towards the idol when suddenly, a trap opens on the wall next to you, exposing blow darts ready to fire. You leap out of the way…

    CRASH! There goes the telly.

    …only to land on the floor face-to-face with a giant cockroach. You quickly get up and jump on the roach.

    …only to trip over the coffee table and smash your face into the sofa.

    There’s a reason we go outside (or at least to larger, dedicated indoor areas) to play real-world games, e.g. sports.

    • Hmm-Hmm. says:

      Yeah, when watching the video I couldn’t help thinking about how the people would have fared if the furniture would’ve been a bit more tricky to play around.

  2. Jim Rossignol says:

    I can only read the name of this thing in Brian Blessed’s voice.

  3. Scoober says:

    Surely the issue with this system is the extremely steep setup costs? In their examples it takes 6 projectors and kinects to cover a room. I can see this being used in museums and other public spaces, but for an individual? Not so likely to take off.

    • Chuckleluck says:

      Maybe this will bring back arcades.

      I can see the inexplicably sticky furniture now.

  4. mashkeyboardgetusername says:

    Well, an interesting step I suppose. Couldn’t help thinking those games look pretty lame though, and I’m not sure how much you can do in what is probably quite a small space for most people (with furniture and whatnot).

    I do wonder if this’d be better as a system to put things in your peripheral vision while you’re primarily focussing on a monitor or TV for your game playing. Even having your surroundings go jungle-y when you’re in a jungle or urban-y when you’re in a city or hyperspace-y when yougettheidea could be pretty cool.

    Also, you could seriously torment a cat with this system. Mice everywhere!

  5. Tiax says:

    Wow, the luminosity produced by 6 projectors really makes the displayed stuff look bland. No contrast at all.

  6. RayEllis says:

    This sounds like more of Microsoft’s “It’ll be fantastic!” rhetoric which then turns out to be a crushing failure. More so if the Kinect is involved. I don’t know very many people who could honestly say they liked the thing, let alone got it to work correctly.

    When a company can’t even get it’s voice recognition software/hardware to respond to something as basic as “XBox on!” they aren’t doing it right. I take their wild dreams of an augmented living-room reality with a huge dose of salt.

    I would be interested to read the terms and conditions for this product, if it ever comes to market. If the section dealing with injuries sustained whilst using the device isn’t locked down tightly, then I foresee a slew of claims for damages as people cartwheel over the backs of sofas and punch fists through windows and TV’s etc.

  7. mattevansc3 says:

    I think its worth pointing out that, as mentioned at the top of the article, this is not a product its an R&D project. This will likely never make it into the consumer space (cost and practicality concerns) but elements from this project will likely make their way into other products.

    • Geebs says:

      “Thank you for choosing XBox Tau. An install of Kinect Roomalive! is mandatory. Our heavily-armed technical operatives will visit your place of residence within the next 48 hours. Please ensure that you remain within your residence during this period. If you are unable to admit our operatives within 5 seconds of knocking, you are bound by this agreement to pay for any replacements of doors, windows and/or pets. Kinect Roomalive! Cannot be turned off at any time. For your convenience, Kinect Roomalive! Is now activated by heavy breathing or the invocation of a deity as well as your denoted phrase”.

      “TV!”

      “…..shhh”

  8. Chuckleluck says:

    This would be cool IF we could have genuine holograms. I don’t see much fun in fighting an enemy if they’re projected against a couch – they gotta be freestanding.

    • Big Murray says:

      One step at a time, my friend. We needed to do 2D graphics before we could play games in 3D.

  9. Premium User Badge

    Neurotic says:

    I’m just sitting here pondering its application in the realm of advanced fapping. If it could somehow scan my sofa or armchair and then re-project it as an attractive woman (designable in the related character creator software), this could be a brilliant thing indeed.

  10. DrManhatten says:

    Finally some real VR/AR or VR done right not that crap called Occula Rift. This is technology worth of this century not technology for last century.

  11. frogmanalien says:

    It’s a fascinating concept, and one I can see catching on far easier than the VR headset (sorry world- but I honestly can’t see mainstream consumers buying into VR- and as long as there’s only a tiny fragment of the market interested in VR headsets it simply isn’t going to get the big developer interest).
    I can see a whole load of issues in this early demo – from the poor quality imagery (light leak) to the challenges of having this in a “real room” (light from outside sources, real world objects getting in the way) plus the cost of projectors is still high (even a single cheap projector would cost, say £100, multiply that by about 5, through in a Kinect or two, a console plus the gear to mount this and suddenly you wonder if you’d need to be able to afford a new house to home it) – but these seem like hurdles that could be overcome in time…
    I look forward to meeting Moriarty