RoomAlive Converts Any Room Into A Video Game

RoomAlive! That's the name of a Microsoft research project "that transforms any room into an immersive, augmented, magical gaming experience". Translated, that means it uses a combination of projectors and depth cameras to map a room and add in a bunch of augmented reality for your entertainment.

As you might expect there's been a certain amount of "FINALLY MAYBE WE'RE GETTING A HOLODECK!" bandied about in response. But I'd like to say to that, why would you want a holodeck? All it ever did was go wrong and try to kill everyone. RoomAlive sounds far more existence-friendly.

It relies on magic boxes packing a projector and a depth camera, arranged to completely cover a room from different angles. Using depth camera scans it creates a 3D model of the space and then, via the projectors, adds augmented content like robot enemies. It's also capable of picking up the player's movements and their interactions with said AR content.

I'm particularly interested in these projection-depth camera units because each of them contains a Kinect for Windows v1 sensor. I've long maintained that the Kinect is the single best thing Microsoft has created in recent memory. You can use it in gaming, sure, but also to create a robotic petting zoo, or to get cockroaches to map disaster zones or to help with sexual assault response training in the US Navy.

Want to read the example Microsoft give in the research paper [PDF]?

When the game starts, the room magically transforms into an ancient castle, the walls turn to stone, and flaming torches emerge from the walls casting flickering shadows onto the furniture. Out of the corner of your eye, you see a glowing idol appear on your couch. You walk towards the idol when suddenly, a trap opens on the wall next to you, exposing blow darts ready to fire. You leap out of the way, only to land on the floor face-to-face with a giant cockroach. You quickly get up and jump on the roach. You reach the idol successfully, and a scoreboard drops down showing that you have just scored the best time for the adventure course.

Hooray! I've never even played with this prototype and I'm already the best in the world at it! Also, does this mean Microsoft want to make an Indiana Jones game?

Anyway, for those of you reading this and wondering about the game dev side of things, RoomAlive uses Unity3D's modular plugin framework and scripting interface. According to the paper, the developer then just adds a game object to their scene to load the plugin and thus has access to the RoomAlive API.

The biggest issue in all of this would be scaling the game and its contents to fit all kinds of different spaces. The research team don't claim to have a complete solution to that but there are several techniques which help deal with the issue. The first is random positioning of targets. Another is mapping which takes account of which surfaces are appropriate to which content/textures. Third is placing content in relation to the user and fourth is allowing the user to position the content themselves – a kind of real-time level editing.

Next on the research list is how to deal with multiple users in the space. For, like, when you have friends over or when your mum decides to let the damn cat into the room or something.