Tabletop Gaming: HoloLens Meets Minecraft

Microsoft wowed me with an impressive display of their HoloLens augmented virtual reality visor yesterday. Jury’s out on whether it was real magic or just an illusion, but the hardware’s integration with Minecraft looks like the stuff of sci-fi. Minecraft worlds appear on your tabletop with full gesture and voice commands allowing you to navigate and interact in realtime.

In the video below you’ll see how two people can play together – one using the HoloLens and the other playing ‘normally’ on a Microsoft Surface (not that anyone actually plays Minecraft on a Surface). It almost seems entirely designed to satisfy and/or develop anyone’s latent god complex, though there are also some practical benefits. HoloLens users can lift the game world up and scout for cave systems and rare materials, before mapping interesting co-ordinates for normal players to go explore.

A single word can trigger a lightning strike or center the view on a particular player, letting the HoloLens act like a Palantír. There was no demonstration of the create mode, which will be the real selling point, possibly because of the visible latency when it came to gesture control. Whether that will be improved before launch remains to be seen. Even so, it’s an impressive display. But what are the chances it’ll be priced affordably?


  1. Joe W-A says:

    I’ve played quite a lot of Minecraft on a Surface!

  2. Premium User Badge

    Oakreef says:

    Black & White 3 for this please

  3. JimThePea says:

    Okay, props for making me care about Minecraft and augmented reality again, maybe lacks the tangibility of Lego, but virtual blocks feel better underfoot.

  4. FreeTom says:

    Hmmm, I watched this live. Either it was by far the most impressive thing in their conference, or else it was a simple lie. And they didn’t *act* like it was the most impressive thing in their conference…

  5. Wertymk says:

    That’s pretty amazing, though I’m waiting for the first Warhammer 40k hololens game.

  6. RaveTurned says:

    Currently imagining a multiplayer D&D-style RPG. Player characters running around in first or third person, DM controlling enemies and NPCs using a hololens overview.

    • PoulWrist says:

      I was thinking it would be great for an actual game of D&D to use as a tool for augmenting scenarios. See your players’ characters and NPCs move around on the table, instead of fickle figurines or tokens, and build dungeons on the fly instead of just drawing on our battlemat. That’d be great.

      • Harlander says:

        So when’s your Kickstarter starting? ;)

      • TheMightyEthan says:

        Hell, it could be used to hugely improve just about any tabletop or board game of any complexity. No more having to use three tables to play Firefly!

      • RaveTurned says:

        Yeah, suppose there’s no real reason why players shouldn’t be observing the game space in the same way as the DM, or actually running about the game world.

      • Xzi says:

        I see your D&D and raise you a Dungeon Keeper-type game. Hell, my personal favorite would be a top-down roguelike. In any case, this is indeed very impressive.

  7. dangermouse76 says:

    If it looks that good through the actual device then yeah that was quite something. The speed at which it rendered in the map as the camera moved was astonishing. And when he moved the camera inside the building to see the player, wow.

    • iainl says:

      General opinion from those who have used it is that yes, it’s that good in terms of response. What it does have a limitation with, however, is that the field of view isn’t anywhere near as large as on the VR competition.

      However, there are two big caveats to that complaint:

      1) Smaller view means fewer pixels, and so more humane GPU requirements for fluid response (and it does have fluid response).

      2) Because the artificial thing just disappears and you can still see the room around the edge of your field of view, it’s neither as critical that your immersion is broken, nor that bad a thing since the person moving in the edge of your vision is a human wanting your attention, rather than a game thing. You’re already asking it to display a limited amount of data when you define the edge of the table as the limit, anyway.

      • dangermouse76 says:

        I had read the field of view was narrower. But also 1st gen device has room to improve. I was imagining a 3D version of Lemmings playing on this thing. Your 2 point really ties into – as mentioned above – tabletop D&D could really take this device to some very interesting places. Manipulating the board and buildings or characters.

  8. Sp4rkR4t says:

    Microsoft need to dust off all the god-game/RTS IP they own for the launch of Hololens. If they do that they will be rolling in money.

  9. melnificent says:

    What I’d love to see is Tabletop Simulator: Hololens edition.

    Or it being used in something like commander mode on battlefield.

  10. quidnunc says:

    this was clearly a staged demo. it would have been more impressive if it wasn’t smoke and mirrors. it smacks of the original kinect demos which were impressive and very very fake.

    • dangermouse76 says:

      Interesting. By staged what do you mean ? Do you think the adapted lens on the camera was not capturing the game in real time. Do you think what we saw was not a live demo of the tech ?

      • quidnunc says:

        by staged I mean they weren’t actually playing it in real time which is of course not unusual for a lot of games on the e3 stage which are also faked but it’s relevant when they are using something with a nonstandard input method which is prone to errors

        obviously they were showing a third person perspective and the view of the AR is much more narrow and doesn’t show that the overlay completely occludes things like your hands which makes the demo very misleading but that wasn’t what I was referring to specifically

        • quidnunc says:

          in case you were actually curious about the latter points:

          here’s a mockup someone made of what it would look like with the current occlusion

          link to

          also how the AR only appears in the narrow field of view

          link to

        • FriendlyFire says:

          What exactly makes you think that the “current occlusion” is some sort of universal constraint? Considering what they’ve already achieved as well as their experience with Kinect, I see nothing in the demo that couldn’t actually have been implemented. It’s not that hard to mask the overlay where the hand is.

  11. AndreiCristian90 says:

    So I guess it’s a matter of time until World of Warcraft is going to be released on virtual reality devices?

  12. Clavus says:

    From the reports a few months back, the HoloLens isn’t really what the camera view makes it out to be. First off, it has a really small FOV. The holograms only take up a small spot in the centre of your vision, it’s not as all-encompassing as the camera makes it out to be. Secondly, they didn’t really fix the problem that all the light is additive, so holograms will be hard to see in bright environments. The camera just overlayed the graphics onto the camera view which isn’t how the actual product does it.

  13. silentdan says:

    You know, dialing the marketing up to 11 doesn’t necessarily create a more effective presentation. The lady keeps her voice in that “cusp of laughter” tone that conveys delight over short intervals, but seems creepy when it’s sustained for several minutes or more. People aren’t that consistently delighted at their own wedding, so it’s hard to credit in a commercial. I get that it’s an important product to MS, and they want to present it to the world as carefully as possible, but they’ve gone so far that it just feels phony. If your presenter is so inexplicably giggly that I begin to suspect the Joker might make an appearance, maybe dial it back a notch.

  14. Ejia says:

    Virtual painting of virtual figures for virtual tabletop games. Make it happen!

  15. racccoon says:

    There ya go!

  16. NemesisZidar says:

    To impress me there is more needed than just something that is the peak of useless nerdyness. I mean, Minecraft in real or simply play freakin Lego. Or…yeah we might be considered being too old playing with Lego? So it suddenly has its target audience because of a taboo in society? Ok.
    As long as they try to advertise such a thing with such a useless game (i have to say im not saying its a useless game at all, i like block building games with killing and crafting, but to play it with a virtual reality device, it is) i am not in the slightest interested in their product.
    Maybe they should choose another good and more fitting game. Its not that i ever thought “yeah it would be so cool if you could play lego, just not in real” but when i thought about VR i wanted to fly Spaceships, Journey across Planets, wander Mazes with Monsters, run away from Zombies or hunt Dinosaurs, control a Tank on a Battlefield or explore Tombs. Not freakin Lego in unreal. Thats like doing a Cleaning Simulator and then buy a VR Device for that, instead of doing it in real life, for the sake of it being virtual.
    Totally uninpressed of that thing till now. I hope they will support better things (games) than that and may use those to advertise their product.

    • wu wei says:

      Just to make sure I’ve got this straight…

      Nerdy & useless: LEGO
      Cool & exciting: “Spaceships, Journey across Planets, wander Mazes with Monsters, run away from Zombies or hunt Dinosaurs, control a Tank on a Battlefield or explore Tombs”

      Is that about it?

  17. Razumen says:

    Wait, can you control the character with a controller, or do you have to use gestures? Seems like a waste if it’s the latter…