What I’d Like To See Happen With Virtual Reality

Using Valve and HTC’s Vive headset was one of the best experiences I’ve ever had with entertainment in any form, but that’s not to say that it’s perfect. There are obvious limitations in the hardware, obvious ways in which it will inevitably be improved in the years to come, and plenty of potential not yet realised in any of the prototypes I’ve played.

So I’ve been thinking. Here’s five (wholly serious) things I’d like to see Valve, HTC, Oculus or really anyone do with virtual reality.

Combine it with procedural generation

One of the first things you’ll do when setting up your Vive is give it the parameters of the space you’re using it within. This is so that you can see a virtual border while playing games – what Valve calls “the chaperone” – to stop you from bumping into walls or furniture.

What I’d like to see is for games to take this information into account and re-structure the virtual spaces players inhabit to fit the actual spaces they’re playing within.

For example, one of the demos I played was for a game called Job Simulator, which re-creates present day jobs through the prism of poor research carried out by robots in the year of 2050. It’s funny and colourful, and it’s a game about the physical objects that make up our daily lives. That includes desks and tables.

So if Valve let me define an area of my room where my desk is, then let games position their in-world tables in the same position. This would allow games to make maximum use of the limited spaces players have in their homes, but also means you’d be able to reach out and feel the virtual walls and floor that surround you.

Clubs and clubbing for outdoor VR

While the minimum space Vive’s laser tracking boxes can be meaningfully used within is apparently six foot by four foot, Valve suggested that there was no upper limit on the space that could be tracked. You could rent a warehouse, chain the boxes together, and run freely through the entire space.

Better still, the tech can also track multiple devices moving through the space at the same time. While none of the demos I played at GDC were multiplayer, Valve said that they were working on prototypes and hoped to have more to show later in the year.

This holds all kinds of potential for dragging VR out of people’s living rooms and into parks, fields and clubs. Imagine frolicking through virtual worlds with a group of friends. Imagine inhabiting a different virtual spaces with a group of friends, but in such a way that you’re all still visible to one another.

Laser tag is about to get pretty hot. LARPing – if you could still call it that – is about to go mainstream. A bunch of people are about to get twisted ankles.

Port the real world into virtual reality

Kinect comes up a lot in conversations about the Vive, because they’re both devices that require a lot of space from your house in order to be used effectively. There are key differences, in that the Vive can be used while sitting down in a box room and so is more flexible, and also is already home to more compelling experiences than the Kinect.

However, the Kinect seems like a decent companion to Valve’s kit. One of the current problems with VR is that, while you feel embodied in the game world like never before, you can’t actually see any part of yourself. If you stretch your arms out in front of yourself you’ll see the floating spectres of your controllers, in whatever form the game has rendered them, but not your hands or arms or sleeves. The Kinect may be able to do something about that, and even if only in crude ways – only in mirrors, say – maybe be able to port some part of you or your room into the virtual world.

Whatever the limitations, I want people to experiment with this stuff, and anything else that can get real world data inside game worlds in real-time. The more reality and virtual reality merge, the greater the potential.

VR headsets in schools

Valve and HTC were shtum about the price of the Vive, but estimates for the Oculus Rift suggested it might be around the $300 mark. That’s relatively inexpensive for a new piece of consumer tech (just look at the Apple Watch), but may be too much for organisations, such as schools, that would need to buy a large number of them.

Yet I desperately want the Vive to be available in as many schools as possible, as soon as possible. The power it has to let people experience otherwise unreachable things, and to teach through doing so, is immense.

I first experienced with Titans of Space, an Oculus Rift demo that takes you through a seat tour of the solar system and beyond. What it’s good at communicating, in a way I’ve never seen, is the scale of different celestial objects. From the earth, to the moon, to Jupiter, to the sun, and to nearby stars, it carries you along an interstellar journey that makes you feel the size differences between objects which – even after books and documentaries and films – had still felt abstract to me.

I experienced it again with the Vive. After being given a demo by Valve, I returned to their booth at GDC to see Job Simulator and asked cheekily if afterwards they would let me see another demo I’d heard about when, the day prior, a game designer told me he started to cry while using it.

This demo streams in live data from Google Earth, which combines satellite imagery with community-provided models in order to render cities and landscapes in 3D. I’ve spent hours poking around the desktop software, visiting places I’ve lived and visited and dreamed of visiting.

It can’t compare to suddenly standing above a city, its skyscrapers reaching your waist, the scene stretching off towards the horizon. I crouched to look between the buildings, and leaned over to peer at street level. And then the world faded to black and back again, and I was surrounded by mountains, peering at tiny fuzzy model trees, the sun setting over the horizon. And then I was in another landscape. And then I was in outerspace, peering at planet earth as a tiny globe, being lit according to the actual factual present position of the sun.

The game designer told me he cried because he imagined using this to show his kids the world. All kids ought to have the same opportunity.

Better lorries

There’s an obvious limitation in the Vive in that its natural movement system – walking in the real world – is tied to the physical space you’re using it within. It’s not clear how this will function for games whose borders aren’t so close, like traditional open world games. I’m told there are prototypes in the works for ways to do that – and at the very least, you could just push a forward button on the controller while standing still – and I’m not generally too worried about it. Games will be built that are designed to suit the strengths and weakness of the Vive, and compared to the Rift, which feels like a cockpit simulator, it feels like those limits are already expanding outwards.

This leads me in one obvious direction. One of the best games to play on the Rift was Euro Truck Simulator 2, because it’s a good game generally, because it’s set in a cockpit, because it’s a pleasant form of tourism. Now that virtual reality can reach beyond the limits of a cockpit and into 15′ x 15′ rooms, it seems time to make Euro Truck Simulator 3: Challenge Anneka, in which you not only drive your truck across Europe, but have a living room in the back of it.

I don’t know what freight you’d carry, but perhaps ill-understood feelings of romance towards tummies of young boys watching at home.

A VR headset that fits on dogs

Because I’m curious.

Thoughts?

This article was first published as part of, and thanks to, The RPS Supporter Program.

47 Comments

  1. It's not me it's you says:

    Huh. I nearly didn’t click on this because literally every other VR ‘think piece’ anywhere has been useless and under informed blather but all of those are actually really damn excellent ideas.

    The concept of setting up something in a large shared space (think lasertag or LARP in terms of how involved physical setup could be, though it wouldn’t necessarily need to be a team based adversarial activity) could be super amazing.

    Imagine playing astronauts on an unexplored planet, with RL props strewn about matched to in-game bobbins for your to discover, theorise about and explain. Man that’d be wicked.

  2. Chaz says:

    All the above, and I’m sure it’ll be a novel way to watch porn in the future too.

    • DwarfJuggler says:

      Porn is the great cause to innovate technology. It’s why streaming speeds are so much faster now :P Could be what makes VR more bountiful in the future.

    • Razumen says:

      Unfortunately,(or fortunately, depending on your mindset), replace future with present, lol.

  3. Premium User Badge

    tigerfort says:

    There’s an interesting looking kickstarter running at present for a VR headset designed for underwater use. I can see that having a range of potential applications, through allowing you to explore a virtual representation of something in deep water without endangering either yourself or it. Tours of virtualised coral reefs and famous shipwrecks, yes, but also real-time “in person” exploration (via a drone programmed to match the diver’s movements) of, say, underwater volcanoes or damaged oil rigs.

    (I have no connection to the KS, and don’t currently plan to back it.)

    • Nalum says:

      Looks interesting alright, unfortunately it was cancelled.

      • Premium User Badge

        Harlander says:

        Shame. VR drone telepresence is the sci-fi sauce I want in my life.

    • Razumen says:

      I’m sure someone will develop something similar for drones, would be exhilarating to (almost) literally fly around the city you live in.

      • Press X to Gary Busey says:

        It was the first thing some people did with the OR-DK1; Quadcopter + camera.
        The demos I saw were kind of pointlessly basic though, just a regular 2D camera feed like you could already get with a laptop.
        I’m sure it’s already being worked on though, VR face-set feeds from drone systems with stereoscopics and wide angle panoramic cameras. I’d be surprised if it’s not already available in newer military drone systems too.

  4. k-pound says:

    Something I’m particularly interested in is the idea of home movie release type ideas. If I could pay to watch a movie (or a movie quality experience) on my headset, I’d do it. VR also has the potential to solve the problems of character perspective and loss of inner thoughts/motives that movies have faced for decades. If we need to hear a characters thoughts or see through their eyes we simply ghost into their head. It certainly be weird at first looking out of someone eyes and hearing thoughts narrated. It would require good writing and good voice acting, but the potential is their.

    • k-pound says:

      Man, I need to proof read when posting from my phone.

    • Aninhumer says:

      The problem with movies in VR, is that you need to be able to support varying viewpoints on the scene. Stereoscopic 3D only gives you two fixed perspectives, one for each eye, but with VR you’re able to move your head around to see behind things. This would be possible for animated movies (although you’d have to render them on the fly, which might not look quite as good), but I don’t believe we have any technology to make this work for live action yet. It might be possible with careful staging and some kind of interpolation, but I’m not hopeful, as VR is very susceptible to uncanny valley visual problems, because of how close to normal vision it is.

      Looking out of a characters eyes might be an interesting cinematic effect, but I’m not really sure how it solves the inner thoughts problem though? It’s still just an inner monologue, which we can already do. Also, you have to be careful moving the viewpoint independent of the viewer’s head, otherwise you’ll give them motion sickness.

      • k-pound says:

        There are obviously limitations as well as new ground to be discovered. The problems of motion sickness and current cinema techniques will have to be figured out over many years. I was thinking of animation in my original post, however. The tech being used at Pixar, the proprietary software Presto, is a glimpse into not-so-far-off real time rendering capabilities. See

        at=67
        And as far as thoughts are concerned, think about many big budget movies. A disembodied ethereal voice speaking the character’s thoughts is the most common device used, but this technique is completely inappropriate for use during dialogue or action sequences. Any direct expressions of character thought is either completely untouched (See Star trek, Guardians of the Galaxy, Star Wars, The Hunger Games, Planet of the Apes, the Hobbit… basically any modern mass appeal movie) or it is relegated to a very specific context like reading a letter or a dream. Band of Brothers, for example, relies heavily on facial expression and character dialogue to convey thoughts. That mini-series did an amazing job with that tool and I wouldn’t change it.
        But think of the possibilities, the emotional impact a war movie could have if we can hear the stream of consciousness of a scared soldier in a fox hole. I’m not saying all or even most productions should use this idea, but it could be a new tool. Audiobooks are the best highly produced example of such a tool. If we could couple the visual aspects of cinema, with the narrative prowess of literature some truly amazing things would be possible.

        • Aninhumer says:

          > A disembodied ethereal voice speaking the character’s thoughts is the most common device used, but this technique is completely inappropriate for use during dialogue or action sequences.

          Well yeah, my point is that this doesn’t really change just because the perspective is a little more immersive. The fundamental problem is that you can’t easily process two things at once. VR is almost entirely orthogonal to that.

          >the emotional impact a war movie could have if we can hear the stream of consciousness of a scared soldier in a fox hole.

          Again, there’s not much stopping us doing that now. The problem is that thoughts aren’t really the same as an internal voice, so it doesn’t work that well. This is why internal monologues tend to be presented as narration to the viewer, not raw thought.

          • k-pound says:

            You could be right. Either way, the area will probably be explored in the coming future. If it can’t work, then I guess I’ll have to wait for the holodeck :p
            It’s such a new medium that we don’t know what will work well. I’m simply expressing an idea that *may* work well, one that I am excited for, one that I would like to see explored (which I’d what this article is about).

      • Razumen says:

        It also would require filmakers to completely reconsider the concept of Mise-en-scène. Films have all but mastered this using only one point of view, but allowing multiple, or even unlimited viewpoints into the mix would require a drastic overhaul on how to present movies to a virtual viewer.

        • k-pound says:

          That’s fine with me. A difficult art form to (re)learn perhaps, but it may be worth it. We’ll have to wait and see. Regardless of what this vr tech does to film, pushing into unexplored areas of art is basically always a good thing in my mind.

    • Vandelay says:

      Theatre would be a better fit. The simplest way would be to just record a performance from the auditorium and it would be a stationary experience for the viewer. Record in 360° and you would then be able to look around the theatre too. No idea, whether we have the tech to completely render the whole auditorium and allow you to get up on the stage though, unless the whole thing was computer generated and not live action.

      A while ago, I heard that Sky had commissioned a 360° David Attenborough documentary that would be usable on the Rift. Not sure if anything came of that.

      • k-pound says:

        The streaming/virtual involvement possibility is limitless if this tech goes main stream. Theater is a great idea! I hadn’t thought of that one. Sporting events, tactical training, medical therapy, tours, etc. Obviously not all in the near future, but in fifty years? You could buy a vr ticket to the super bowl.

      • Razumen says:

        Maybe with improvement in 3D scanning techniques we’ll eventually be able to record whole performances in 3D and view them back as if we were there with a headset. Combine the recorded actors with 3D generated backgrounds that change with each scene, and it could revolutionize live theater.

  5. crazyd says:

    I’m one of the boring people who isn’t interested in the motion tracking capabilities, and just wants traditional games to be more immersive. Anything more than headtracking seems like a lame gimmick that won’t hold up. Motion tracking has been nothing but a failure in the past, I’m not sure why people seem to think it’ll work out now.

    • Kefren says:

      I suppose one possible advantage is that instead of four hours sat in a chair playing a game, I would be moving around more, which would be beneficial to health over such sedentary shenanigans. Not sure if that is any part of the thinking though.

      • goosnargh says:

        This aspect could be huge. Like crossing physical activity off the list “How not to arrive at future depicted in Wall-E”.

    • rowan_u says:

      Those of us that have the hardware already know its working :) I’d like to see VR catch on in a mainstream way of course, but it’s not something I’m super worried about. We are going to have headset now, even if they are super niche like HOTAS controllers, or Motion Simulators. And that’s really all I wanted when I backed Oculus.

      • crazyd says:

        I’m sure it works great for custom designed games made to function in a single small environment, but I don’t really care all that much about those games, and am doubtful that they will blow up, just like how motion games have failed to do so in the past. I just want traditional game types where I sit down to be more immersive. Give me Elite: Dangerous or Dying Light with a VR headset, and I’m excited. Give me Job Simulator, and I’m yawning after 5 minutes of novelty. I don’t want my games limited to my physical environment.

    • Razumen says:

      Maybe because everyone that has tried it has been blown away? I don’t think every game will necessarily benefit or need the player to physically walk around a room, but that’s the great thing with the VIVE, is it is viable for both seated and standing experiences. I mean, just playing a standard game with the headset would allow you to naturally peak around corners, or track moving objects more naturally, not to mention more natural aiming with the controllers (imagine aiming through ironsights in game by having to actually physically move the weapon up into your line of view, rather than hitting an arbitrary button)

      Previous motion controllers have been rather half assed (Wii) or actually really good but neglected because they were an optional accessory (Playstation Move). It’s the same for PC, motion controllers exist, but without a decent VR headset, they were basically a detached experience using them on a 2D monitor.

      • crazyd says:

        People were blown away by the Wii when it was introduced. I agree, it’ll be cool for a bit for a small set of very specific game types, but the next Battlefield isn’t going to take place in one small room to take advantage of it. I mean, everything that’s been shown has been simple minigame type stuff. I’m certainly interested in Valve’s VR, I just don’t see much application for the motion control aspect outside of a small number of gimicky puzzle games. Maybe I’m wrong, guess we’ll see.

    • Kamek says:

      I’ve used the Vive, and the motion controls didn’t feel like a “gimmick”. Quite the opposite – they felt fundamental.

      I think comparisons to older motion-control games are totally moot. In Wii-style games, your movements are ‘translated’ onto a flat image on a screen. That translation is always confusing and weird. In contrast, the Vive’s hand-tracked controllers BECOME your hands, as far as your brain is concerned.

      I expect hand-tracked controllers to become the ‘natural’ interface for VR, in the way that the mouse is the ‘natural’ interface for the desktop.

    • Press X to Gary Busey says:

      I’m another one of the boring people who just don’t want to drag meatspace into my games. I want the free-looking and not much else.
      One of my main genres, roleplaying games wouldn’t even work with full-body motion controls; In every game I would just be playing as myself LARP’ing and not as a character with vastly different physical abilities.
      A first-person fantasy sandbox RPG as an example:

      Forget fancy combat moves or acrobatics unless I can do them IRL. Every move/ability my character can make is limited by my own body. Weird weightless weapon swinging (to match arm movement), plus I don’t know any melee combat techniques.
      No menus. Remember all spell gestures/words. No “inventory”. Interact with NPC’s by physically speaking to them (not even technically viable), killing any sense of roleplaying as another person.
      Walking/running/jumping around a huge world thousands of times larger than my living room, and not flat. Forget horses or riding any kind of vehicle. Even taking a step that’s impossible in the game, e.g. into a wall is another illusion killer, or climbing stairs for that matter.

      Just a few of the million issues I wouldn’t see by just playing with kb+m or a gamepad with VR to replace the mouse/right stick.

      • Premium User Badge

        Harlander says:

        Not everything in there is unsurmountable even with current technology, and frankly, even the failures would be interesting.

        I’d call menus hovering in your field of vision an acceptable break from reality in some cases (the system menu where you quit the game for one thing). An inventory wholly managed by manipulating in-game physics objects would be pretty interesting, as would gesture-based spells you have to look up in a physical spellbook if you forget ’em.

        Interact with NPC’s by physically speaking to them, killing any sense of roleplaying as another person.

        I wonder if piping an altered version of your own voice back into your ears would help make you feel more like someone else…

        • Press X to Gary Busey says:

          A sci-fi setting would probably be an easy way around a lot of my “problems” and an actual inventory to manipulate is something I’ve wanted to see since long before VR. :)

          I’d love to see someone try to design around to all the issues too, even if they would fail in the end.
          What I wrote was entirely inside The Old Moldy Box thinking regarding a very traditional genre. Working full body motion tracking is an exciting technology but I think game designers will have to come up with new kinds of games in parallel to the old rather than shoehorn full motion controls into old genres, or at least experiment with partial implementations.
          I’ll wait and see. I hate to sound like a technophobe.

      • crazyd says:

        Phew, I’m not alone! It’s weird to feel like the only person who sees how limited this is in it’s scale.

  6. Gil-estel says:

    In a previous article here on RPS, the writer mentioned a VR painting demo. I think it could be amazing to create something in VR and then export it to a 3d printing machine and have it spit out your creation (although probably at a smaller scale).

    • DantronLesotho says:

      ^^^this; I think that would be great. You could really craft your own weapons and armor for LARPing once 3d printers get big, cheap, and sturdy enough for it. Or build sculptures for a garden or something. The possibilities!

  7. Turkey says:

    I’d like to see someone make Virtual Bart a virtual reality.

  8. DantronLesotho says:

    I want an Uatu simulator. I can watch cosmic events happen in a comic book world as they happen.

  9. Vandelay says:

    I think that museums could put this tech to great use. You’ve seen artifacts from Tutankhamun’s tomb, now actually visit the tomb itself as it would have looked! Stand beside Martin Luther King as he delivers his ‘I Have a Dream’ speech! Sit with Van Gogh as he paints his self portrait and he tells you about his influences!

    This could really bring history and culture to live for so many people, particularly kids.

  10. dustin says:

    So, with all the new VR whiz-bang coming out, and RPS obviously being rather excited about it, is the “PC Gaming” title going to become more flexible? Does Gear VR count as COVERABLE under these evil draconian standards, set by the blood-lust RPS editors? If people are prancing around in the headsets in a field, is it RPS-coverable if the rendering is done on a PC somewhere nearby (ideally, by gently-hovering drone (an idea we’ve discussed around the water cooler here) ).

  11. Razumen says:

    Has anyone played Space Engine with a VR headset? That seems like a prime way to really experience the scale of the universe. Approaching a black hole would become an even more nerve-wracking experience.

  12. MountAndGames says:

    The idea of motion tracking seems horrifying to my (entirely uninformed, didn’t know it was a thing beyond head tracking before I read this) perspective. Yes, being able to set the borders of the room and where the furniture is located helps, but it seems we have a choice between having to overlay games onto the real room around you, losing most of the fun of immersing in another world, or else place gamers at significant risk. The idea of using large spaces (combined with general knowledge of games) implies high activity on the player’s part, and I think people are underestimating how much we use our eyes to judge the world around us. It doesn’t take a large irregularity in the surface of the floor to put you on the deck if you can’t see it and are distracted – the 2 major characteristics of this type of play. What about if other people decide to walk through the area or think a functionally blind gamer is a fun target for practical jokes? What about animals, they aren’t going to be able to tell that you’re not seeing the world as it is and WILL get underfoot. This all seems terribly underthought.

    • Premium User Badge

      Harlander says:

      It’s daydreaming, not a business case.

      • Caerphoto says:

        It honestly saddens me how some people can be so negative and pessimistic. You see it in the comments on any new technology, things like “but what if this happens? what if [unlikely scenario]?”, “it’ll never work because x y z”, “this is pointless”, “have the designers thought about [obvious issue]?” etc.

        WHERE IS YOUR IMAGINATION AND HOPE, PEOPLE?

  13. LukeW says:

    I teach a gaming/programming class to a bunch of high school students in Tas and one of the things we cover is VR. We have a class Oculus DK2 and I’m always trying to come up with new experiences for the kids on it. The level of engagement you can get from a kid when you strap a headset to them is pretty amazing and, after 10 years of teaching, there’s nothing that I could compare it to.

    Having said that, the limitations at the moment are pretty tough to work around. Only one student can use it at a time and the amount of effort to get programs running on it means that most teachers wouldn’t bother. I do love the idea of strapping headsets to 30 kids and taking them to historical events throughout history as directed by the Curriculum.

    While most of my teaching around the OR comes from the student’s Unity programming (which works fairly well on headsets and allows students to write programs directly to it [we’re now moving to UE4 too]), one benefit of having it sitting there is that it’s a great motivator. Currently in my classes I have a copy of Project Cars, a G27 and the OR setup. If the students have been working hard that lesson I invite them to come up and try to set a time on one of the tracks. They then get to have their name put up on the leader board Top Gear style. I find that even kids who are reluctant to engage with classroom learning will do their work if it means they get to drive a racecar for a few minutes.

  14. gscr says:

    Awesome article posting.I found your website,it is remarkable.I found great articles on your site.Thanks for this great work and posts.halo games

  15. guygodbois00 says:

    Job Simulator? Thanks, already playing one right now.

  16. MageJohn says:

    What I want to see come to VR is actually a strategy game, such as Civilization V. This might seem a crazy idea, but hear me out. With Civ V, for example, imagine standing in a room in you’re palace, in front of a table. On the table is a map, a little like the Stategy View in Civ V, but actually on paper. Or maybe it’s a sort of holographic projection of the world you see in normal Civ. You can walk around, inspect it from different angles, and of course move troops and play like you normally would. You could actually have your advisors in the room with you, and imagine going up to the window, and see your city spread out below you!

    Like was suggested in the article, you could mark where a table was physically in the real room, and use that as your strategy table in game.

    Or what about a space RTS? Standing in a ships control room, surrounded by holographic displays and maps, directing your ships into battle, recieving important reports directly from crew members.

    The advantage to this is that you have a really credible excuse as to why your keeping the player in the same room, whithout artificially constricting them in-game.