How Green Is My Uncanny Valley

I'm willing to admit this might be a niche issue to have

I have The Vanishing Of Ethan Carter installed on my PC and ready to go. But there’s something that’s been playing on my mind regarding that game before I’ve even booted it up. It’s been nagging at me ever since I watched a video from Andy Kelly’s Other Places series – the one which focuses on Ethan Carter’s Red Creek Valley – and it finally crystallised a problem I’ve been experiencing for years without being able to put it into words.

Just after a shot of a dam there’s a lingering shot of a churchyard. In the foreground a handless statue of Jesus marks the grave of a woman named Thusnelda. In the background the autumn trees sway in the breeze and the weed-infested grass – well, I want to say that it sways but it’s a sway which comes via a clump-by-clump waggle. That grass is why I’m proposing there exists a foliage version of the uncanny valley.

The uncanny valley is the idea that our comfort level around representations of human figures suddenly dips as they approach realism. The idea is that we are unsettled by things that are almost-but-not-quite-exactly-human. But perhaps it doesn’t have to be limited to humans or animal. Perhaps it can be extended to plants. The uncanny grove? The uncanny vegetable patch? The uncanny meadow? Hell, maybe even a *literal* uncanny valley.

To offer a bit of backstory; in the last year I’ve become very aware of how trees and grass move in the wind when it comes to videogames. It came about because I finally decided to get into the Skyrim modding scene. But with the power to change anything in the world I didn’t like or felt could be improved, Skyrim became my own personal dollhouse. Instead of a game it was a project. Screw that main dude and his beef with dragons – what about the important things like the colour or texture of the ice floes?

I began to explore the environs of Tamriel with a notepad in hand, writing down anything I felt could be improved to better match my ideal vision of the world. Obviously this is a terrible idea and you end up thinking of the game more as a tableau – something to be preserved in aspic – and not anything resembling a living world. Regardless, I was unsuccessful and that was entirely because of the trees. They never looked quite right.

I spent ages going through mods trying to find one which felt authentically tree-ish and became so fed up by my failure that I never play Skyrim anymore. It’s a game I have come to regret having access to on PC because having the power to tweak it combined with my own aesthetic peculiarities in a way which ruined an otherwise-fantastic game.

Watching the grass moving in a Red Creek Valley graveyard I was struck by a familiar feeling of discontent. But the thing is, I’ve read a lot of the blog entries by the creators of The Vanishing Of Ethan Carter, The Astronauts. I know the lengths they went to in creating the environments of the game. It seemed odd to me that grass would have escaped their attention so I dug back through their archives.

One of the techniques they used was photogrammetry. It involves going out to photograph real-world objects in situ then using software to convert those photos into 3D art assets. These are then subject to compression processes (otherwise a single rock can take up an entire game’s polygon budget). After that the scanned assets are mixed with traditionally constructed game artwork and subject to stylised lighting and processing.

The goal of all this is not to achieve photorealism. The intention behind the work and the field trips is that The Astronauts never want your brain to become distracted by things like texture tiling. As they say on their blog, “It may look photorealistic to some, it may seem magical to others, but if we did our scanning right, it should above all, feel right.”

Applying the same mentality to the foliage proved a formidable task. Graphic artist Adam Bryla freely admits that. “Rendering subtlety, richness and dynamism of vast, wind-torn foliage by using just a bunch of flat, static triangles? One heck of a challenge.”

In fact, Michal Kosieradzki who specialises in animation and particle effects spent all of 2013 working almost exclusively on foliage. According to Bryla, “When tree branches are simplified to just a few flat planes, you get large batches of leaves facing the light the same way, so again, we had to get real crafty. We distorted the geometry information to trick the lighting engine into thinking that those flat triangles are not flat, and we added a smart shader that mimics how leaves overshadow each other. We even added a shader that mimics how each leaf changes its orientation towards the sun as it trembles in the wind.”

I should say at this juncture that I have no problem with the trees – the trees are gorgeous from the footage I’ve seen so far. It was the grass which triggered the feeling of slight discomfort.

On the grass front Bryla notes that “[Kosieradzki] went through almost every imaginable method, from billboards (flat grass planes that always rotate to face you, so you never see the flatness), all the way to painting the grass using particle effects.” From this it seems as though the team had a similar general approach to grass as they had to all the other aspects of the game, certainly in terms of the basic modelling.

In reading about the uncanny valley phenomenon two things struck me. One (which I fully intend to gloss over here as quickly as possible…) is that the uncanny valley might not even exist and thus extending it to flora is simply a way of dressing up my own aesthetic peccadilloes as potentially scientifically interesting. The other is that the look of the thing isn’t actually the main trigger for discomfort, it’s more likely to be disharmony between the look of the thing and other aspects such as how it moves or feels. The reasoning for that last one is that static art assets are often a little ahead in terms of technical sophistication than, for example, rendering 3D motion. As a result something can look lifelike, but then move in a less than lifelike manner – we recoil because the object didn’t behave as expected.

If I had to pick a reason for my own discomfort (a reason that isn’t the one where the uncanny valley doesn’t exist) I’d say the most likely candidate is that there’s a slight mismatch between the rendering of the grass blades and the rendering of their movement. It’s such a tiny thing, and one which I know the team involved put a lot of work into resolving. I also only became aware of it because I was watching a video rather than playing the game. But having previously equated the uncanny valley exclusively with representations of people I’m inclined to extend the definition. I think the uncanny valley could actually apply to all things affected by natural movement.

This article was originally published as part of, and thanks to, the RPS Supporter program.

48 Comments

  1. thedosbox says:

    The other is that the look of the thing isn’t actually the main trigger for discomfort, it’s more likely to be disharmony between the look of the thing and other aspects such as how it moves

    I find myself distracted when clothing doesn’t move realistically. Batman’s cape in Arkham Asylum was a particularly egregious example.

    And yet, I’m fine with non-animated clothing…

    • Diziet Sma says:

      I agree, but the thing with Batman’s cloak never occurred to me until you pointed it out. I think I just mentally put it down to “Hell it’s Batman’s Cloak”, like that thing lets him glide and he can hit people with it so it’s bound to move a bit funny.

    • soulblur says:

      Metal armour which bends also distracts me. Perhaps it’s difficult to model characters which move different depending on the type of armour they’re wearing.

    • SRTie4k says:

      This comment actually reminded me about the most impressive cloth physics I’ve seen to date in a game. I just started playing Metro: Last Light, and just the other night I got to the part where the Blacksmith unveils the railcar to Artyom. I could not believe how incredibly realistic that animation of him pulling the drape off the railcar was, everything moved perfectly, there was no clipping or weird physics, it just simply unfurled and fell to the ground in a heap the way a real sheet would.

      After so many games where the cloth physics are downright laughable, I was in utter and complete awe at such a simple display of perfect cloth physics.

      • Rig says:

        That’s because it was prerendered. Still looks great, though.

        • Cvnk says:

          link to youtu.be

          Doesn’t look pre-rendered to me. Perhaps pre-calculated but it’s clearly not a typical per-rendered cutscene in the traditional sense of that term.

          • LaurieCheers says:

            Yeah, pre-rendered was the wrong term – he meant pre-baked.

            However, looking at the cloth in the video, there are definitely some glitches (especially if you watch how it clips into itself as it lies on the floor). So I don’t think they’ve even pre-baked it – I think this is just a regular cloth simulation.

          • SRTie4k says:

            I actually watched that video earlier today and it definitely does not reflect the PC version. I would guess the PS3 version doesn’t use the same physics engine that the PC does. The PC version of that scene was sooo much more fluid than that.

      • Canadave says:

        Oddly enough, some of the best cloth physics I can recall seeing in a game dates back to The Legend of Zelda: Wind Waker. It was a lot of little stuff, like how Link’s cap moves as he walks, or the way that the boat’s sail catches the wind. The strips of cloth on the moblin’s pike-thing is also stupidly impressive to me, even now: link to youtube.com

  2. Premium User Badge

    Wisq says:

    To me, the grass looks like underwater weeds rather than grass in the wind. Partly because they move so fluidly, and partly because multiple patches of grass are moving in different directions at the same time.

    I don’t commune with nature nearly as much as I ought to, but in my experience, a) grass tends to stay pretty much upright unless exposed to a decent wind; b) wind tends to either blow most of the grass the same way, or else travel through it in distinct eddies; and c) the movements tend to be sharper, more jittery, and it’s more quick to return to the original shape once done. Branches can bounce around because they’re giant springs, but every blade of grass is firmly rooted in the ground.

    That said, the vistas are lovely and it’s only in these standing-still videos that things get a bit weird. So, a job well done overall.

    • Rizlar says:

      Yes! Everything seems to bounce around too much (the trees as well, I would say). It looks like they are about to burst into song and sprout cartoon faces.

      Realistically there would be more varied degrees of movement, like you say. Trees might be virtually static, with only a few twitches of leaves and some movement in the top branches, until a gust of wind comes along and branches bend, leaves blown to one side, before springing jauntily back into shape with a whole host of different movements.

      I wonder if the slight exaggeration of movements is necessary while playing the game, small movements being less noticeable when you are running around the environment with your attention elsewhere. It definitely seems like the uncanny valley isn’t just limited to depictions of humans anyway.

      • amateurviking says:

        Was about to post exactly this. Vegetation just doesn’t move that much unless it’s very windy/blustery, grass especially as air moves more slowly as you approach the ground.

        I imagine it can be hard to resist the temptation to animate everything all the time (especially as I imagine it is computationally less expensive than the occasional random twitch).

        • Zunt says:

          I wonder if it’s to do with the brain’s powerful ability to detect patterns. Naturally wind tossed branches move in a mathematically chaotic way, so no discernible patterns. The animator’s grass and branches move in deterministic loops, which trigger our pattern recognisers.

          • jezcentral says:

            Very probably. The point being that grass and shrubs and what-not don’t move in the regular way that they do in PC games (like a rubbery metronome). We need more irregularity, but I should think programming this kind chaos is hard for game engines to deal with.

      • Premium User Badge

        Wisq says:

        I wonder if the slight exaggeration of movements is necessary while playing the game, small movements being less noticeable when you are running around the environment with your attention elsewhere.

        Yeah, probably this. It’s way more visually impressive to see everything moving unrealistically than to never notice that anything moves at all because they made it appropriately subtle.

        Even for a game like Ethan Carter or Skyrim, it’s rare for people to actually walk places, much less to ever sit down and appreciate nature. I tried to walk my way through Ethan, but past the first couple areas when “where do I go next?” stopped being so obvious and I started having to backtrack, I began running everywhere because “I don’t have time for all this walking” etc.

        I wonder if we’ll ever cross the valley into vistas and environments so realistic (and with enough tiny, subtle dynamic aspects) that you actually just want to stop and sit down and watch for a few minutes. The “one city block” approach to modelling nature?

      • doomlaser says:

        Ah yes, the Overzealous “Tacoma Narrows” foilage..

    • VelvetFistIronGlove says:

      Chiming in to say that the excessive and excessively wavy motion of the grass bothered me as well.

      But the grass wasn’t the main aspect of Ethan Carter’s landscapes that felt unreal to me: the lighting was. Despite the gorgeous and real-feeling models and textures, the lighting still feels wrong. Perhaps I should lay that at the feet of UE3 and its lack of GI?

      Also bothering me was the distant vistas of the far side of the lake—the distant trees and mist felt all wrong.

    • ubik says:

      Wow, that’s right. I remember thinking “very cool, but it’s all moving too much, and too regularly.” But yes, put it all underwater and with a bit of current it would look spot on. The sort of tech that Rapture could use.

    • Premium User Badge

      Dukey says:

      Some of the best grass I’ve seen recently was actually in the technical Alpha for Kingdom Come: Deliverance. It still moves too much, but standing there staring at a field of grass I could actually see gusts of wind blowing across the field and the grass/flowers/shrubs all suddenly moving in the same direction. It was actually quite mesmerising. The problem then was that they’d clearly put so much effort into this effect that it made it painfully obvious that the bigger trees weren’t behaving the same way at all, just randomly blowing around.

  3. Premium User Badge

    Earl-Grey says:

    When most of my brain has melted away and my frail, decrepit old cadaver is being wheeled around the Last Resort home for the old and terminally insane by a heinous, fascistic nurse with cigarette-stained fingers, the little grey matter that still set of the right neurons through the correct synapses will be routed so that until the day I finally shuffle off this mortal coil I will continue to shout at everyone misfortunate enough to encouter me that the pursuit of photorealism in games is a pointless struggle.

    • ubik says:

      I have to disagree here. Unachievable, you could possibly argue, but pointless? Ethan Carter presented a very rich and interesting environment (which it could have done more/better with, but whatever) and I don’t consider it wasted effort. Not all games need to have astounding graphics, but that doesn’t mean that it’s not nice to have a few that do.

      • Premium User Badge

        Earl-Grey says:

        Yes, well, I could have said unachievable.
        But then what would I do with all this hyperbole I’ve got lying around?
        I’ve got tons! Millions upon billions of tons!
        -See?! I can’t control it.

  4. Wowbagger says:

    Soft on the inside, crunchy on the outside peccadilloes!

  5. machinaexdeus says:

    I was wondering if it is to do with prediction more than graphics. Your brain is predicting all the time what is going on around it. Most of our vision is filled in by our brain predicting what is going on rather than actually seeing as such. Therefore, if something looks natural the brain will know how it should behave, if it doesn’t you get an odd feeling, brain logic is broken, extra power required to process this new image that is different in a way it shouldn’t be. Maybe that is the uncanny valley and therefore applies to all natural things.

    • Gap Gen says:

      One thing I like about Minecraft is that the systems are very simplistic, but they behave in ways that make some kind of sense – chickens drop eggs, eggs make baby chickens, sheep drop wool if you shear them, etc. I suppose it’s on the far side of the uncanny valley still; the fidelity isn’t such that we expect realistic physical behaviours, just ones that approximate real behaviours at the same level as the graphics.

      Similarly, this is possibly why high-res textures on low-res polys look bad – the realism is mismatched, so our brain has trouble picking where to draw the line in terms of suspension of disbelief.

    • LTK says:

      I almost didn’t realise that you said virtually the exact same thing that I did one post below. Shame on me.

    • arisian says:

      Another issue here is that what you consider “natural” is based largely on what you’ve been exposed to (and there’s actually some science to back this up, in early developmental psychology and neurology). When the very first motion-pictures were developed, they had to limit the framerate to 24fps so that people unaccustomed to the technology didn’t feel it was “uncanny” (or, alternately, get motion sickness). For those of us who’ve been exposed to video imagery since childhood, this isn’t a problem; we can watch 120fps video without any problems (though there’s still an issue of whether it matches what we’ve learned to expect, which is why some peolpe think 24fps looks more “cinematic”).

      With computer graphics, things that 10 or 20 years ago might have seemed “uncanny” may now be common enough that they don’t bother us as much. The gradual nature of the improvement of computer graphics can actually help here, because it means that you can get used to changes a little at a time, rather than suddenly being presented with something that looks almost-but-not-quite-enough like something familiar. That helps give the brain time to build a proper model of the thing-on-the-screen, so it isn’t stuck trying to use a model of the thing-in-the-world that the thing-on-the-screen is an (abstracted) artistic representation of.

  6. LTK says:

    I think the obsession with making things look real is vastly inferior in capability to the obsession with being able to tell whether something looks real. Humans all have a sense of how the world works through appearance and movement (though some more than others) which is crucial for predicting near-future events, such as, is that car going to hit me if I cross the street, is this cup going to slip off the table, or can I stand on this chair without it falling over? We get this information from our constant reality-testing, making unconscious predictions about whether something will move or stop, and adjusting our expectations accordingly.

    This is why moving foliage or character animations never look real to us is because we all intrinsically know what the real thing looks like, and on an unconscious level that sense of unreality is always present. So maybe we start to pay extra attention when something looks almost real, because our reality-testing doesn’t just dismiss it quickly as unreal, but has to do more integration to come to a conclusion. This switcheroo might be a bit unpleasant on a very basic level: “Hey, I thought that this might be real but it wasn’t, you tricked me!” If our reality-testing is constantly being taxed on a higher level in near-photorealistic situations, I can imagine how it would produce the feeling of unease that they call the uncanny valley.

    • DantronLesotho says:

      I don’t know, I think a lot of game designers have gotten really close to getting realistic. The movement of the objects in the Team Ico games is one thing that strikes me, as well as the textures in Half Life 2 and faces in LA Noire for example. It just needs the proper art director to find a way to pull the veil over our eyes. I think part of the reason why games fall into this valley is because it was born from 3d rendered movies, which in the early 90s operated on a frame-by-frame basis so our brains didn’t have to interpolate their movement like it would for film or even in real life. Now people have a much better idea of how to play on the brain’s foibles to make a more realistic effect, and I would wager that within 10 years we won’t be able to tell the difference. Even now, with the MMA and other sports games, there are some times you’ll see the footage and have to do a double take.

  7. Wowbagger says:

    I think RPS is the only website I go to where the comments are regularly as interesting as the posts themselves. High fives for everyone!

  8. Kempston Wiggler says:

    Coincidentally, this has been bothering me during my time in Planet Explorers. I’m playing the single-player campaign in which the biomes are large and, although skilfully hand crafted in terms of topology and ‘character’, are all absolutely covered in the type of grass you describe: clumps of sprites moving as if to simulate wind effects but ‘Uncanny Valley’ enough to tickle my awareness of them as being wrong. Ironically, the trees/forests in Planet Explorers are low-poly enough for my brain to accept them as purely game constructs that I’m just not as bothered by. I think it’s why games like Eidolon have always been so appealing to me: stylistic approaches can represent nature without that bothersome sense of wrongness – of digital unreality – disrupting immersion.

  9. Viroso says:

    It’s about abstraction vs realism.

    When you abstract, you leave out many pieces of the puzzle. The audience will fill in those pieces themselves. Their brain is used as the graphics card.

    link to presentation-process.com

    So the audience will abstract. You’ll see this picture and think the blue shape is a circle. The brown part is what you left out, what you obstructed.

    It gets ugly when you try to show everything. Since there’s just not enough computer power for photorealism, this is what you’ll create attempting to draw a circle

    link to tinyurl.com

    The audience can’t abstract anymore, they can’t look at one part and use it as a stepping stone to picture the rest. You’ve already shown everything.

    The uncanny valley is lack of abstraction. As your representation approaches realism, every missing piece is an wider gap. You can apply it to everything, like you mentioned with grass.

    Think Fallout New Vegas. Didn’t Nevada seem too small? Awkwardly small. You could walk anywhere within minutes. Rival groups existed 10 meters away from each other, their war wasn’t very convincing.

    Compare it to some old NES RPG. Everybody’s tiny, so are their houses and cities and there’s a separation between exploring cities and exploring the world. In the world map, you were about the size of a town.

    Which one feels weirder, Nathan Drake getting shot or Doom guy getting blasted by demonic fireballs and running over a medkit?

    Even a book adaptation to theaters. That actor they chose just doesn’t feel right.

  10. norfolk says:

    I wonder how, if at all, the uncanny valley phenomena is related to prosopagnosia (or its opposite, Capgras Syndrome).

    For those who aren’t acquainted with the details of these two remarkable disorders:

    “Prosopagnosia, also called face blindness, is a cognitive disorder of face perception where the ability to recognize faces is impaired, while other aspects of visual processing (e.g., object discrimination) and intellectual functioning (e.g., decision making) remain intact.”

    “The Capgras delusion (or Capgras syndrome) is a disorder in which a person holds a delusion that a friend, spouse, parent, or other close family member (or pet) has been replaced by an identical-looking impostor.”

    In other words, our neuro recognitions centers work on both an aesthetic and intuitive level. People get messed up when one of those centers is disrupted.

  11. Dale Winton says:

    The game Betrayer that was out last year had the best “grass blowing in the wind” effect I’ve seen in a game. Looks a lot like sykrim. Pity the game was a bit rubbish

  12. Kollega says:

    I have an anecdotal story that proves, at least to me, that uncanny valley is not limited even to visual effects. In Battlefield 3, there are a few missions where you play as the members of GRU special forces, and they speak to eachother in Russian – and the thing is, apparently one of them is voiced by a native Russian speaker who consistently gets things right, and another is voiced by someone who seems to only have a very basic grasp of the Russian language and just reads from the script with no regard for proper pronounciation or subtleties of grammar. And to a native Russian speaker like myself, the end result is consistently jarring.

    Bottom line: uncanny valley can be applied to a lot of things. It’s a scientific fact!

    • Kempston Wiggler says:

      This must affect every native language speaker at some point. For years my friends and I have made a sport of judging poorly-rendered Scottish accents in TV and Film, like this beautiful piece of nails-on-chalk from Stargate Atlantis, Dr Carson Beckett:

      Youtube Link

      His mid-series death was a mercy on our poor ears (and our eyes, considering they replaced him with Jewel Staite)

  13. Monggerel says:

    That’s what I always say: the only good tree is a charred stump.

  14. Ross Angus says:

    I believe this is relevant:

  15. ikehaiku says:

    Funny, I actually tweeted just last night : “The Vanishing of #EthanCarter = #UncannyValley applied to vegetation” – Glad to see I’m not the only one.
    To me, that’s why so many people (myself included) got seasick with it. The foliage behaves like wave in the sea, not actually like foliage. Yet, its rendering is so close to reality…

  16. stevev45 says:

    My take away: It’s clearly easier to texture stone in a realistic looking way than it is wood, especially timber.

  17. Michael Fogg says:

    This article makes me think of the reaction of ‘academic’ painting to impressionism (“It’s just a bunch of blobs!”). The unrealness, the artificiality (so to say) are all part of the art (of real time 3d computer graphics). The distortion of reality that happens in the creative process between the ‘model’ in existence and final aristic output is at the very core of art, it’s what’s it all about. The final destination of computer game graphics is not 1:1 photorealism. Pixels in a 3d scene can be seen as an equivalent to the traces of a painter’s brush on canvass.

  18. liquidsoap89 says:

    The environment issue that always irks me is buildings with perfectly square walls and flat corners. Playing through Dreamfall Chapters had that thought come up a few times. The outside corners of lots of the buildings were just flat, perfect lines. Obviously technical limitations are a cause here (it certainly wasn’t art direction at fault in this example), and those will diminish over time.

    We just have to be patient.

  19. NotOscarWilde says:

    I would just like to say that I have just bought a year-long RPS Support subscription because of this article. Walking around modded Skyrim and enjoying the grass, the trees, the sounds, even the noises and warmth of the occasional secluded tavern — that is something that I enjoy immensely. I’m glad there are gaming journalists who get to write about those experiences (and getting them just right).

    Grass on, Philippa.

  20. AyeBraine says:

    I was aware of developers’ ambition of creating natural lanscapes. And the first thing I’ve noticed when I carefully inched out of that tunnel (well, second – first was the attempt to run back into the tunnel, which told me a lot about the world and my place in it…), the first thing that sprang into my eyes, were epileptic trees. I actually thought of that trope name from TVTropes then, because the trees just wobbled: tirelessly, rhytmically, in an exact and stilted manner.

    Yes, you are right: nobody had simulated grass or shrubs in the wind properly yet, and every time they’d be better off leaving it motionless. In the real life, the greenery stays still most of the time, and it’s actually the motion that catches our attention. And both grass and branches move in elastic bursts, constantly trying to come to rest again.

  21. deviltaz44 says:

    This was such an interesting read but i am wondering to ones that get too caught up in the uncanny valley aren’t you missing the point of video games being an escapism of the real world?
    Why do the physics have to react like they do in real life for instance? Does it matter that grass doesn’t behave in the same way as reality?, as it is only in the confines of a video game. I just feel people are looking a little too deeply into reasons why and missing the beauty alot of games possess.
    It is uncanny though someone mentioned Metro Last Light as i am playing this at the moment and it is truly an incredible game graphically, at least on PC. I don’t have a rig powerful enough to play at 4k but i am enjoying it at 1440p and it is just stunning. I haven’t tried the newer versions which are meant to be more optimised though but i am still impressed.
    Perhaps if things like grass bother you so much in a game look for ways to switch off these particular effects. Quite often this can be done in config programs for a game and you often don’t need alot of nouse to work this out.
    I do agree with what others have put here though and that is that this was such an interesting article and very interesting responses but just keep in mind we are playing video games and they don’t have to reflect reality in any way shape or form to be an incredible journey to travel on.