How Green Is My Uncanny Valley

I have The Vanishing Of Ethan Carter installed on my PC and ready to go. But there's something that's been playing on my mind regarding that game before I've even booted it up. It's been nagging at me ever since I watched a video from Andy Kelly's Other Places series – the one which focuses on Ethan Carter's Red Creek Valley – and it finally crystallised a problem I've been experiencing for years without being able to put it into words.

Just after a shot of a dam there's a lingering shot of a churchyard. In the foreground a handless statue of Jesus marks the grave of a woman named Thusnelda. In the background the autumn trees sway in the breeze and the weed-infested grass – well, I want to say that it sways but it's a sway which comes via a clump-by-clump waggle. That grass is why I'm proposing there exists a foliage version of the uncanny valley.

The uncanny valley is the idea that our comfort level around representations of human figures suddenly dips as they approach realism. The idea is that we are unsettled by things that are almost-but-not-quite-exactly-human. But perhaps it doesn't have to be limited to humans or animal. Perhaps it can be extended to plants. The uncanny grove? The uncanny vegetable patch? The uncanny meadow? Hell, maybe even a *literal* uncanny valley.

To offer a bit of backstory; in the last year I've become very aware of how trees and grass move in the wind when it comes to videogames. It came about because I finally decided to get into the Skyrim modding scene. But with the power to change anything in the world I didn't like or felt could be improved, Skyrim became my own personal dollhouse. Instead of a game it was a project. Screw that main dude and his beef with dragons – what about the important things like the colour or texture of the ice floes?

I began to explore the environs of Tamriel with a notepad in hand, writing down anything I felt could be improved to better match my ideal vision of the world. Obviously this is a terrible idea and you end up thinking of the game more as a tableau – something to be preserved in aspic – and not anything resembling a living world. Regardless, I was unsuccessful and that was entirely because of the trees. They never looked quite right.

I spent ages going through mods trying to find one which felt authentically tree-ish and became so fed up by my failure that I never play Skyrim anymore. It's a game I have come to regret having access to on PC because having the power to tweak it combined with my own aesthetic peculiarities in a way which ruined an otherwise-fantastic game.

Watching the grass moving in a Red Creek Valley graveyard I was struck by a familiar feeling of discontent. But the thing is, I've read a lot of the blog entries by the creators of The Vanishing Of Ethan Carter, The Astronauts. I know the lengths they went to in creating the environments of the game. It seemed odd to me that grass would have escaped their attention so I dug back through their archives.

One of the techniques they used was photogrammetry. It involves going out to photograph real-world objects in situ then using software to convert those photos into 3D art assets. These are then subject to compression processes (otherwise a single rock can take up an entire game's polygon budget). After that the scanned assets are mixed with traditionally constructed game artwork and subject to stylised lighting and processing.

The goal of all this is not to achieve photorealism. The intention behind the work and the field trips is that The Astronauts never want your brain to become distracted by things like texture tiling. As they say on their blog, "It may look photorealistic to some, it may seem magical to others, but if we did our scanning right, it should above all, feel right."

Applying the same mentality to the foliage proved a formidable task. Graphic artist Adam Bryla freely admits that. "Rendering subtlety, richness and dynamism of vast, wind-torn foliage by using just a bunch of flat, static triangles? One heck of a challenge."

In fact, Michal Kosieradzki who specialises in animation and particle effects spent all of 2013 working almost exclusively on foliage. According to Bryla, "When tree branches are simplified to just a few flat planes, you get large batches of leaves facing the light the same way, so again, we had to get real crafty. We distorted the geometry information to trick the lighting engine into thinking that those flat triangles are not flat, and we added a smart shader that mimics how leaves overshadow each other. We even added a shader that mimics how each leaf changes its orientation towards the sun as it trembles in the wind."

I should say at this juncture that I have no problem with the trees – the trees are gorgeous from the footage I've seen so far. It was the grass which triggered the feeling of slight discomfort.

On the grass front Bryla notes that "[Kosieradzki] went through almost every imaginable method, from billboards (flat grass planes that always rotate to face you, so you never see the flatness), all the way to painting the grass using particle effects." From this it seems as though the team had a similar general approach to grass as they had to all the other aspects of the game, certainly in terms of the basic modelling.

In reading about the uncanny valley phenomenon two things struck me. One (which I fully intend to gloss over here as quickly as possible...) is that the uncanny valley might not even exist and thus extending it to flora is simply a way of dressing up my own aesthetic peccadilloes as potentially scientifically interesting. The other is that the look of the thing isn't actually the main trigger for discomfort, it's more likely to be disharmony between the look of the thing and other aspects such as how it moves or feels. The reasoning for that last one is that static art assets are often a little ahead in terms of technical sophistication than, for example, rendering 3D motion. As a result something can look lifelike, but then move in a less than lifelike manner – we recoil because the object didn't behave as expected.

If I had to pick a reason for my own discomfort (a reason that isn't the one where the uncanny valley doesn't exist) I'd say the most likely candidate is that there's a slight mismatch between the rendering of the grass blades and the rendering of their movement. It's such a tiny thing, and one which I know the team involved put a lot of work into resolving. I also only became aware of it because I was watching a video rather than playing the game. But having previously equated the uncanny valley exclusively with representations of people I'm inclined to extend the definition. I think the uncanny valley could actually apply to all things affected by natural movement.

This article was originally published as part of, and thanks to, the RPS Supporter program.