How Virginia's Cinematic Editing Works

This is The Mechanic, where Alex Wiltshire invites developers to discuss the inner workings of their games. This time, Virginia [official site].

Virginia is a new game from studio Variable State about two FBI agents investigating the disappearance of a child. But its story is less about that mystery than it is about the lines you draw between the fragmentary events, images, locations and characters you witness, as well as lines you draw towards things you sense you haven’t.

Like Brendan Chung’s Thirty Flights of Loving, Virginia tells its story through a technique that’s absolutely native and everyday to filmmaking but it’s novel to games, at least outside of cutscenes. Games are meant to be unbroken realtime, right? And yet powerful and subtle dramatic effects are possible through:

THE MECHANIC: Cutting

(Light spoilers and references to events in the game naturally follow.)

“From the first day you’re sat in front of a television or you go to the cinema, you get a pretty advanced knowledge of cinema and the use of the cut, and you take it completely for granted,” says Jonathan Burroughs, who co-wrote and designed Virginia. “But playing Thirty Flights of Loving, a game that embraced the use of the edit, it made me aware it was missing completely from the vocabulary of games.” Burroughs has a wonderfully gentle English voice, and is very firm on the point that Virginia wouldn’t exist without Thirty Flights of Loving. (He also speaks in sickeningly elegant sentences, despite constantly apologising for rambling.)

“In the context of realtime play you don’t get cutting at all, but it’s so fundamental to film, being able to juxtapose imagery for emotional and intellectual meaning, to contract time and space for the purposes of brevity or to create interesting omissions that allow the viewer to fill in the blanks with their imagination and create new meaning. Just by chopping stuff out or reassembling footage you can create these new emotions and meaning, and games typically don’t do that at all. To then play Thirty Flights for the first time - if I’d never seen that game I’d have assumed that editing just doesn’t work in games, and that’s why it’s never done. To see that it works and that it feels natural, it doesn’t feel jarring, was revelatory. The opportunity to explore that seemed too good an opportunity to miss.”

And so began the process of building a game that amounts to over 150 separate scenes in which Variable State quickly realised that a interactive context heightens jarring leaps. The experience of watching film is passive: you have no control over the content and pacing of what you’re being shown, so unless it’s done badly, it’s unlikely you’ll feel particularly distressed by a cut. In a game, though, you’re often acting with your own intents, and so if a cut should happen before you’ve reached the doorway you were headed for, or the object you wanted to inspect, Burroughs says it can feel very off-putting.

Development was a process of learning how to avoid such circumstances. In the early days there were many cuts where, as the player walked down a corridor or through an environment they’d cross an invisible line and a new scene would appear. “We found they had a tendency to be jarring when they occurred at points the player can’t predict, and they ended feeling quite on-edge. You’d be like, ‘If I turn this corner am I going to be elsewhere?’” says Burroughs.

Over time, an informal language developed, where as a player you learn that interacting with various objects will probably lead to an animation that feeds into a cut. “We found that helped make it more comfortable; typically you’d instigate a bit of performance from the player, and then a cut would occur when control had been taken away.” It’s an effective technique, one that Variable State had to be careful not to use too much, developing into a rhythm of walk a bit, click, cut, walk a bit, click, cut. “It was all a process of trial and error, using a technique as far as we could take it before it felt rote or felt too jarring if used too much.”

Another issue was presenting too wide a break in continuity between scenes, which would make them feel too disconnected and leading to the player becoming emotionally disengaged. One example is the sequence which begins in Halperin’s office and ends in the first visit to the quarry and the discovery of the red bird. At first, the game cut directly from the office to the quarry, but it was too abrupt, offering too little sense of what happened in between. “Often we do that intentionally to try and create ambiguity in the story but sometimes it’s too much,” says Burroughs. They tried fading down and fading up into the quarry scene, but the team wasn’t a fan of this technique. “If nothing else worked we’d go for it, because it creates a chapter break, a bit of breathing space, but it always felt like we’d failed if we had to resort to it.”

In the end, they created a very brief bridging scene where you get a vista shot of the quarry as you and Halperin arrive. “It’s only two or three seconds, but you understand how you’ve left that previous environment and arrived in the next one, and it removed the frustration that was there beforehand, where for a few seconds you feel disoriented. There’s a logical progression to events and there’s an opportunity to think about different things instead of trying to understand why the cut occurred.”

One cut that works really well - and memorably - is where progression through a space is shortened by cutting through it, such as walking through the FBI headquarters and down the stairs to Halperin’s office. It’s also used to move time forward, such as during the scene as Anne and Halperin wait in the car opposite the entrance to the observatory. Because the environment changes minimally, you get a sense of how the story has progressed.

Other techniques can bridge scenes, such as bleeding sound from the next scene into the end of the preceding one. Just before you first visit the police station you hear typewriters that build up its ambience before you see it. Another example is when you drive to the roundhouse and the band’s music begins to play as you pull up in the parking lot. “We go the other way around as well,” says Burroughs. “Getting the sound before the visual cut bridges a contraction of time or space or gives the sense of a journey. But the other way around, where the sound lingers after the shot’s gone, it can create emotion.” Towards the end of the game, when you go to the diner for the last time, the sound of birds in flight from the previous scene lingers.

Lyndon Holland’s music does a great deal of the work in smoothing out the cuts, setting emotional contexts and making associations between scenes. As a mark of that importance, Holland soon became deeply embroiled in fine-tuning the logic of cut timings so they married precisely with his score.

His music is integral to unifying the dizzying hallucinogenic montage towards the end of the game, a succession of different scenes from various points in time and space, some real, some imagined. Holland’s music was written and recorded before the sequence was fully edited, despite the team knowing its scenes weren’t connecting together well at all. “It felt like a process of film editing, where we were shaving seconds off one scene, but because it was a fixed piece of music which was quite hard to change, we had to add in seconds elsewhere, or add in extra shots. An adjustment to a sequence in the middle would have consequences for all that followed it.” They had to rework animation, add geometry to scenes and build new scenes to make it all work.

As well as bleeding sound across cuts, there are match cuts, where a related image is presented in the next scene, such as when Anne picks up the bill in the diner and then we see her holding a similar pose with the boy’s missing poster in the next. Here, Anne’s irritation with Halperin leaving her the bill to pay is carried over into the future. Or when there’s a continuity of movement through a cut, such as during the end sequence, where Anne reaches for the director’s hand on the stage and then there’s a cut to her holding the mask in the observatory.

Virginia also uses dissolves, where one scene fades into the next, an effect that isn’t fully successful, as Burroughs is happy to admit. Virginia takes a screenshot of the last frame of the preceding scene and displays it as it fades into the new one. The first scene is therefore frozen during the dissolve, though Burroughs would love for both scenes to be active. “We try to do them as quickly as we can,” he says, but the central issue isn’t actually with Unity, Virginia’s engine, having to render both at once (“It would probably be achievable,” he says). It’s a sticker design issue with retaining player control through the cut. “If you were wiggling your camera all over the place it would be slightly strange to cut between scenes but be in control of both. I think it would break the immersion.”

During the whole course of the game you never get a loading screen between scenes, highlighting the fact that as well as a creative act, cutting is also a technical one. Virginia uses Unity’s standard asynchronous loading system: when you first load the game, you enter a Unity scene with the game’s frontend and all logic for player movement and other systems present. As you play, the game brings in geometry and assets for the next two game scenes, but with their visibility switched off. Then, at the point of cutting, the visibility for the current scene’s assets is switched off and the visibility for those of the next is switched on. Aside from presenting some knock-on issues with dynamic lighting that the team had to solve, the system works in such a way that you never sense the heavy lifting going on underneath.

Virginia feels finely polished, but it’s very much an experiment. There are many valid questions to ask about creating a linear narrative like this in an interactive medium, and as entertaining and interesting as I found it to play on its own terms, it’s perhaps better to see it as a question, or a new step along a journey into storytelling in an immersive medium.

Burroughs seems to feel similar: “Having gone from Brendan making that incredible cognitive leap [to use cuts in Thirty Flights of Loving] and us trying to embellish what he did and explore cutting a little bit more, I’m interested to see if there are limits on how this can be applied to games. I think there are perfectly good reasons for why games tend to choose to present themselves as continuous, unbroken realtime without cuts, and I’m interested to see if there are opportunities we failed to explore.”