Music To My Thumbs: Transcribing Braid

By RPS on June 20th, 2012 at 12:00 pm.


Julian Benson asked if we’d like to see Braid transcribed into musical notation, and vice versa. We said yes. Here’s how he got on.

Gaming is one of the few media without an instructional notation. The other arts have tools to relate concepts into symbols that can be digested at a glance, allowing great detail to be condensed into a conventionalised code. Musicians use musical notation, movie makers – story boards, writers – short hand, linguists – phonetics, and so on. If musicians were stuck with the same methods we use – strategy guides, walkthroughs, and videos – each composition would be an interminably long tome that lost all immediacy and comprehension – “Pluck the G-string on the third fret, then immediately after the A-string on the fourth fret, pause, play the G string again”. The other arts developed their own notation systems, yet we haven’t. Why?

Now, we may not need a short form, we’ve managed to get by without a gaming equivalent so far, but that is not to say it couldn’t be useful. Notation has advantages for both musicians and composers that aren’t available to us: the complexity and the pace of a piece can be judged in a glance; (it has lots of symbols; then it’s fast and complex). It’s easy to share ideas, tweak sounds, and write down ideas in brief:, you don’t need to detail the entire concept of the piece to communicate what one musician needs to do, a single instrument’s part can be written in isolation. It’s also possible for someone to lay out an entire score; (instructions for many musicians playing different instruments) and see how all the mechanisms lock together.

Also, as I’ll show later, in developing a short form, we open up possibilities for our games which are difficult to accomplish currently.


The same can be done for games.

The seeds of the idea were born one night whilst listening to a piece by the minimalist composer Steve Reich, and playing a particular world in Braid in which your motions are locked to the level’s timeline. It’s a platformer that’s always introducing new temporal gameplay mechanics, but what makes this one relevant is that it essentially renders the level into a piece of music. As you move to the right time moves forwards, as you move to the left time moves backwards: all the elements of the level, the enemies, the fireballs, the hostile plants, play or rewind depending on the direction you move. Walking from extreme left to extreme right is like scrubbing a 12 second track on your computer.

At this point, I began scrubbing back and forth on the track on the Reich piece.


[Steve Reich - ‘violin phase’]

In Steve Reich’s ‘Violin Phase’ three violins play the same short phrase of music repeatedly but out of sync with one another. To listen to it, it sounds like a mess of noise: each violin is playing a different note from the others causing disharmony.

Much like the three violins, all the elements in Braid’s levels, (the fireballs, the mindless goombas, the aggressive pot plants), appear to be in a state of discord. They’re firing, marching, and snapping and they’re doing it all at once.

But in both Braid and Reich there is an element that makes sense of their discord . In Reich, it’s the fourth violin. Whilst the other three step on each others’ toes, all playing different notes, the fourth plays a note simultaneously with one of the other violins thus amplifying that note above the cacophony. In this way it plays a different phrase of music from the other violins but never a unique note, one of the others is always playing that note also.


The audience is most aware of the music played by the fourth violin as its phrase is made up wholly of amplified notes. Whereas the other violins change their music infrequently, slowly moving through five phrases during the 15 minute piece, the fourth is shifting its phrase frequently, each time pointing to a pattern that was hidden in the noise.
In Braid, the player is the fourth violin. As players work their way to the collectible jigsaw pieces dotted about each level they highlight a pattern of movement that in this case avoids all the numerous aggressive elements.

Essentially, each level in Braid can be further broken down into sections: each one centred around attaining a single puzzle piece. By getting to these puzzle pieces the player is pointing out a new pattern in the level’s complexity of action. When the players get to the end of the level, and the audience reaches the end of the Reich piece, they no longer perceive that in-game world is a mess of conflicting noise. They’re now aware of all the patterns that weave through it.

It’s within these two examples that we have the necessary similarities to start applying musical notation to the game, to prove it’s possible that games can be communicated in short form. It’s a little ham-handed – musical notes are clearly not meant for this – but we can make them fit for purpose.

First, though, we’re going to have to change a fundamental principle of musical notation. Inherent to that code system is rhythm and timing. Though, here, when transcribing for games, at least in this proto-version, the symbols have little to do with timing. What I mean is that two notes next to each other does not mean two actions that need to be performed in quick succession, only that one comes before the other. Instead, the focus is on conveying non-linear movement. Whereas musicians start at the left of the line of notes and play rightwards until they reach the end of the page, gamers do not usually work linearly. Rare is the game that simply has you play left to right with no back track whatsoever. You are often climbing up ladders, dropping down pits, and playing merry hell with linear directional movement. Yet, despite this – so long as there is a singular goal you are working towards – your movement never stops being linear and can be treated as such. Going up is still essentially moving forwards, if each step takes you along the path to the goal. As long as the transcription accounts for this direction then it can still be written. Though, that all said, I can’t wrap my head around notating a 3D space, which is why I only refer to 2D examples here.

These transcriptions are rough, haphazardly stitched together in Photoshop, requisitioning symbols without much thought beyond practicality. But that is the point. To see if it is practicable. Can we see gameplay in similar terms to how we see music-play? And, if we can, what can we use it for?

An explanation of this level without notation would be something like this (right, deep breaths everyone): to get to the jigsaw piece in the image above the player climbs the ladder to the right, activates the lever to the left -which operates the two doors next to the jigsaw piece, raising one and lowering the other – they then climb the ladder again, move to the left end of the platform (reversing time and bringing the falling goomba back up to jumping height), they jump on to the goomba, bounce back onto the platform, move to the right and jump on that goomba to vault the lowered door and finally reach the puzzle piece. (Breathe.)

As you can see, a straight written tutorial is confusing and bloated.

How about an annotated image?


Whilst managing to get the point across, it has a number of drawbacks: I’ve had to circle thin air to show where the second goomba will be rather than where they are when you jump. You can’t separate the annotations from the image without them becoming meaningless. Plus, it’s a mess to look at.


That’s all of the above instruction in seven notes. 100 words down to to 11 symbols.

Far more immediate, far cleaner.

This is how it works: if the note is full it means you need to interact with one of the level elements, if it’s hollow you don’t (I’ve not counted ladders as interactive because in games they’re simply a space where you can travel upwards, in the same way that falling is travelling downwards). The stem coming out of a note indicates upward or downward movement and the arrows above the notes is the direction you’re to move in. So, those first five symbols mean: climb up the ladder, move left and interact with the object on the middle floor (the lever), move right and go up the ladder again.

That sixth symbol means go right, but there is no interactive object to the right, instead, there is an empty down symbol, meaning, you must walk till you drop off the ledge.

The curved line join symbol at the top is used to convey that two actions are immediately linked, so where the drop symbol is joined to an interactive up with a flick, what’s happening is that you’ve hit the goomba and you need to direct your bounce to the right (if it were to the left the flick would be on the opposite side). The next symbol is another full flick meaning you must jump off the object ahead of you (the other goomba) and walk to the last interactive object, the jigsaw piece.

All that information is held in seven notes.

We need only transcribe the player’s movements because the level elements themselves are dumb: the enemies can’t see you, they don’t aim for you and so never differ in their patterns. In the same way, the three violins in Reich’s piece play with a complete disregard for the other instruments whilst the fourth – though technically dictated as much as the other three – is musically aware of the others in that its notes are are always amplifications of theirs. This is true to the point that often ‘Violin Phase’ is played by a single violinist taking on the role of the fourth and three tape players acting as the first three. The violinist is able to improvise patterns over the background noise of the other three, so long as they are playing the note in time with another violin they are free to make their own patterns. Likewise in gaming, the player can move anywhere in the space, but in order to progress to the goal of the level, certain movements in time with the dumb level elements are necessary

How can this, albeit limited example, be expanded to other games?

If we developed a universal notation system for gaming, or even a set of notation systems that accounted for genre, it could make the lives of many people involved in our industry easier. It would provide a succinct means to communicate ideas for mechanics, levels and systems in a language not specific to any one department.

Current game-making curriculum rely heavily on case studies. If we were able to capture basic design principles in a concise symbol form it makes it easier for students to study games in their own time, applying the abstract principles without the guiding hand of an instructor.

There are benefits for players also.

Braid works as a start point because it skirts the edge of Reich’s work, close enough that the notation of one can be co-opted to fit the other. However, with that in place we can easily expand to other games: VVVVVV and Super Meat Boy can both receive the same treatment, though for slightly different purpose. As games, they’re both the haven for speed runners – those players who aren’t simply content to complete a difficult platformer, they must sprint through it. Whereas an average player may be able to complete Meat Boy in a week speed runners have done it in just 22 minutes:

Speed runners are the concert pianists of gaming. They practice and practice each stage of a game, finding a route through noise of each level, sounding out the optimal path, refining their pattern working at it until they have it note perfect.

Notation gives speed runners two things: an ability to view an entire level with all its interlocking movements and a language with which they can share their routes. Much like a conductor is able to lay out an entire concert score, with all its different parts, and comprehend the whole, a speed runner can see a completely transcribed level, the movement of all its parts. With that information they can see and devise a route through all the obstacles, all on paper: they can show it to people, other speed runners: it can be consumed faster than video, it can be tweaked an altered by the speed runner community.

Though, to be of real use to speed runners, then bringing timing into the notation system would be vital. As I said before, what I’ve detailed above is a rough and ready approach. Simply taking another art’s notation system in lieu of our own. I’m not saying this is the one we should use but using it to show something of what we can achieve with it. The real applications and possibilities for a notation system would not start appearing rightaway. Developing a system would be gathering a toolbox to see what people make with it. Think of it as a pen & paper Minecraft without Creepers and pet wolves.

However, considering we have the rudimentary notation system here and I’ve been harping on about the benefits of abstraction for a while now, how about something else a little off the wall. Take something that already works musically and do the reverse: transcribe a piece of music into a level of Braid.

Here we have the first line of Ode to Joy. Now originally this would be played bythe left hand and right hand of a pianist but we can separate it into player and game elements. It’s not an exact copy of the Beethoven piece: I’ve changed the standard crotchets (interactive ups) into jumps to the right. I’ve also changed the level of the fireballs and plants slightly to make for a more threatening level, and removed two problematic notes. That all said, you play it on a piano and it will still sound like ‘Ode to Joy’.


Looking between the score and the level we can see you must jump straight upwards immediately, that’s to dodge the plant that comes out of the pipe immediately to your left. Next, jump up the two platforms, over the ledge, and over the fireball that comes from the smaller pipe to the left. Jump over the plants down the ledges. Once you reach the lower platform, the first plant, which emerges at intervals from the pipe, attacks again – though you are well out of range. You must then jump up the next two platforms and jump from the pipe onto the finish flag before the second plant emerges from the pipe beneath your feet.

No, it’s not the most exciting level but it is ‘Ode to Joy’ as a game; (something I suspect Beethoven always wanted).


You can also see how, by transcribing from this way round, the original timing of the piece and its musical flow has found its way into the structure of the level. It retains its pauses, its dips and rises, and the enemies attack in time with the position of the player.Transcribing a more complex piece would bring with it a more nuanced level, one with lulls in the action and a rising crescendo both in the level’s geometry and its enemy placement.

Of course, if we were to continue pushing the short form model derived from musical notation, its limitations would become clear very quickly. We can’t lift it for our purposes wholesale: it can’t handle 3D space, unpredictable AI, or complex player actions for a start. What this example has shown is that it’s possible to transcribe our interactive medium into concise abstract symbols, that it’s within our reach as gamers to develop a new language for our games. The same example could have been made with many of the available systems of notations available. Though, each would have had their problems and their limitations. For the best fit we’d need to develop something wholly our own. It would take time, modern musical notation is the product of generations of development, but even a basic notation system beings to open up possibilities for us as designers and as gamers.

Also, as you can see. once we break something down into an abstract representation it allows us to link it to abstracts of other media more easily – music to games and vice versa is just the beginning: for instance, there’s a short form to transcribe dance movements which means it may be technically possible to perform games as a dance: Half Life ballet Kickstarter, anyone?

Many thanks to DragezeeY who created the Mario tileset used in the Ode to Joy image.

, , .

74 Comments »

  1. Kynrael says:

    “That sixth symbol means go right, but there is no interactive object to the right, instead, there is an empty down symbol, meaning, you must walk till you drop off the ledge.”

    Isn’t it “go left” ?

    Otherwise, great article, thought provoking !

  2. pakoito says:

    Where is the actual full transcription? Or maybe a musical video of the piece?

  3. Fuzzball says:

    Wow, this was very intriguing. I think it would be interesting to have an international competition to come up with a universal notation for games (maybe just platformers to begin with); I’m sure there’s someone, somewhere out there with a fantastic idea for it.

    • 4026 says:

      I’m not convinced such a universal language is achievable. Much more likely, you’d be able to create notations for specific types of game. In the same way that the Algebraic Notation for chess can (with some tweaks) be applied to Shogi or Go, I’d expect it’s possible to construct a similar language for, say RTS strategies (Build spawning pool in grid F4, Zergling squad 1 moves to grid P17) that would apply equally to Starcraft, Company of Heroes or Total War. But it’d be totally unable to deal with, say, InMomentum.

  4. omgitsgene says:

    DPMFP

  5. Secundus says:

    wow, games are art now, thank you rps god.

    • Mollusc Infestation says:

      Games have always been art. Just like dead sheep in formaldehyde.

  6. thestage says:

    “What this example has shown is that it’s possible to transcribe our interactive medium into concise abstract symbols”

    But “our interactive medium” is already the product of a concise abstract symbology. When one plays an A note on a piano, one is playing an A note on a piano, the sound being itself a physical characteristic of the world, with the instrument acting to direct it in a given, repeatable manner. What is being directed in a video game? The game is constructed and filtered through the symbology of programmed code, and we interact with it through an interface (controller, keyboard, etc.) that is the very pitch of abstraction.

    The easiest way to understand the difference lies in the use. Why is music notated? So that it may be repeated. The composer of a piece of music, in writing it down, is in actuality creating it. Do this, this and this–and you have the piece. When he conceives of it, the conception is transferred to the written medium (or perhaps it is mentally already rendered in written medium, but to enter the realm of semiotics too heavily here is to go far, far beyond the scope of what I might hope to accomplish in the medium of comment) as a means of dissemination. Were one to provide the theoretical notation to a video game, this would not reproduce the game. One can provide a transcription of a game of chess using that game’s “notation,” but to read it can only ever reproduce a specific game of chess; one is not able to derive from this notation the game of chess itself. You speak of “discord” (dissonance) in Braid and in Reich, but the equivalent, should their be one, to what you hear in the piece of music would be the game intermittently turning itself off, changing the rules or method of control, removing the visual element, replacing the systems of platforming (“tonal systems”) with fundamentally altered ones. In music what is dissonant are sounds used in specific ways that do not accord with–well, with what? And what is the analog in video games? You’ll find that a difficult question. A bit of a tangent, but further illustrative of the arbitrariness of your association.

    That such a method clumsily transferred to video games could only ever be second order should indicate the difference. There is indeed a difference between a “notation” and a “language.” One “writes down” music according to rules of notation, in which a concept of 1:1 pretends to attain, but one only ever discusses music in terms of language, of nebulous figurations and signifying obtrusions which can never hope to approximate it, but may simultaneously reach beyond. The free play which is inherent in the term “game” makes a mockery of notation. At best you may signify possibilities rather than delineate some structuralist grammatical fantasy. But what is a notation that deals in possibilities? Not a notation. A notation leaves no room for the player or the play, which does not exist in terms of reproduction but of production itself. One cannot simply insert a note into the Ode to Joy without either dissolving the Ode or rendering it, with expertise, into a signature performance (at best). In Mario, however, the player is free to jump whenever he’d like–and one could (and probably should) argue with ease that the game does not exist without this free play, that to “perform” Mario is not to trace its line, but to jump and to jump some more until the line is discovered,the quality of the game lying in its ability to coax and respond to this. Video games, so to speak, are not designed for the sake of hearing.

    And Mario is but one kind of game; the only one, at that, that is liable to present itself at all as a musical analog. A “notation” of the kinds of games you readily understand as being problematic (“3D space…complex player actions”)–and many others besides–begins to look an awful lot like a language instead, a critical lexicon defining the parameters of presence to which one would appeal as pedagogical building blocks. Verbs and agency and mechanics and the like, together with current generic jargon and other borrowed devices.

    • The Snee says:

      So in other words, what you are saying is that the notation for games is the game itself? That seems to particularly make sense in the context of platformers like mario and braid.

      • thestage says:

        One of the things I am saying–the thing I think is relevant to this question, at any rate–is that there exists no a priori criterion upon which a notation of video games could hold any meaning.

        • qrter says:

          I think the thing is that methods of transcribing have been developed to solve a problem. This game transcription thing is an answer in search of a problem.

          Transcribing is essentially about communicating, this new system seems to not know what it’s trying to say or even who it’s saying it to.

          • brulleks says:

            Precisely this.

            It’s a pointless waste of time designed to appeal to those who prize any demonstration of intellect (even if only tongue-in-cheek) over practicality.

            So, exactly the sort of thing that they’ll end up researching in the very University in which I am currently sitting. I’ve seen far more pointless things being researched here, it has to be said.

          • Nixitur says:

            brulleks, I don’t think it’s pointless in the slightest. I’m sure the ancient Greeks would have thought that trying to create a notation of mathematics that includes things such as variables was pointless, too. After all, they could deal just fine with their notation and ways of expressing thought.
            The modern musical notation isn’t as old as people think either. Most music ever composed is lost because nobody could think of a uniform notation that could encompass all of music.
            However, I don’t see how that idea for a notation could ever apply to games themselves. It may be an interesting shorthand notation of how you want the player to go through your level (so, extremely useful for discourse between level designers of games of the same genre), but it could never encompass the game itself since interactivity does not lend itself well to linear notation.

      • MadTinkerer says:

        It also makes sense in the context of Doom.

        In the games I am working on, I hope to eventually merge Kinaethetics with procedurally generated music by matching the music (generated) in the background to the beats of sword swings, foot steps, and other audio and visual cues made by both the player and NPCs and monsters. A bit like the musical cues of the Ultima games taken to the next level.

        • Soon says:

          I’d really like to see/hear how that turns out. It’s something I’d be attempting were I remotely capable.

          I find there’s often some discord between game music and what’s actually on screen. Why not match the timing with the animation and such. In my head, it would add more impact to every action, immeasurably improve feedback, and make the game seem more fluid to play. Although, I think it would need to be somewhat subtle to work best. So good luck with it.

    • Melf_Himself says:

      I agree with this verbose individual.

    • Snargelfargen says:

      Isn’t much of musical notation left up to the player’s interpretation though? Some notation such as accents and grace notes are vague on purpose, leaving it to the musician’s judgement. Also, codifying a set of actions still leaves a lot of room for variation, which is why the same piece never sounds quite the same when played twice, especially if done by different players. The notation only accounts for the approximate force with which a note is played, or how high Mario jumps.. That’s not even bringing in the vast body of music that’s based on a common set of rules, but ultimately relies on improvisation, such as blues or jazz.

      Code, or keypresses could possibly be used, but that sort of system would be almost entirely opaque to the reader, whereas a simplified system similiar to musical notation could provide a much easier to grasp overview, while still leaving player agency intact.

      Steve Reich uses notation to discover novel forms of play. I think a similiar system could be used to to good effect for constructing games. Physics puzzlers and platformers certainly come to mind, but who knows what else would be applicable?

      (Apologies if I haven’t made complete sense, it’s been a long time since my music theory classes)

      • TomxJ says:

        “Steve Reich uses notation to discover novel forms of play. I think a similiar system could be used to to good effect for constructing games.”

        This is the exciting princliple of this article. I’m not bothered by a universal notation script for speed runners, but creating a game and then applying musical notation as a design aid is all sorts of exciting. As we already see Music helps define, pace, mood and even narative in many games.

        Tom j

        • deathchuckles says:

          Musical notation leaves room for improvisation, it’s true. But if the point of musical notation is to allow the music to be repeated, disregarding for the moment the manner in which it is repeated, then one wonders what the point of a gaming notation might be. Shouldn’t a game teach us how to be playing the game itself? What can someone glean, then, from the notation of a game? An understand of how it is played? The route a player took from A to B? The most engaging way to discover that is to let the game teach you. (This is disregarding the few weak examples in the article, such as speedrunning.)
          For instance, who here, besides the author of the novel, needs notation for a novel? Of course the article touches on the fact that no one needs to be taught how to read from left to right; but why would we need notation for the concept of a novel, to understand its core ideas? The best way to do that is discussion which transcends the mere repetition of notation. In fact, being a writer myself, notation for a novel is quite useless. What is useful is the great collection of notes that in some places act as summary of a plot or academic supposition, but which also discusses tangentially many different ideas one needn’t grasp in order to read a book. No one needs to rewrite a novel, and few need to play a game in an exact way.

    • Zetetic says:

      There is indeed a difference between a “notation” and a “language.” One “writes down” music according to rules of notation, in which a concept of 1:1 pretends to attain, but one only ever discusses music in terms of language, of nebulous figurations and signifying obtrusions which can never hope to approximate it, but may simultaneously reach beyond.

      I don’t think you need to draw out any difference between a notation and and a language – “instructional notation” does capture well that it’s simply a very minimal language almost entirely in the domain of getting someone to do something with this or that instrument – to produce sounds that are to music as the sounds of phonemes are to our spoken languages. That music is a rather different language doesn’t lead us to saying that that “musical notation” is no language at all.

      A “notation” of the kinds of games you readily understand as being problematic (“3D space…complex player actions”)–and many others besides–begins to look an awful lot like a language instead, a critical lexicon defining the parameters of presence to which one would appeal as pedagogical building blocks.

      I think you’re being unnecessarily elaborate. I think we already have languages that are analogous to musical notation – the languages used by code and data structures (intended for the computer, one way or another, and for others humans in a slightly different way) and indeed rulesets and game boards (intended for the human to interpret). These languages are used to order our computers, or us and our friends, to do this and that and – hey, presto! – a game appears, just as music notation orders us to make this sound and then that sound and Ode to Joy appears!

      If there’s something particularly interesting about the game as opposed to Ode to Joy, is that the game sets up a ‘language’ of sorts, of its own (or at least more explicitly than Ode to Joy does) leveraging your existing knowledge of platformers or FPS games (and their grammar and vocabulary as some would like to say). This article then, as others have pointed out, seems confused between translating into marks on a page some story or other that you might produce in that language of the game and far simpler instructional language that one might follow in order to produce, say, a speedrun.

    • Rapzid says:

      I smelled BS right away and you’ve pretty much summarized in a far more articulate way than I could have why.

      I would like to point out though, that with any deterministic system, you will always get a the same output from the same inputs. That means that for games that don’t rely on any sort of attempt at real randomness you would get the same results from the same inputs. In a sense, the game would be the instrument at that point, the inputs the ‘notes’, and the results the composition. Take a game like Trials 2 as an example. Complex looking physics and very difficult. But record the inputs of like 4 keys and you can replay every run precisely.

    • deathchuckles says:

      God damn, you finally, FINALLY made me register with this post.
      I’m in agreement with thestage. The easiest rebuttal entails reasons of complexity and the state of games as art. I’m going to end up not crediting someone with something, because I know someone has said this, but the first thought most will think is that because of the complexities of games, the easiest notation of a game is the game itself. While not quite accurate, its a good rule of thumb, for a very important reason. As a state of evolution, games are becoming more complex (and I’ve developed many theories on realism in games to debate whether or not this is a complete matter of an increase in hardware or a fulfillment of the core desire of an interactive medium). Notation, by definition, is the removal of complexity by adopting a symbolic alphabet to portray the medium in abstract. The easiest notation of a game would be a less complex game–for instance, Braid being notated by the use of Super Mario. This is a metaphor; what I mean is that the modern platformer would need to be notated by something equivalent to an 8-bit platformer. This is the antithesis of an evolution in games, as art, music, animation, etc. are all being constantly improved to serve our desire for increased complexity. To notate interactivity is to simplify the interaction, thus failing in the notion of a more complex interactivity.
      That is, of course, if you are notating the interactivity, for, as thestage points out, notation is primarily used to serve repetition. I would argue, then, that to remove the interactivity of a game is to fail to grasp the point of a game. It is the principle of interaction that makes a game; it’s the reason why Dear Esther is a game and not a film, because the protagonist is under full control of the player, and because procedurally the game may play out differently each time. To notate a game of Dear Esther in order for it to be repeated would deprive the game of its only purpose for being a game, and remove one of the constant themes of gaming in general, evident not just in software games but tabletop RPGs, board games, etc., which is the theme of possibility, that you may play a game twice and, while the rules remain the same, you will achieve a different outcome. To notate a game for repetition, or to make a game simple by notation, is to remove the element of possibility, and to decrease the amount of options a game may have available.
      A third argument runs that because a game is, like the graphic novel or the film, an amalgamation of media whose constituent components may be notated differently, it is impossible to develop a trans-medial notation to satisfy our needs. I challenge the author to develop a true notation for a film, for example; you will find that a story board will not be enough, as it only accounts for plot and cinematography, not casting, score, etc. If a notation could be developed, it would likely be over-complicated in addition to not being necessary.
      I am ultimately not convinced by the article’s argument for the necessity of gaming notation.

      • rockym93 says:

        Interactivity is exactly why this kind of notation is useful, though. If you have a way to record what you did, and how, that gives you the ability to compare and dissect and discuss different courses of action. If I solve a puzzle one way, and you solve it another equally valid way, a kind of notation is an important tool in being able to talk about the game.

        Think about the choices in games like Mass Effect. The solutions you pick make for vastly different play experiences, which is really interesting and unique to games, and being able to describe those decisions formally is, I reckon, a useful thing to be able to do.

        • Gira says:

          “Vastly different play experiences” in Mass Effect.

          Good one!

  7. Berzee says:

    The actual news story here is,

    “New shorthand strategy guides for simple sidescrolling platformers.”

    isn’t it? The title and somewhat the content of this article makes it sound like the *game* has been somehow boiled down to notes, but as far as I can tell that wasn’t the intent.

    It’s the walkthrough that has been abbreviated, not the game. I fear we will get mainly comments (I almost made one myself) saying “THAT MUSIC THERE IS NOT THE GAME” — which is perfectly true, but may ignore the interesting idea actually presented, which is whether or not that music there is the walkthrough.

    • Docslapper says:

      Yeah, it’s a simple notation for walkthroughs rather than games themselves.

      Games are already created from a notation (code), unlike music (which is created from physical stuff going twang, zing, thud or bang). Musicians take code (score), operate instrument, play music. Computers take code (code), operate processors, play games. So a ‘notation for games’ that actually encodes games is redundant, because games are already completely specified by their existing notation.

      This walkthrough notation is pretty limited, for a number of reasons, like:
      - there’s a single operator for ‘interact’ (closed note). Lots of games, even simple side-scroller platform games, have multiple methods of interaction and getting the wrong one will defeat the walkthrough.
      - any introduction of random events defeats it. You’d have to specify a set of behaviours to deal with the event, rather than a fixed music-like sequence of actions.

      Interesting idea, but games are much much more complex than music, so we’ll need a much much more complex form of notation. For a start, two people playing the same piece of music will always follow the same note sequence in the same rhythm. Two people playing the same game will never follow the same move sequence in the same rhythm. There is only one ‘solution’ to a piece of music. There is usually a massive collection of winning ‘solutions’ to a game, even a single level of a simple platformer.

      However, this is the Holy Grail of automated testing; being able to simply provide a score or script for an interaction with a computer program that tests a feature or reproduces a bug, in a standard format that can be used over different programs/features.

  8. PPOY52 says:

    omg wtf asdfg

  9. The Snee says:

    I would have thought of the idea of notation as instruction; the series of commands needed to perform the “peice” on an instrument. I would argue in this case that the game itself is an instrument, and you probably need different notation depending on the game.

    Take fighting games for example. If I say 236a street fighter players should recognise that as a hadoken, with the numbers from a keypad being easier to describe than “diagonal down right”. Looking at any in depth fighting game discussion you get information about jump c, cancel to 623 d etc. Long combos can all be written out in a formula that others can use. A complex, difficult formula that I can never pull off, but a formula nontheless.

  10. Melf_Himself says:

    This is a nice idea. I think it’s got too be far too game-specific though, mechanics just change so wildly. Music has the same notes regardless of which instrument you play.

    It would be probably a lot less obtuse to, instead of using music symbols, use short words
    eg
    up, left, interact, right, up, fall, bounce-right, bounce-up, interact

    If you name the words well and there are not too many, you could change that to something like
    U,L,*,R,U,F,B-R,B-U,*

  11. Jimbo says:

    →, ↓, ↘ + punch

  12. jhng says:

    Brilliant article — there are the beginnings of some fascinating ideas there.

    The development of musical notation was absolutely fundamental to the evolution of new musical forms of expression. If we do feel our way towards a more abstracted concept of gameplay notation, this might have a really positive effect.

  13. Darth_Pingu says:

    Fighter games. They’re there already.
    When practicing cap routes in Tribes: Ascend, I sometimes think of this aswell. Basically many games consist of doing an exact sequence of keystrokes. Mostly it’s the time (milliseconds) one holds the key down that differs. (VVVVVV could have a score guide)

  14. Floomi says:

    You know what else has its own notation? Juggling. There’s a few notations, but one is Siteswap. Agree that notation for platformers is a good idea, although hijacking musical notation doesn’t seem entirely fit for purpose (not that that was the author’s intent).

    • Hmm-Hmm. says:

      That’s awesome. Thanks!

      • Leandro says:

        Man, now I want to know what other crazy stuff there’s notation for. Cooking? Pinball? Sculpturing? Soccer?

        Congrats RPS on an unexpected and amazing article. Mind blown.

        • rockym93 says:

          Rubik’s cubes. Trying to learn to solve a Rubik’s cube from the internet is almost impossible, because they assume you already know what all their notation means.

          I DON’T WANT TO LEARN ALGORITHMIC SEQUENCES DAMNIT, I JUST WANT TO NOT HAVE TO PEEL THE STICKERS OFF AGAIN.

  15. Fincher says:

    Something I’d expect to see from Extra Credits rather than RPS, what the hell is going on? There’s always a danger of disappearing up one’s own backside when talking about Braid, but this article takes it to a whole new level?

    You should really stop looking at video games as an evolution of film when you should be looking at them as an evolution of the boardgame. Did the addition of narrative to pen and paper RPGs turn them into an artform? This guff is mindnumbing.

    • 4026 says:

      Man, you’re so right! No-one’s ever spent time devising a system of notation for a boardgame. Whereas they do it all the time for films.

      • Fincher says:

        Is a system of notification an indicator of artistic merit? No.

        • 4026 says:

          Did the article ever suggest it was? No.

          Does phrasing your argument as a rhetorical question make it more convincing? No, it does not.

          • Fincher says:

            And I never denied that boardgames have a system of notification. Also, less smarm in your replies, please. Your last post read like a nasally imitation.

          • 4026 says:

            Hm. I may have misread between the lines of your initial comment, then. My apologies.

            It’s a fair point, to be sure, that this article could have broadened its horizons beyond a single (musical) system of notation, and taken into account all the others being mentioned in the comments to support its thesis that notation systems for games are possible, necessary or even desirable. In particular, oblique references to “story boards” as a system of notation are unhelpful (and poorly explained), while considering boardgames would have gone some way to bridging the huge ludic/narrative divide.

            Relatedly, I find the whole “ludic/narrative dichotomy” thing to be at least as tedious and clichéd as people still fighting the “are games art?” debate. That guff is mindnumbing.

  16. Dervish says:

    This reminds me of the awful “verbs” method of describing games that became fashionable a few years ago. It’s adding a needless and inappropriate level of abstraction that actually makes analysis harder and descriptions more vague in favor of making it more literary and artsy-sounding (in this case, more musical and artsy-looking).

    Notating actions is a sensible idea but aping musical notation and then throwing out timing is incredibly silly. You seem aware of the clumsiness and make comments to this effect, but you still did it, and I can’t imagine a reason to do it other than because you liked the idea of it looking sorta like music notes (i.e. being artsy) which is a bad starting point for a language.

    As others have pointed out, we already have the much simpler notation of keypresses, and even piano roll-like input files for emulators. You could invent some kind of shorthand notation for in-game actions to make the files easier to read (instead of remembering what the keys do), but bottom-up transcription like that is a better starting point. If you really wanted to keep the music notation idea, you could have done it that way, with positions on the staff corresponding to various inputs and note length being locked to duration in seconds. Harder to read, but hey, that’s musical notation for you.

    I do think there’s a lot of interesting analysis that could come out of data-mining inputs for different games and genres, but it’s gotta be strict and scientific.

  17. Maldomel says:

    I’m lost. Also, I don’t think games need a notation like that. But nevertheless, this was an interesting read, even though I don’t get it.

  18. Ergates_Antius says:

    “If we developed a universal notation system for gaming, or even a set of notation systems that accounted for genre, it could make the lives of many people involved in our industry easier. It would provide a succinct means to communicate ideas for mechanics, levels and systems in a language not specific to any one department. ”

    Whilst this may be an interesting idea, or art project, I don’t see how it’d lead to anything particularly useful, or how it could make anyones life easier.

  19. Mollusc Infestation says:

    Conventions of musical notation and the need to ensure compatibility between instruments has had an enormous impact on the evolution of music. Much the same way that the development of higher level programming languages have in computing.
    While it would be useful to find a shorter form than looking at pages of code, it seems like it’s still the minimum complexity with which a game can really be adequately notated.
    On the other hand, if you are only communicating a particular strategy or even a specific run, one could argue that streaming videos are the equivalent to a modern form of notation, simply using the greater resources which are now at our disposal.

    • Skabooga says:

      A full soundwave pattern (or whatever the technical term is) could be considered the fullest and most accurate representation of a musical piece, much like the code (or video walkthrough, from another point of view) could be considered the same for a video game. Perhaps it is just my limited imagination, but it does seem much easier to condense musical works than video game works, not that that should stop those who like from giving the latter a shot.

  20. waltsontanaka says:

    As gamer and musician I found the idea pretty amazing, and the use of musical notation for a first example, although not completly precise, very successful; as I was reading the article it seemed to confusing the idea, but after the description of the stage seeing the notation I could really see the meaning of each symbol right after any explanation, with clear instructions.

    Of course, it’s “mechanical”, it only describes your actions (like a fighting game combo guide) and don’t tell (at least with this notation) what the stage may look like (although you may be able to deduce some parts of it for the kind of actions you take), but still it’s a clever idea that you may transcribe it to a short form that may be shared in a much easier way between gamers, and plataformers will probably be the first to benefit from it. Even now I can start imagining Super Mario Bros stages like this, and it’s really amazing.

  21. tsvagabond says:

    Very cool.

    But I think the better notation system lies actually in Feynman diagrams.
    It may seem crazy, but while they were originally constructed to describe sub-atomic interactions, they have been further extrapolated into a language to describe any complex physical interaction.

    In a sense, they manage to take rigorous mathematical formalism and translate it to easily interpretable pictures.
    For example: instead of a diagram representing an electron interacting with a muon via a photon, it could represent a player interacting with a zombie via a shotgun.

  22. Skabooga says:

    So, we could make all the goombas b flat?

  23. liljamie says:

    I think the real application is not in describing platform games as is, but rather as tool for designing platform games. By using notes, you now have a way of abstractly visualizing movements both for yourself and to share with others – all without having to compile or share code.

    I guess for me the better question is not how can we think about games as music, but rather how transforming games into music allows us to better design and implement games.

  24. Hoaxfish says:

    Probably coincidentally, Diablo Hardcore Inferno got beaten this week (pre-difficulty reduction): http://www.youtube.com/watch?v=YaDA7GxAtXM

    It’s probably a good compliment to this, as both players are incredible calm as they go through the motion. It’s probably a good point that they are simply following a “predictable course” as you might do with a platformer, but are essentially compartmentalising the whole thing.

    They have a rough plan, their pre-configured character choices, equipment, but at various points they’re relying on how Diablo is acting in order to determine what their current moves should be, rather than taking the lead themselves.

  25. Masem says:

    Time to get Douglas Hofstadter to update his book to “Godel, Escher, Bach, and Blow”

  26. Harbour Master says:

    I think Raph Koster’s attempts to discern a notation for game structure are probably more effective (talk from 2005) – http://www.theoryoffun.com/grammar/gdc2005.htm

  27. Raiyne says:

    I think the idea of this transcription is kind of pointless.

    For the most part, I think, the map/level is the established harmonic structure / chord progression, and the player is merely playing or ‘soloing’ over it, most of the time, improvising. In the case of platformers, the levels keep the player in check, with the certain required actions to overcome specific obstacles, but the pace and method with which the player executes his commands varies from playthrough to playthrough. Same thing goes for all single player games with set levels.

    Speed runners and perfectionist min-maxers will have specific instructions and actions to follow, but for most people, I believe the fun is in the improvisation, and that applies to all sorts of games, not just platformers.

    When it’s multiplayer, or procedurally generated content, then it’s all about the improvisation – it just depends on how ‘idiomatic’ the player is restricted to be playing as.

  28. Malawi Frontier Guard says:

    This whole thing comes up every couple of months, and every time I think “Here goes that musical notation for games guy again!”. But it’s not the same guy. It’s always different guys.

  29. Sauerkraut says:

    Clever article, all kinds of cool. BUT.

    Notation’s born of necessity in music – how else would you know how to twang/blow your instrument? The player’s got too many choices to make a coherent noise without thorough training and/or notation (jazz and other improv being an obvious exception).

    Almost all games, on the other hand, give an awfully limited number of choices. At any point in a platformer, you probably won’t have more than a dozen possible actions to take. Only one or two of those will be at all logical. Even sandbox games really only give you a few options at a time. So, notation’d be of little use to a player. Like other comments’ve said, some unimaginable notation might lead to rad stuff fo rdevs, though.

    I guess this is just another “Game notation’s cool but pointless” comment. Derp.

  30. Arky says:

    see also this classic by Alex Galloway / Radical Software Group:

    http://artport.whitney.org/gatepages/artists/galloway/