Adaptive/Dynamic Music

[Splynter]

I've been thinking a lot lately about how games have approached music and sound with regards to their interactive nature. I'm referring to so called Adaptive or Dynamic Music: musical scores and sound effects that change in tone, style, mood, etc. depending on cues and states in the game world. Some simple examples would be the switch to a higher tempo version of a track when approaching the time limit in a Super Mario game, the fade into and out of combat music in too many games to mention, or a change to a muffled, somewhat garbled version of a track on diving underwater.

The first game in which I truly noticed this effect was Super Mario Sunshine. In this clip, http://www.youtube.com/watch?v=ir1Z25IATDk the player starts out riding Yoshi. Notice the bongo (?) beat that plays. When the player hops off Yoshi, the bongos fade away, and when Mario gets back on Yoshi later, the bongos are layered back in. The change in immediately notifies the player that the game state has changed, and the specific 'flavour' of the new layer highlights Yoshi's character.

The game that uses this adaptive approach to music the best in my opinion would be Portal 2. In this video, you can see how using a jump pad adds cool, electronic noises to the backing music: http://www.youtube.com/watch?v=YFekhE70vEw
Now, Portal 2 is generally fairly subtle with the backing music, and it's mostly just ambient sounds. The jump pad beats highlight action moments. Other puzzle elements emit their own distinct sounds, as well, such as the light bridges.

For an example of the sound effect side of things, consider the most recent Zelda game: Skyward Sword. Your attacks emit the usual sounds when they connect or miss, whooshes and smacks you'd expect a sword to make. However, there's also a layered in musical note. If you watch this battle clip you'll see what I mean. http://www.youtube.com/watch?v=UtckgYN_T-Q
Each successive attack plays another musical note in a sequence. This is great, and really accentuates any attack combo in a musical way. If you listen to the mix though, there doesn't seem to be that much of a connection between the sequence of notes played for attacks and the background battle track, though I could be mistaken. If so, I'd call that a missed opportunity. If successful attacks played notes that blended seamlessly with the background track, I think the effect would be quite cool. Granted, that wouldn't be all that easy, considering the system would have to be able to choose a note matching the current key. You could experiment with things like a preset modality, or maybe something like a pre-baked timeline of dominant tones with the backing track, to do so.

I'm not really sure that I have a point beyond 'this stuff is cool, and only video games can really do it properly'. Surely there are lots and lots of things that can be done with dynamic audio to affect player mood and highlight things without having to rely only on visual cues. It seems to me that, unfortunately, the majority of commentary or criticism of video game audio tends to center around how realistic guns sound, or the way explosions distort other effects- valid and important points, sure. Anybody else have examples of games that have used this technique to great effect, or things you'd like to see done in the future?

[Smashbox]

You might also enjoy this old related thread, which I found interesting:

http://www.rockpapershotgun.com/foru...ility-in-Games

[Splynter]

You might also enjoy this old related thread, which I found interesting:

http://www.rockpapershotgun.com/foru...ility-in-Games

Oh yeah, I remember that thread. Hell, I even posted in it. Shows how good my memory is. The discussion there wasn't that focused on the sound side of things, however. Also, looks like it derailed pretty quickly.

[fiddlesticks]

The earliest example of dynamic music I can think of is Monkey Island 2: Le Chuck's Revenge, which used a system called iMUSE. It was developed by Michael Lands who thought it up whilst creating the soundtrack for the original Monkey Island. He wanted the transitions between musical themes to be very smooth, so that the soundtrack would sound like one continous theme rather than various different ones. To achieve this, the game would subtly alter the music depending on the scene you were currently in. For instance, in the graveyard section the music you hear while standing outside the actual graveyard is slightly different from the one you hear inside of it or the one you hear inside the tomb. It's difficult to describe in words, so here are two Youtube videos showing it off.

From what I remember, iMuse was used in most of the following Lucasarts games as well, though at least according to this review, it's sadly absent from the Special Edition of MI2.

[bad guy]

Here is a nice making of music from Red Dead Redemption.
My link takes you to the adaptive music part, although the whole video is certainly worth a look:
https://www.youtube.com/watch?v=IJAIFmOdB00&t=3m16s

[LTK]

A few good examples I can think of, off the top of my head, are FTL, where every ambient theme has a corresponding combat theme that follows the same... key? Chord progression? I don't know much about music. Let's just say the ambient and combat themes share a lot of elements. I thought they executed the dynamic music extremely well.

Another one is Fly'n, which doesn't have dynamic music per se, but when you change vision (or dimension), the background music becomes somewhat muted, shifted towards the low tones, more bass-heavy. I quite liked the effect that this gave: it makes the 'subtle' dimension feel more... thick. Or introspective. Focused. It feels like your character is shifting its attention to its other senses. Aside from that, what it also does is add more melodies to the ambient music once you pass a certain point in the level. This has no apparent relation to the gameplay, but it serves to spice things up a bit, which I also appreciate.

What bothers me about dynamic music sometimes is that in some games you can use the music as an environmental cue. This was particularly obvious in the Witcher 2: when you were walking in the forest and you heard the combat theme start up, you knew that somewhere there was an Endrega crawling down a tree, and that totally killed my suspension of disbelief. Ideally, dynamic music approximates your mental state, but doesn't dictate it.

[soldant]

For a blatantly obvious one, the iMUSE soundtrack with TIE Fighter.

[Chaotic Kitten]

I love the feeling that the music can provide in games when its dynamic in a more subtle way than, say, Final fantasy 7, it feels so... words.. >.> Good. I like it.

We are taking a new(I think?) approach to this concept, all of the monsters in Legend of Dungeon are going to have their own track, so the music will change depending on what monsters are nearby, and amplify with multiple similar monsters. It's not implemented yet, and I can't wait to hear how boss monsters with minions are going to sound!

[Gwilym]

Portal 2 is definitely the best. I haven't heard sound effects treated so musically since Full Metal Jacket. That's a long wait. And I loved how some of the musical elements came from within the world, as if they were sound effects... absolutely brilliant.

I also liked Riddick's slapdash solution, which was to run three tracks simultaneously and just crossfade between them. They all run at the same BPM and use the same chord changes, which is very easy to hear when you compare the tension and action tracks, but more subtly done when you compare those to the ambient version. Because combat was fairly swift and decisive you'd get the effect of the music suddenly rising to a climax before settling back down. But if you found a way to drag out combat you'd get to hear the action track in its entirety, which was kind of hilarious since it was just one long 'loud part' with no downtime at all (Call of Duty, basically).

Moving away from PC games, Metal Gear Solid 2: Substance had a nice music test that let you control each track's adaptive elements - which were admittedly quite primitive, and not particuarly adaptive, but still fun to play with despite the narrow confines.

Actually, Metal Gear Solid 3 had some really good adaptive music during boss fights (The Fury was particularly well done), but the adaptation would take slightly too long, kicking in a few seconds too late to be truly effective... or even noticeable, really. A real shame because it was quite well done.

[Patrick Swayze]

All the Bungie Halo games have had wonderful dynamic music that blended in such interesting ways. Marty O'Donnell really is a genius.

Halo 4 on the other hand just plays music on a loop and and sticks out like a sore thumb dreadfully. Real shame because the soundtrack is good, but Halo, I just expect more.

[thekev506]

The only real downside to dynamic music is it can be a little restricting in terms of key and tempo, but if the actual music is of a good enough quality this isn't really much of a problem to begin with.

Something I really love is good use of effects processing; it's usually unimposing and simple but it makes such a huge difference. Really obvious example is the use of EQing in Mass Effect 3 when you visit purgatory, so that when you're outside the club the music is muffled and boomy, but as you walk through the doors it brightens up, as it would in real life.

[Velko]

Semi-relatedly, when you start Crusader Kings 2 there's this menu music (the game has, by the way, awesome music overall). When you get to the part where you select which kingdom/duchy/whatever to play, a click on the map plays a note. A note from the theme of the menu music, which is playing on the background. You can actually play along the menu music by clicking around the map, if you click in tempo! It's a thing I have to do every time I start a new CK2 game, to play along for a while.

[Jesus_Phish]

One of the worst things about dynamic music is it tells you when everything is going to be ok. Case and point, Dead Space. Whenever you get attacked by any necromorph the music suddenly ramps up, the strings come in and it's all hands on deck in the orchestra pit. Seconds after you kill the last necromorph of the battle the music ends with a very sudden dud noise which is like hearing a reminder that "everything is ok now, you can relax, the threat is gone". Gears of War is equally as guilty, letting you know each and every time when the bad guys are all dead and you can stop looking for them. Nothing really takes you right out of it like being told "it's all cool!".

Bastion did it well when it came to dynamic music. The music really set the pace of some scenes, in particular both songs with lyrics.

[GameCat]

System Shock 2.

It have layerd music that adapts to situation, so it doesn't even have "correct" arrangements.

[Patrick Swayze]

One of the worst things about dynamic music is it tells you when everything is going to be ok. Case and point, Dead Space. Whenever you get attacked by any necromorph the music suddenly ramps up, the strings come in and it's all hands on deck in the orchestra pit. Seconds after you kill the last necromorph of the battle the music ends with a very sudden dud noise which is like hearing a reminder that "everything is ok now, you can relax, the threat is gone". Gears of War is equally as guilty, letting you know each and every time when the bad guys are all dead and you can stop looking for them. Nothing really takes you right out of it like being told "it's all cool!".

Bastion did it well when it came to dynamic music. The music really set the pace of some scenes, in particular both songs with lyrics.

Yes! It'd be nice to be kept on edge a bit longer!

[coldvvvave]

Portal 2? That Laser theme and Gel themes were very good, I spent more time in those areas just to play around with music a bit.

[RogB]

For a blatantly obvious one, the iMUSE soundtrack with TIE Fighter.

which they kind of ruined when they released one with 'new fangled' cd audio :(

[Sparkasaurusmex]

Overall I like dynamic music and think it will be utilized to even better effect in the future.

One negative is in games like Elder Scrolls the music can give your character a sixth sense of when a sneaky enemy is attacking.

[Jesus_Phish]

Your right, TES has a terrible way of doing it. The moment there's a hostile threat in comes 1 of 4 battle songs. The moment it's over it goes. I still rememeber the walk in Morrowind to Balmora, the music changing constantly as Cliff Racers would attack, die, attack, die, attack, get stuck and go away, attack, die. You'd be lucky to hear the same audio track playing for more than a minute.

[Internet]

I'm surprised nobody has bit trip runner here (haven't played the others). In Bit Trip games the every action has a musical cue, so you get a better song when you play better. Also the music itself gets fuller the better you play. I'm not an audiophile at all, but the effect is quite powerful.