Frequency Domain Is Your Brain On Synesthesia

By Nathan Grayson on June 1st, 2013 at 9:00 am.

Music is the epitome of sonic pleasure, so naturally, we’ve spent eons trying to see it because we will never be satisfied. Games are no stranger to that pursuit. The likes of Audiosurf, Symphony, and their ilk have interpreted music through the lens of various genres, but Frequency Domain might be the most direct attempt I’ve seen. The goal? To turn every aspect – each pulsing beat, humming strum, and snaking synth – into boundless neon volcano dance floors of rolling color. You speed along – a blur, a performer, an emotion – ramping off jutting spikes of sound and then cannon-ball colliding into the senses-engulfing tidal wave of music below. Sound dips in and out, other melodies soar about like dragonflies. Is Frequency Domain a “game” in the traditional sense? I don’t care. It gave me goosebumps, and you should absolutely try the free, four-song demo.

Sadly, the demo doesn’t feature any Daft Punk, but the available songs are still quite good – especially the fourth, in my opinion. You can shift between them by hitting the number keys. Also enjoyable: holding down space for speed that literally outruns sound. I ended up tapping it to each song’s beat, taking the game from surreal dreamscape dance floor to surreal dreamscape dance floor after having way too much to drink.

As of now, Frequency Domain’s far from complete, but the plan is to keep it free even once it’s crossed the finish line. Because each song requires a human touch to get it up and running (and soaring and gliding and tip-toe pirouetting across mountaintops), the initial release will feature a pre-selected set of songs. Eventually, however, players will be able to add and upload their own.

“I’m trying to expose every interesting part of the song to the player, parts that often get lost when you only have an algorithm attempting to analyse a song (often dismisses quieter frequencies as noise, etc). Later this year, my plan is to release a version which exposes the design tools I use in Unity to the players in-game, allowing them to load their own song and craft their own unique experience for any track. Depending on how people approach it a single song could have multiple unique interpretations!”

Exciting! Frequency Domain is already quite an impressive audiovisual experience, so I’m very much looking forward to seeing where it ends up. It still needs a little work (I managed to get stuck in the floor, leave the level entirely, etc), but there’s potential for something pretty special here. It probably won’t take over my life or anything, but I’m all for a quick jolt of synesthetic bliss every once in a while. Godspeed, Frequency Domain. Godspeed.

, , .

17 Comments »

  1. Jakkar says:

    That was… Well… Remarkably dull. The choice of tracks which take up to a minute to conclude the minimalistic intros was perhaps not the best idea – nor the woefully narrow field, or the lack of roll controls on the mouse.

    Unsure what they hope to achieve with this demonstration/trial, it seems too soon for public exhibition.

  2. Premium User Badge Rikard Peterson says:

    Interesting. Unlike most things of this nature (particularly those where everything is automatically generated), I can see this working just as well with different genres too.

    The Mac build isn’t packaged correctly, but you can get it to run if you secondary-click on the app, choose “Show bundle contents” and then run the “FrequencyDomain_v0_3_PublicFeedbackBuild1_Mac” file.

  3. vivlo says:

    this is, basically, a fly through 3D-FFT analysis of a song.

    • LionsPhil says:

      Yeah, that quote in the article doesn’t make a lot of sense, because that basically is a Fourier transform.

      A pretty lazy one, too, just generating the terrain as the sound streams, so the shapes the player is surfing on are actually a split-second behind. I suspect it would be more interesting to generate further ahead, so you can see big dynamics coming, and put the player actually on point in time being played.

  4. Lanfranc says:

    So how does this actually work? It seems like the songs have to be adapted to the program in some way? And are presumably distributed along with it?

    If that’s the case, I predict this will lead to some vaguely unpleasant letters from copyright lawyers sooner rather than later.

    • smoozles says:

      The dev is looking for artists to distribute with the game. A version that plays all music-files will come later

      • LionsPhil says:

        The dev is looking for artists to distribute with the game.

        Tame artists with every copy? Oh man, I am going to make them draw so many things.

      • Lanfranc says:

        So not Daft Punk, then, presumably,

        The “players will be able to add and upload their own” part could still be a bit of a nightmare.

        • Premium User Badge Rikard Peterson says:

          I assume the uploading wouldn’t include the actual audio. Then the lawyers wouldn’t have any issues.

  5. Muzman says:

    So it’s kind of one of those play along visualisers like The Polynomial (albeit using a completely different system and approach)

  6. SuicideKing says:

    Couldn’t get past half the video. Sort of weird. Initially thought you’re a creature that can only use sonar navigation.

  7. sagzorz says:

    Hey everyone, I’m the dev! I just woke up to find out about the article, this is awesome!
    I’ll try to address some of the issues:

    - Sorry about the lack of mouse control, I hadn’t thought of it! I’ll try to add it to the next update.

    - About the word “upload”: what I meant is “load” your songs into the game. I don’t have any video upload mechanism planned (it would be nice though!).

    - Absolutely correct about it being a lazy 3D FFT. The sound amplitude values are generated at the front edge (horizon) and trickle their way down. The exact moment the sound “hits” you depends on where you are in the field (you can move both in width and depth axes).
    As for why I’m not generating it further ahead as you suggested, there are some precise design reasons for it.
    I’d love to discuss this more if you want, though I highly recommend using the discussion section of FD’s Steam Greenlight Concepts page
    It’s a centralized place for discussing the game, so everyone can gain from it !

    - About the Mac build: I’m on Windows so I can’t test locally. But I remember putting the .app executable into a zip file (along with the README). Did the unzipping fail? Please let me know if that’s the case, it wouldn’t be the first time.

    • vivlo says:

      hah, don’t get me wrong there : i’ve always thought it would be wonderful to fly through a 3D fft ;)

    • Premium User Badge Rikard Peterson says:

      What’s wrong with the Mac build is that you put the app (and the readme file) in a folder with an .app extension – that causes the OS to treat the folder as an app bundle instead of a folder. Just rename the outermost folder, and it’s alright.

  8. 12kill4 says:

    Played with this on my friend’s Oculus Rift yesterday; was quite interesting, but prone to sending you flying off into the black abyss, unable to return to the music and lights…. Can’t say I was particularly impressed when things were working properly though…

    • sagzorz says:

      Sorry about the quality of the Oculus build, it was my first time at it and I just hacked it up in an afternoon . I might be able to work a devkit again soon, I definitely want to make it better!

  9. MrKap says:

    I loved those guys ever since rolling and scratching on homework! Anyways, synethsia if I got that, well it’s probably minimal and strongly related with asmr, see http://www.asmrstudio.com