Spatial Audio and Adaptive Scoring: When Music Has a Place to Be

Stereo music lives between two speakers. Spatial audio gives it somewhere to be: above you, behind you, near a doorway in a virtual room. That changes what composers and mixers have to decide.

It also connects to adaptive music in a deeper way. Both rely on the final sound being assembled at the moment of listening, for one person in one place.

Objects, beds and a renderer

HOFA College's explainer on Dolby Atmos music mixing describes two kinds of signal. Objects are mono signals that can be given spatial coordinates and placed freely, with up to 118 channels available for independently placeable objects. Beds work like traditional surround, placed on a fixed 7.1.2 bus, and do not move.

The key is metadata. Positions travel with the mix as coordinates, and a renderer turns the individual signals into a playable result. HOFA notes the renderer can generate multiple distribution formats from a single mix, whether headphone delivery or theater configurations.

A mix that adapts to the room

This is a quiet form of adaptation. The mixer does not make one fixed file per speaker layout. They describe where sounds belong, and the playback system works out how to deliver that on whatever it has.

It does not respond to the listener's behavior the way game music does. But the principle is related to what we describe in stems, remixes and listener control: keep the parts separate for as long as possible, and decide at the end.

Head tracking: sound that stays put

Apple's support page describes Spatial Audio as creating a "theater-like environment with sound that surrounds you." It offers a Fixed option, which turns on Spatial Audio without head tracking, and a Head Tracked option, which turns on both, so audio sounds as if it comes from your iPhone or Mac.

With head tracking, turning your head changes how the sound reaches each ear while the source stays where it was. The music is now a thing in a place, not a stream inside your skull.

Winifred Phillips on scoring for VR

Composer Winifred Phillips addressed these questions at GDC 2018. Her session abstract covers "3D versus 2D music implementation, and the role of spatialized audio in a musical score for VR," along with diegetic and non-diegetic music and techniques that blur the two.

In her Game Developer write-up, she cites research from the University of Hull that "spatial treatment had no effect on the emotional impact and enjoyment of music." That is an honest caution: placing a sound in space does not make it more moving.

The hybrid answer

Phillips argues for hybrids rather than a single choice. She says 3D elements help connect a score to the world, can grab attention when needed, and can create drama when music moves from 2D to 3D.

Two examples show how. In Fail Factory, a heavy machinery impact that existed only in 3D felt "disconnected from the rest of the 2D score," so she combined a spatialized impact sound with a metallic clang in the stereo music mix. In Dragon Front, an epic main theme played directly in the headphones, then moved to occupy a position in the environment "as if it were issuing from in-game speakers" in the hub area.

The second example is adaptive scoring in a small way: the same theme changes its relationship to the listener as the situation changes.

Spatial audio as an access feature

Spatial cues are also an accessibility tool. Xbox's audio guideline lists spatial audio as a way for players to determine direction through sound alone. Our article on accessibility and adaptive music covers that side.

For music makers, the practical advice is to treat spatial placement as one expressive choice among several, not a replacement for a strong song. A well-written melody holds up in mono, in stereo and in space.

What to take away

Three ideas connect spatial audio and adaptive music. Keep sounds separable. Decide how each part relates to the listener. And let the delivery system do the last step of assembly.

For more on how those ideas play out in headsets and live spaces, see adaptive music in VR, AR and immersive experiences.

Objects and stems share an instinct

An Atmos object and an audio stem have something in common: both keep a sound separate so a later step can decide what to do with it. A stem lets a producer rebalance or remix. An object lets a renderer place the sound for a specific playback system.

In both cases, the earlier decision to keep things apart creates freedom later. That same freedom is what lets adaptive music add, remove or reposition a layer when the situation calls for it.

For writers and producers, the habit to build is to think of a song as a set of parts that could each be handled separately, even if the final result is a single stereo file.

What head tracking changes for adaptive music

Once sound is anchored in space, the listener's movement becomes a possible input. Turning toward a sound can make it clearer. Walking toward a virtual source can make it louder and closer.

For game and installation composers, that adds another dimension to a score: not only what plays, but where it seems to be, and how that relates to the listener. Phillips's examples show that a thoughtful mix of placed and unplaced sound can serve both realism and emotion.

At home, most listening will stay simple. But the habit of asking where each part belongs is valuable even when the result is plain stereo.

A quick test for any mix: listen on headphones, on a phone speaker and on a small system, and ask whether the song still works. If it does, the writing is carrying the weight, and spatial treatment is a bonus.

Sources

• HOFA College: Music in Dolby Atmos, how does 3D mixing work?
• Apple Support: Control Spatial Audio and head tracking
• GDC Vault: Music in Virtual Reality (Winifred Phillips, GDC 2018)
• Game Developer: Composing Video Game Music for Virtual Reality, 3D versus 2D (Winifred Phillips)
• Xbox Developer Docs: Xbox Accessibility Guideline 105, Audio