What Is Interactive Music? A Plain-Language Guide

Ask five people what interactive music means and you may get five answers: a video game score that swells when a fight starts, an app where you tap the screen to make melodies, a song you can remix, or an ambient stream that never plays the same way twice. All of them are fair. They share one idea: the music is allowed to change after it leaves the studio.

The single word hides real differences, though, in who makes the decisions and in what the composer actually writes. This guide sorts the field into three families, shows where each came from, and explains why the distinction matters for listeners, for creators and for anyone licensing music.

The umbrella term: dynamic music

Game audio researchers have worked through this vocabulary for years. Karen Collins, in her book Game Sound (MIT Press, 2008), uses dynamic audio as the umbrella and defines it as "audio that is changeable, a broad concept that encompasses both interactive and adaptive audio." Under that umbrella she separates sound the player controls directly from sound that reacts to the state of the game.

That split is the most useful starting point. Add a third family for music made by rules, and you can place almost any interactive music product on the map.

1. Listener-controlled music: you make the change

In listener-controlled music, a person makes a deliberate musical choice and hears the result right away. Muting the drums, switching to a different arrangement, changing the key, tapping in a new note: the listener is acting a little like a performer or a producer.

A well-documented early example on phones is Bloom, the app Brian Eno and Peter Chilvers released in October 2008. Touching the screen adds notes, and the patterns then repeat and slowly mutate. Create Digital Music's hands-on review at launch described exactly that loop of touch, repetition and change.

Switching production styles on a song is also listener-controlled. On Starchild, you can move a song from a country production to an electronic one while it plays, and the vocal carries on from the same spot. The listener chooses and the song responds.

2. Adaptive music: the context makes the change

Adaptive music changes in response to signals the listener is not consciously steering as music: the state of a game, where you are standing, how fast you are running, the time of day. You influence those signals, but you are not picking the music directly.

Video games are the home turf here. A score that moves from exploration to combat, or adds layers as a level gets harder, is adaptive. Audio middleware such as Audiokinetic's Wwise and Firelight's FMOD exists largely to make this practical, letting composers define segments, layers and transition rules that the game triggers while it runs. A post on Audiokinetic's blog by sound designer Chenzhong Hou draws the same line Collins does, sorting music design methods into user-driven and game-driven categories.

Adaptive music also shows up outside games: in fitness apps that follow running cadence, in albums that change as you walk through a city, and in installations that respond to a room.

3. Generative music: rules make the music

Generative music is built from a system instead of a fixed score. The composer writes rules, sounds and probabilities, and the system produces the actual notes, often differently every time it runs.

Brian Eno popularized the idea in a talk at the Imagination Conference in San Francisco on June 8, 1996. He described thinking of "a system or a set of rules which once set in motion will create music for you," and admitted that putting on a record and knowing he would hear the same thing as last time had become "a little bit irksome."

Generative systems can be interactive or adaptive too, which is where the families start to overlap. Bloom is generative, since its patterns mutate and the app keeps playing on its own when left idle, and it is listener-controlled, since your taps seed it.

How the three families compare

• Who decides: in listener-controlled music, the listener; in adaptive music, the context or system state; in generative music, a set of rules.
• What the creator writes: complete parts or versions; parts plus rules for when to use them; rules and raw sound material.
• What stays fixed: usually the composition and its recordings; the composition, with arrangement and timing left flexible; often very little, since the notes themselves can change.
• Typical examples: style switching, stem players and remix apps; game scores, pace-matched running music and location-based albums; ambient apps and algorithmic soundscapes.

Why the difference matters

For listeners, the families describe how much agency you get. Some people want to steer, some want music that quietly fits the moment, and some want something new every time.

For songwriters and producers, they describe the job. Listener-controlled formats need complete, polished versions that line up with each other. Adaptive formats need parts that can be recombined at musical boundaries. Generative formats need rules and sound sources more than finished takes.

For licensing, they describe what is actually being licensed. A fixed song offered in several productions is still one composition with a clear writer. A generative system raises harder questions about what the work is and who wrote any given output. That is one reason Starchild labels each song for AI transparency and keeps the human-written composition at the center while the production around it changes. For the longer version of that idea, see what an adaptive song is and the humanistic case for adaptive music.

Where to hear each kind today

Listener-controlled music is the easiest to try: any player that lets you switch arrangements, change key or mute parts counts. You can hear one song in many genres on a Starchild song page and notice how the vocal holds its place while the production changes underneath it. Adaptive music is most familiar from games, where it often works so quietly that players never notice it. Generative music lives in ambient apps and installations. Once you know the three families, you start spotting them everywhere, often inside the same product.

Sources

• DiVA: The Effect of Dynamic Music in Video Games, An Overview of Current Research (L. F. Smith, 2020, quoting Karen Collins, Game Sound)
• In Motion Magazine: Brian Eno, Generative Music (talk, June 8, 1996)
• Create Digital Music: Hands-on with Bloom, new generative iPhone app by Eno and Chilvers
• Generative Music: Bloom
• Audiokinetic: About Dynamic Music Design, Part 1: Design Classification