How Does Generative Music Work?

Generative music is music made by a system that keeps making choices while it plays. Nobody presses record on a final version. You hear one run of a process that could have gone differently.

Here are the three parts every such system has, three tricks from famous examples, and a 20-minute experiment you can run in any audio editor tonight. This guide is about rule-based music. For how it differs from adaptive and procedural music, see our short guide to the three terms.

The three parts: a pool, some rules and a clock

Take any generative system apart and you find the same three parts.

• A pool of material: notes, loops or phrases that a person wrote or recorded.
• Rules for choosing: which items are allowed, how likely each one is, and what cannot follow what.
• A clock: something that decides when the next choice happens, so everything stays in time.

Brian Eno described such a system in a 1996 talk. The program he showed told a small synthesizer what to play "according to the rules that I've set." The rules covered the scale, how a note moves to the next ("big steps, or small steps") and even how a note wobbles. There were "a hundred and fifty of these kinds of rules," and all were probabilistic. In his words, "there's a greater chance that it's going to play a fifth, than a flat fifth for example." Of what came out, he said: "It is not a recording, and I have never heard it play exactly this before."

Trick one: loops of different lengths

The simplest generative system needs no randomness at all. Eno traced the idea to Steve Reich's It's Gonna Rain, in which identical copies of a taped loop play on two machines. The machines run at slightly different speeds, so "they gradually slip out of sync with one another." The sound turns into an echo, then a cannon, then "all sorts of things." Reich wrote in 1968 that "once the process is set up and loaded it runs by itself."

Eno's Music for Airports, recorded in 1978, layers tape loops of keyboard instruments and voices at differing lengths. Of the vocal piece he said, "I started all the loops running, and let them configure in the way they chose to configure."

Here is the arithmetic, using made-up lengths. Start loops of 17, 19 and 23 seconds together. All three start over at the same moment only after 17 × 19 × 23 = 7,429 seconds, a little over two hours. Until then, you keep hearing the same three parts in new positions against each other.

Trick two: weighted choices

A rule can be as small as a table of odds. Say the next note comes from four notes of one scale, with weights of 40, 30, 20 and 10 percent. The common note anchors the tune, and the rare one adds a surprise. That is the same logic as Eno's fifth and flat fifth. A musical dice game works the same way, with fixed bars and two dice instead of software (see musical dice games and the history of interactive music). Rules can also forbid things, such as no leap bigger than a fifth.

Bloom, an app Eno made with Peter Chilvers, shows the idea in your pocket. You tap the screen to make patterns, or leave it alone, and "a generative music player takes over when Bloom is left idle."

Trick three: rearranging parts people wrote

Most game music called generative works this way. Cale Plut and Philippe Pasquier write in Game Developer that in the games industry, generative music "uses random chance to arrange pre-composed and pre-recorded musical clips in new ways." They cite Ballblazer, from 1984, which made riff variations by removing or altering notes and stitched them into longer phrases. They also cite No Man's Sky, which arranges fragments chosen by the player's environment. Patrick Hutchings and Jon McCormack add that No Man's Sky's audio director, Paul Weir, described "hundreds of small carefully isolated fragments of tracks" recombined by rules built around game events.

Never repeating is not the same as interesting

A system that never repeats can still be dull. Plut and Pasquier call this the "10,000 bowls of oatmeal" problem: generative methods can give you "endless amounts of stuff that is not very interesting." The cure is human: strong material and tight rules. Parts that must fit in any order also limit what a composer can say. If the highest-energy music has to move to the lowest-energy music at any time without sounding jarring, they note, the difference between the two will be limited.

There is a second catch. Tero Parviainen's essay on how generative music works points out that It's Gonna Rain and Music for Airports were made with generative methods, but the results do not stay generative: every time you listen to the recordings, you hear the same thing. The music is generative only while the system runs.

Is generative music the same as AI music?

No, though they overlap. Plut and Pasquier define generative music as music created with "some degree of systemic automation," where a machine does the composing, arranging or performing. A table of odds that a person wrote fits that definition. So does a trained model: their own research used a Transformer model to write extra parts for an adaptive score. In this guide, a person wrote the rules.

Try it tonight: three loops, 20 minutes

• Set your editor's tempo to 60 beats per minute, so one beat lasts one second.
• Pick three short sounds: a soft pad, a simple piano pattern and a bell. Keep every note inside one scale, such as C major pentatonic (C, D, E, G, A), so any combination should sit well together.
• Trim the loops to 17, 19 and 23 beats. Leave plenty of silence at the end of each. Eno wanted "a silence at least twice as long as the sound" in his vocal loops.
• Put each loop on its own track, set it to repeat and press play. Let it run for ten minutes.
• Change one thing: add a fourth loop of 29 beats, which moves the full repeat from about two hours to about 60, or lower the bell's volume. Listen for what changes in the character of the whole.
• Write down what you like. Those preferences are your rules.

What this means for adaptive music

Generative and adaptive systems share one habit: their quality comes from material written to fit together in any order. Your three loops worked because they shared a scale. A generative layer can also listen to the world. The rules in No Man's Sky are tied to game events, so that music is adaptive as well as generative. For the human-written side of the same idea, you can hear one song switch between finished styles on Starchild's adaptive music page. To write parts that fit, see designing a song so it can adapt.

Sources

• In Motion Magazine: Brian Eno, Generative Music (1996 talk transcript)
• Tero Parviainen: How Generative Music Works, a perspective
• Wikipedia: Ambient 1, Music for Airports
• Generative Music: Bloom, by Brian Eno and Peter Chilvers
• Game Developer: Deep Dive, A framework for generative music in video games (Cale Plut and Philippe Pasquier, 2022)
• IEEE Transactions on Games via arXiv: Adaptive Music Composition for Games (Hutchings and McCormack, 2019)