Ever noticed how, when you're deep into a game, you hear all sorts of sounds, such as footsteps, explosions, and ambient noises, all of which seem perfectly timed to your actions? Most of these sounds are pre-recorded audio samples made by talented sound designers.

These small snippets of audio also come in many forms, including one-shots, like a single gunshot, or loops, like the constant hum of a spaceship's engine. However, what you may not have known is that not all sounds in video games come from pre-recorded samples. Some sounds are created on the fly right as you play the game.

This is what we call procedural audio , which I like to think of as a game’s personal little composer. With it, we're able to create sounds dynamically during gameplay without relying on pre-rendered audio files. This means the sound you hear when a character steps on gravel might be slightly different each time, making the game feel even more immersive and realistic.

In this guide, we're going to take a look at all there is to know about procedural audio, including its history, how it works, and some cool examples of how it's used in modern games.

So, whether you're a budding sound designer or just curious about the tech behind your favorite games, stick around! We've got a lot to cover.

What Is Procedural Audio?

Procedural audio is a fascinating aspect of game sound design. The idea is to create sound at runtime or generated on the fly, right as you play the game, rather than being pre-recorded and played back.

In layman's terms, procedural sound design creates sound effects based on pre-determined behaviors. Think of it as a system that knows how to generate a footstep sound whenever your character walks on different surfaces without needing a pre-recorded sample for each step. Instead, it synthesizes the sound in real time, making each footstep slightly unique.

This technique is similar to the procedural generation used in other parts of games, like environmental art and level design. Just as a game might create a new forest or dungeon layout each time you play, procedural audio makes up soundscapes you hear based on the game's current state and your interactions.

By using procedural audio, sound designers can create deeply integrated soundscapes, which react to player actions and the game environment in a cohesive and believable way.

However, there are trade-offs to using common procedural sound design techniques.

One major challenge is the complexity of ensuring the sounds are of high quality and realistic. Creating convincing procedural audio can be more technically demanding than using pre-recorded samples. Additionally, it can require more CPU, which might impact the game's performance, especially on lower-end systems.

A History of Procedural Sound Design

Back in the early days of gaming, procedural sound design wasn’t just an art form. It was a necessity for sound. The limited RAM in early gaming systems couldn't handle the storage demands of pre-recorded audio samples, the alternative to procedural audio. This constraint pushed developers to generate sounds in real-time as the game played.

In-game audio began with the iconic game Pong on the Magnavox Odyssey in 1972. Interestingly, the original Magnavox Odyssey didn’t come with any sound at all. It was the Atari version of Pong that made history by incorporating sound. Atari achieved this by using the Television Interface Adapter (TIA) , a piece of hardware designed to handle both video and audio outputs.

The TIA could generate sound waves using two oscillators. By manipulating these oscillators, the Atari version of Pong could create simple yet effective audio, marking the first instance of procedural sound design in gaming.

In the game, there were three primary sounds, each generated procedurally:

• The first sound was the beep when the ball hit the paddles, giving players immediate audio feedback on their hits
• The second sound was a deeper pong sound when the ball collided with the walls, differentiating it from paddle hits.
• The third sound was a higher-pitched noise for scoring, signaling a point had been made.

Though basic by today's standards, these sounds added a new layer of engagement and feedback that was crucial for the immersive experience of the game.

Of course, as technology evolved, so did the complexity and quality of procedural audio.

The Evolution of Procedural Audio

In the late 1970s, procedural audio began to take shape across various gaming consoles. Three notable systems from this era were the Atari 2600, the Fairchild Channel F, and the Bally Astrocade. Each of these consoles utilized procedural audio to enhance the gaming experience within the constraints of their hardware.

Advancements in the 1980s further pushed the boundaries of game audio.

In 1983, the Vectrex introduced a new level of audio synthesis, while Nintendo made significant strides with the release of the Nintendo Entertainment System (NES) in 1985. The NES used a five-channel audio system that supported a wider frequency range from 54Hz to 28kHz and could perform pitch bends. Iconic games like Super Mario Bros. set a benchmark for procedural audio with memorable sounds such as the coin collection "ping," the power-up "mushroom" sound, and the "jump" effect.

By 1986, the Sega Master System brought an even greater evolution of game audio. It incorporated both sampling and electronic synthesis, using four audio channels (three for music and one for sound effects). The Master System was equipped with the YM2413 chip from Yamaha, the same chip used in their professional synthesizers, which significantly enhanced the quality and complexity of the sounds it could produce.

The evolution continued with the release of the Sega Mega Drive (Genesis) in 1988 and the Super Nintendo Entertainment System (SNES) in 1990. Both of these consoles introduced more sophisticated audio capabilities, including higher-quality samples and more channels for richer soundscapes.

However, it wasn't until the Sega Saturn release in 1994 that we got to see another major milestone in the evolution of game audio. It featured a sound chip and a sound processor capable of supporting up to 16 channels of audio with 44.1kHz CD quality, setting the stage for the high-quality sound we expect in modern gaming.

Pre-Recorded Sound Effects and Music

In 1994, the gaming world experienced a monumental shift with the release of Sony's PlayStation. This console introduced a significant upgrade in audio capabilities, offering a 44.1kHz sample rate and 24 channels of stereo audio. The PlayStation's sound chip was a game-changer, allowing for reverb effects and looping.

With newfound flexibility, composers and sound designers could create more complex and immersive audio landscapes to enrich each player's experience.

Before the PlayStation era, creating audio for games required a deep understanding of audio programming and procedural audio. Those making the sounds needed to be well-versed in complex coding and signal processing to generate and implement sound effects and music. This made the process very labor-intensive and often restricted the creativity of those who were more musically inclined but less technically skilled.

In many ways, PlayStation revolutionized this process by allowing pre-recorded sound effects and music to be easily implemented into games. Composers and sound designers no longer needed to worry about the intricacies of procedural sound. Instead, they could focus solely on creating high-quality sound effects and music, which they would then pass on to the developers for integration into the game.

Is Procedural Audio Out of Fashion?

Despite the rise of pre-recorded sound effects and music, procedural audio is far from out of fashion. Many post-PlayStation games continue to leverage the mathematical models of procedural audio. Let's take a look at some of the most popular.

Modern Games That Use Procedural Audio

Spore

In the groundbreaking 2008 game Spore, audio programmers Aaron McLeran and Ken Jolly employed advanced procedural audio techniques to create a dynamic and immersive auditory experience.

They used an adaptation of Pure Data called libpd, an embeddable audio synthesis library designed to integrate Pure Data's powerful capabilities into other applications. Pure Data, for those who don't know, is an open-source visual programming language for multimedia, which is used extensively in the creation of interactive computer music and audio.

Libpd allowed the team to generate music and environmental sounds based on infinitely changing variables within the game. For instance, as players created and evolved their creatures, the sounds these creatures made were generated in real-time, reflecting their unique characteristics and behaviors.

This use of procedural audio ensured that each player had a unique and personal experience with the game.

No Man's Sky

No Man's Sky is another prime example of how procedural sound design can create a rich and dynamic game world. The development team faced the unique challenge of crafting a soundtrack that could adapt to the game's procedurally generated universe. With most of the game's assets, including planets, ecosystems, and even creatures, being algorithmically generated, a traditional pre-recorded soundtrack would not suffice.

To meet this challenge, the team at Hello Games employed Wwise audio middleware, specifically a custom plugin known as VocAlien. This tool was essential in synthesizing vocalizations for the game's diverse and unique creatures. VocAlien generates sounds based on the characteristics of each creature, such as their size and type, ensuring that each sound is appropriate and unique.

The procedural audio system in the game also allowed the creators to essentially 'perform' sounds. This means that the soundscapes created are not static recordings but dynamic pieces of audio that change in real-time based on the creatures' base animations and behaviors.

As a result, the sounds you hear while exploring are intricately tied to the on-screen actions and environmental conditions.

Elite Dangerous

Elite Dangerous, the 2014 mega-hit online sci-fi space exploration game, set a high bar for immersive audio in gaming with its use of procedural sound design. The game's developers employed procedural techniques to create dynamic and adaptive sounds, particularly for spaceship engines and graphical interfaces.

Mini Metro

Mini Metro is a minimalist subway simulation game from 2015 that uses procedural audio to create an adaptive and engaging soundtrack that enhances gameplay. The developers, Dino Polo Club, aimed to integrate procedural music from the beginning, leveraging the strengths of procedural techniques to match the dynamic nature of the game.

Rich Vreeland, the composer, used a procedural music system that responds to the player’s actions and the evolving subway system. Each city in the game has its own set of musical qualities, such as rhythms and harmonic choices, which change dynamically based on how players build and alter their subway lines.

Just Cause 4

In Just Cause 4 , the developers used procedural sound design for the whoosh sound effect generated when the player passes an NPC vehicle in traffic. This effect is created using runtime synthesis from FMOD audio middleware.

This whoosh sound effect was synthesized using a blend of white and brown noise. White noise contains equal intensity at different frequencies, creating a consistent hissing sound, while brown noise has more energy at lower frequencies, producing a deeper, softer sound.

By mixing these two types of noise at different ratios, the team was able to vary the output of the combined sound based on several in-game variables, such as the distance to NPC vehicles, the speed of those vehicles, and the speed of the player's vehicle.

With this approach, the sound effects in the game engine were able to adapt dynamically to the player's actions and the environment.

Final Thoughts - Looking to the Future of Procedural Sound Design

Procedural sound offers immense flexibility in modern gameplay. Sound designers can create dynamic and adaptive soundscapes that respond to player actions and environmental changes in real time, making sure each player's experience is unique while enhancing immersion and engagement by providing sound effects tailored to the specific context of gameplay.

However, despite procedural audio's advantages, sampled audio remains the gold standard for achieving the highest fidelity and realism. Pre-recorded samples capture the nuanced details and natural characteristics of real-world sounds, providing an unmatched level of audio quality. Why not just program some of the million samples of sword hits available in sample libraries instead of synthesizing your own?

Looking ahead, procedural techniques are likely to continue evolving and augment traditional sound design methods. By integrating procedural sound design with sampled audio, game creators will be able to leverage the strengths of both approaches for more dynamic and realistic soundscapes.