A Symphony Hall for Your Head: What Is Spatial Audio and Should You Care About It?

This summer, Apple Music announced, with great fanfare, the addition of spatial audio to its streaming music platform. You might be asking yourself: Do I care?

Unless you’re a particular kind of Reddit-level audiophile or experienced music producer, your answer was perhaps, “Sounds cool, but not really.” And it probably spawned the question: “But wait, what is spatial audio?”

If that’s the general response, it makes you wonder if the release of this new tech is in fact the game changer Apple claims it is, or if it’s marketing spin to help differentiate Apple Music from so many competing audio streaming services.

Spatial audio, for Apple’s purposes, focuses mainly on giving listeners a surround-sound experience through headphones.

Spatial audio, sometimes referred to as immersive audio, is a squirrelly concept, which means it can lead to a number of misconceptions, as we’ll get into below. For Apple’s purposes, it focuses mainly on giving listeners a surround-sound experience through headphones.

“I think it is difficult to define because we are still trying to fully develop the technology that will completely translate the concepts on the consumer side,” said John Escobar, a professor in the Music Production and Engineering (MP&E) Department. “Spatial audio gives content creators a way to get one step closer to delivering material that more closely emulates the way humans perceive directionality. This means that the storytelling capacity of audio content is hugely increased. For a listener, this means they will feel like they are at the movies (in surround), or at a concert/live event, etc.”

James Donahue, a colleague of Escobar’s in MP&E, defines that “directionality” as adding the idea of “height” to a listening experience. “By adding height to things, it gives you this expansiveness; it gives you a ceiling. Before that, [the sound] was always sort of in front of us and behind us, and left and right, but it was all on a single plane.”

Misconception 1: Spatial Audio Is a Brave New World

Donahue, who’s worked extensively in the classical music world and conducted considerable research on acoustics in the Byzantine era, isn’t using “ceiling” metaphorically here. In fact, it’s right there in the name—spatial audio, audio that is configured to build a sense of physical space. This, as Donahue explained, is nothing new.

While Byzantine cathedrals are often regarded for their visual and physical grandeur, Donahue points out that the grandiosity goes beyond what we can see. “The Byzantine people thought a lot about architecture,” he said. “They thought about it in ways to enhance the music to make things more intelligible, to make it sound more reverberate and spacious.” They did this by building ceilings so that sound would bounce off the surfaces in a particular way. We now know that effect as “slap echo,” but the Byzantine people referred to it experientially: the sound of angel wings.

“It was all about immersion as a spiritual event…. So we've been thinking about [immersive audio] for a very long time. And we've been attempting to sort of capture all of this and make it available to the masses.”

Misconception 2: The Technology Is the Most Important Part

It’s easy to hear words like “spatial” and “immersive” and think, "Wow, now everything is going to sound mind-blowing." Another professor in the MP&E Department, John Whynot, points out that it’s important not to get blinded by the flashy lights of new tech. "‘Immersive’ is a word that suggests hype,” he said. “There is hype, but let's face it: if it's sound, it's immersive, whether or not there are speakers behind you, or it's from some kind of binaural trickery.”

In Escobar’s course Fundamentals of Immersive Audio for VR, students have a chance to work with these state-of-the-art tools, including a studio equipped with Dolby Atmos, the technology behind Apple’s spatial-audio offerings. Escobar is quick to point out that while the technology is advanced, it’s still just technology—a new canvas on which to create.

The true power of spatial audio is the storytelling potential. I have had several students create really impressive projects pushing the boundaries of getting the listener/viewer to feel ‘immersed.’

— John Escobar

He’s noticed that once students move past their initial sense of awe, breakthroughs start happening. “The big revelation usually happens when they realize they can place a sound anywhere in a 360-degree sphere and it will play back in a variety of systems,” Escobar said. “Again, the true power of this is the storytelling potential. I have had several students create really impressive projects pushing the boundaries of getting the listener/viewer to feel ‘immersed.’ Two student examples that come to mind are a Boeing 747 takeoff flight simulator, and an audiobook-style adventure that takes the listener through some caves, escaping mythical creatures, and avoiding falling boulders.”

Misconception 3: Making Audio Spatial Makes It Better

In the wake of Apple’s announcement, many producers and engineers have warned about the pitfalls of seeing spatial audio as a “That Was Easy” button for improving audio. It has to be applied intentionally and artfully. Donahue points out that Billie Eilish’s music, which puts a lot of emphasis on atmospheric and textural sound, works really well in spatial audio. Applying spatial audio to music that wasn’t intended for that treatment, however, can be disorienting and off-putting, as if the sounds are being thrown at you from every angle.

“We are being given tools to create art,” Escobar said. “So, it is our job to transcend the technology and make it part of our muscle memory, just like we do with any of our instruments.”

But can spatial audio truly transform your sense of space? Well, take the cases of Escobar and Donahue, whose own studios often feel worlds, and even eras, away.

“The other day I was testing a new speaker system, using a drum multitrack recording as my test source,” Escobar explained. “Once things were working, the 8-year-old drummer in me was transported back to those first times I sat behind a drum set in a dusty garage in the suburbs of Quito, Ecuador.”

Donahue said that his house was built in 1882, but you would never know that, based on the very modern home studio he built in the basement. The room continues to transform, based on the sounds he creates. “I said, ‘I’m going to build a space, and it’s my space, and it's going to do what I want it to do.’ It is my Byzantine church, that's for sure.”