Sound Design is one of the most vital aspects of a game - and while the content, genres fidelity and style of video games have altered vastly over the decades, the need for compelling sound design has not. From the insistent but sinister bounce of Evil Otto in Berzerk, to the fully realized worlds of Halo, sound design has been absolutely foundational to the fidelity, clarity of experience and the way those experiences are seared into your brain.
The sound of a Star Wars light saber, or the wailing-whoosh of a speeding TIE fighter are unmistakable. Star Wars superfans might have watched the movies a dozen or more times. But gamers will live through our sound designs for literally thousands of hours. If Star Wars has worn a groove in your memory with vital sound design, then Halo and games like it, have probably carved out a whole Death Star trench. The vroom of the Warthog, the shuddering buzz of a charged plasma pistol and the joyous intonation of RUNNING RIOT.
Robbie Elias has been art directing for your ears since 2006 and shares his insight and experience here.
You’ve worked on Halo 4 and Halo 5: Guardians, but Halo hasn’t been your only soundscape – what other games and genres have you worked in and how does the approach to those real life spaces differ from sci-fi?
When I was first starting out doing sound design for games, I was pretty lucky to land a contract gig with Microsoft on their Central Audio Team, which back then was called Sound Lab. We were a group of Audio Professionals that acted as a shared resource, supporting all the First Party games that Microsoft’s Publishing Team was working with. Depending on the game, and the resources that were available to them internally, we would provide different types of audio services including Sound Design, Audio Implementation, Field Recording, Voice Over Recordings and Audio Mixing.
While I was there, I was able to work on several titles including Fable 3, Crackdown 2, Alan Wake, Toy Soldiers 2, Microsoft Flight, Trials Evolution, Ryse: Son of Rome, Ascend Hand of Kul, Kinectimals and Star Wars Kinect. It was a great first job to introduce me to the world of game audio because I was able to develop my skills in sound design on a variety of different genres Some of my favorite memories from that time include designing all sorts of abstract and spooky ambiences for games like Alan Wake; or, driving all over the state of Washington to field record vehicle sounds for Microsoft Flight and Trials Evolution.
The approach to designing ambient sounds for real life spaces and sci-fi environments are more similar than you might think. Implementation wise, they use the same methods and philosophies (including marking up rooms and placing 3d ambient emitters). On the creative side, the content might differ drastically but there are fundamental similarities that do exist. When crafting any sound scape for a game, there is an aspect where the sound designers might try to create a ‘distant’ layer of sounds for objects that are heard but not seen to create the illusion that the world is deeper and richer than what the player is seeing on screen. This can include distant sounds of birds (or space birds), people chatting, explosions, sirens and even metal creaks or groans. These types of sounds help to make the world more believable and create the feeling that there is so much more to explore in the game.
For sci-fi environments I find it helps to ground the sounds in a bit of reality. For example, if you are trying to design a sound scape for an alien world you might want to lean on distant bird/wildlife sounds that the player is already familiar with. People are already conditioned to the sound of birds all around them in nature, so you can use that to your advantage as those sounds won’t get as annoying to them if they have to spend a lot of time in a specific area. If you went completely abstract and alien with your distant wildlife sounds, then the player might get too distracted, and possibly even annoyed, by the constant sounds playing in the environment. The notion of grounding things in reality applies to most sounds in a game (with the exception of possibly UI). You could create sound effects completely from a hardware or software synthesizer and put it in the game but you run the risk of it sounding out of place in the Audio Mix, or not have the feeling that it is believable, if you do not ground it by adding a layer of real-world sounds.
This is something that many sound designers might use when designing sounds for sci-fi weapons. They might spend a lot of time creating some other worldly sounds using a synth and then mix that with some real-world recordings of gun sounds to make them sit better (audio wise) in the game. There is a well-known audio saying that “as a Sound Designer, if you have done your job right nobody will notice the Audio is there at all.” It will become a cohesive experience with the visuals and do its job to support the experience as a whole.
Sound design has changed a lot since the monotone beeps and bongs of the Pong and Atari eras, what do you think have been the most important advances over the years – and what challenges does ever increasing fidelity bring?
Sound design has definitely changed a lot since those times and, to be honest, it would be hard to classify one specific change as being the most important. If I had to call out one of the more significant changes, it would be the advancements in audio middleware for games. This has had one of the biggest impacts on designing and implementing sounds for games. Middleware software such as Wwise and Fmod have created a starting point for audio technology so that programmers no longer have to reinvent the wheel for every game that is out there. Instead, they can start a production cycle with the framework needed for sound designers to do their work already present. The software comes with the functionality for interactive music systems, dialogue trees, vehicle engine transitions and so much more – all ready to go on day one. Additionally, they give control to the audio team to manage their resources in ways they have never been able to do previously. Some other notable advances come in the form of DAW (Digital Audio Workstation) and hardware developments.
Sound designers can now afford to own a large collection of microphones and field recorders without breaking the bank. This enables them to capture more unique sounds and rely less on the commercial libraries that everyone uses. There are more audio plug-ins out there and they are cheaper which allow sound designers to craft more unique and custom sounds helping to break down barriers that used to exist for those just starting out.
One last change that is worth mentioning is more of a social development, but the introduction of social media, such as Facebook/Linkedin groups, Slack and even Twitter, has brought the entire game audio community closer together. There is a lot of knowledge sharing happening which is feeding the development of more immersive experiences for the player– and that is always a good thing.
With all these changes the amount of sounds that an audio team can create to support a game has shot up drastically. As a result, we now have a whole new set of challenges to face when it comes to resource management. We need to be mindful of the amount of voices that are playing consecutively in a game as well as the amount of memory being used to load those sounds into the game.
The issue with voice count is related directly to the advancement of games and what they can produce. Now that games can have more and more enemies on screen, the amount of demand for sounds to support their actions has increased as well. In many cases, to the point where it can break the game if proper voice count management is not considered. Each voice uses CPU which unfortunately means that there’s no way to have an infinite amount of voices playing at one time. Memory is a similar issue that can arise based on the number of objects and characters a game can load in at one time. Sound designers need to be aware of how many variations of a sound they can afford to have along with understanding how many different sounds you can use to create a specific moment. You can begin to see amount of resource management and planning that must be done way before production can even begin.
New techniques and standards for traditional 5.1 and 7.1 surround have been joined by Atmos and more vertical audio, and obviously bring great fidelity – but what do you think they can bring to a gaming environment and why is that important?
That is a very interesting question. There has been a lot of major developments in 3D audio and surround formats, especially since the introduction of VR to the home environment. I still remember how shocked and amazed I was the first time I tried 3D sound technology in headphones. The space and depth it created really opened up the audio experience and added something I never thought possible. Through audio processing they really created a smoother transition between the right and left channels and even a sense that something was coming from behind you.
Recently, Dolby has expanded Atmos technology to the point where you no longer need ceiling speakers or a surround system. You can even find 4K TVs currently out on the market that include built in Atmos (which is even more mind blowing to me). In the last year I picked one up, and to be honest I wasn’t expecting much, however, it ended up introducing me to a whole new experience [sound-wise]. Environments sounded richer, more immersive, and positioning of sounds in the 3d space felt more accurate than it ever has before.
For games, this helped to create a more accurate and believable audio experience for the player. Sounds can now be processed to communicate a more positional feedback that even include height differences. You can listen to sounds and feel if something is coming from behind, above or even below you. Now the sounds of the game can drive where and when the battles happen in an FPS like Halo.
In a competitive shooter with this technology, a player can use directional sounds that are important gameplay feedback like enemy gun fire or footsteps to their advantage. As an avid gamer, who loves a good FPS, I am constantly using this to identify nearby threats with the hopes that I can get ahead of a possible altercation or flank. I slowly walk around the map listening for any signs of movement and then I try to launch into action with a surprise attack. PUBG and Apex Legends both do a great job of hyping up the enemy movement sounds in the audio mix. Systems like Atmos give you a bigger field to help distinguish exactly where that sound is coming from and they can add a layer of realism that has been sought after in games. I am pretty excited to see what they come up with next.
As a sound designer, you must have a pretty sweet setup at home. Tell us about it, but also tell us what we should be doing to make the most of more mortal headphones and speakers to get the most from our games.
Well it depends on if you are talking about my home gaming system or my home sound design system. For gaming, I have an Xbox One X and 4K LG TV with the built-in Atmos enabled. When I play with my friends online, I like to use a Turtle Beach headset. As for my sound design rig, it’s a bit more complicated as I actually opted for a more portable system with a beastly laptop equipped with an Intel i9 processor and 32gb of RAM). I’ve paired that with several external hard drives (mostly G Tech) for my sound library of field recordings and back-ups. Software-wise, I run Pro Tools and Reaper as my DAWS (Digital Audio Workstations) with a large collection of plug-ins that includes Isotope RX, Sound Toys and Altiverb.
As a sound designer, I also have several microphones I use which include a Schoeps MS and several field recorders (a pair of Sound Devices 702T’s, a Mix Pre 6 and a Zoom F8 to name a few). The Mix Pre 6 also doubles as my audio interface for my computer. Lastly, I have been building up a fairly large Eurorack synthesizer over the last 3 to 4 years. Eurorack, is a synthesizer format that allows people to collect different modules and piece them together to create a custom hardware synth/sampler based on their needs and wants. It is the perfect addition to the toolset of a sound designer working on a sci-fi game like Halo. I actually have Kyle Fraser (our Lead Sound Designer) to blame for getting me addicted to Eurorack Synths, it was his feature on the Halo 5 Sprint Series that pushed me over the edge and convinced me to start collecting them.
I think calibration is the key to extracting the most out of an audio experience. There are loudness standards enforced on games, and several studios follow them, but at the end of the day it is really up to the user (and how loud they set their playback system) that controls how the sound experience is perceived. Personally, I prefer to calibrate the loudness of my system to how loud I want higher SPL sounds, like gun shots, to be. I usually fire up a game, and if it is an FPS, I turn the sound up until the sound of the gun is at a level that I feel comfortable with.
Then I start to play. Hopefully, if the Sound Team did a great job mixing, everything should be playing back at an appropriate level relative to that gun’s sounds. In past Halo games when I put a sound in, I usually mixed its volume relative to weapons such as the AR or Magnum. Those sounds would usually be pretty dialed in (volume-wise) early on, so I could trust that I can mix my sounds using them as a reference.
When you’re thinking about creating a new sounds for a Halo Level from scratch, how do you approach the challenge – and what aspects of the process do you think would surprise people most?
Whenever I approach designing sounds for a level or map’s sound scape, I begin by meeting with the stakeholders. Usually that involves Level Designers, Artists and the Audio Director. We meet and discuss everyone’s vision for the level, which allows me to get a better understanding of the story and any key sounds they might want represented in the sound scape. Before I begin creating assets, I try to identify any technical limitations or programmer support that may be needed. Then, I start the sound design process.
Usually my first step when designing any sound is to begin collecting source sounds through a variety of different methods. I start by checking my sound library to see if I have any special recordings captured in the past that could work for the level. then, I start recording whatever fresh source sounds I can get my hands on. If it is a mysterious Forerunner or energy based sound I start on my Synthesizer, if it is a reality-based sound then I head to the Foley Room. This is a room in our studio that is dedicated to recording all sorts of different sound effects. It is filled with several different props that we have collected over the years including an old beat up typewriter, sewing machine, treadmill, several different pieces of metal, a destroyed piano, an old tire from my previous car and a sledgehammer to destroy things in the search for the perfect sound. It also has a homemade Foley Pit that you can fill with several different surfaces including dirt, sand and foliage.
After I collect all the source I need, I begin the sound design process which involves processing sounds with plug ins, layering and manipulating the sounds until they work with the visuals provided. After all the sounds needed for the soundscape are accounted for, I record them down, clean them up, name them and start to implement them into the game (this process goes fairly quickly).
Once everything is playing back properly in-game, I begin a rapid iteration process. Sounds can sound great in Pro Tools but sometimes not work as well when played back in-game. It’s in this phase where I identify any issues and quickly swap them out until I am happy with the overall sound.
Then I begin the review process. The stakeholders now have a chance to hear the latest iteration of the sound design. During this time, the sounds might change even more until all parties are happy with the result. From there the experience is handed off for final mix and I begin the process again on another level.
I think people (who are not sound designers) would be shocked by how many of the sounds they hear in a game originate from a completely different source. Sound designers often take sounds or audio recordings out of context and use them to create sounds that might be completely different from the object that they were recorded from. For example, the metal ticks and clicks of computers that have been frozen in the Halo 5 multiplayer level [Stasis] are actually the sound of my old car’s engine cooling off after a fast drive down the freeway.
Another instance is the sound of the lava in the Halo 5 multiplayer map [Molten] is comprised of several “slimy” type sounds I recorded in my office using mixtures of flour and water that have been layered with sizzle sounds of a hot iron being pressed on a wet cloth. My favorite example is from Halo 4, where the sound of the Forerunner Knight’s screech is partially a collection of Tasmanian Devil sounds that I recorded with a coworker over the course of two weeks. That is the most creative part of being a sound designer…you never know what will work until you try it.
What is your favorite game of all time, and why? And what aspects of that game would you most like to see in Halo Infinite?
I can’t say that I can name any one specific game that is my favorite, because I love so many games for many different reasons. If I had to list a few, I would say: WCW vs NWO World Tour, Final Fantasy 7, Metal Gear Solid, Halo 5, all Rockstar games, PUBG and, more recently, Apex Legends. As for what I would like to see from those reflected in Halo Infinite? I am not too sure. It’s kind of a difficult thing to answer, because I think Halo really should be true to itself. At least, that is what I learned while doing sound for it since Halo 4.
If you drift too far from the original formula, it can become something that is unidentifiable and foreign to the hardcore fans. But if I had to choose something, I would say a rich story for the fans to enjoy. Similar to games like Final Fantasy 7, or Metal Gear, where the player can get emotionally involved with what is going on in-game and have a lasting experience that they will remember for years on end. That, and a great competitive multiplayer experience like Halo 5.