Using Audio Benchmarks to Improve Communication in Early Game Development
In the early stages of game development, conceptualising the game is one thing, but bringing those ideas to fruition is the biggest challenge. First of all, it's difficult to know where to start, and it's incredibly time-consuming. However, once you start sharing these challenges as a team, being able to communicate these ideas in a way that makes sense for everyone else is where the true magic happens. The question I've been asking myself is this: How can I effectively communicate my ideas in this phase of development? There are numerous disciplines in game development, each with its own entire world of creativity, expression, and workflows. While most of these share a similar medium of communication— visual media, game audio development is unique in that the communication comes from sonic representations of ideas, which most can argue is highly subjective and misunderstood.
When I mention terms like "colour," "tone," and "saturation," I would argue that most people will fit these into the visual categories of graphic design, photography, or colour correction in video editing tools. Interestingly enough, these terms are also highly used in audio to describe the characteristics of sound, and the technicalities of how they change the qualities of a sound are exactly the same in audio production. What makes these terms different is their level of subjective understanding. For example, if I want to describe a picture, I can use universally understood terms like "warm," "bright," "blue tint," "high saturation," or "high noise." But what if I use these same descriptors to describe a sound? Will these still be universally understood? If you're not a musician or audio enthusiast like me, then it's not so likely!
This is why I believe that while working in a game development team as a sound designer, my best form of communication isn't asking the team to review every small detail of a sound or piece of music. Instead, I need to communicate by using audio as an overall experience and connect these experiences to our shared vision for the game. This involves a higher-level, top-down approach while working and communicating with sound.
Now let's try another exercise. I would like you to think about your favourite video game. Think about the experiences you had with that game. Now in a few words, how would you describe those moments?
What caused these feelings to happen?
What comes to mind first?
Chances are, you would have used words that relate to an emotional response, and the game music would have played a big role in those experiences. Once we understand the role sound plays in games, it becomes much easier to learn how to communicate with other team members outside of the audio realm. This is where the term "Audio Benchmarks" comes into play.
The Power of Audio Benchmarks
Recently, I've been reading a book called Game Audio Mixing by Alex Riviere (2023), which has been a very useful resource for understanding the technical details and different stages of game audio work. Riviere says that a benchmark is a representation of how the team wants the audio to sound at a particular moment in the game. It's a small mix made in a linear fashion that's completely outside of the game engine. This means we can work within the DAW to build up a sonic palette for different stages of the game. This sets the expectations of the audio quality as well as helps us learn what the guidelines and processes are for achieving the same level of quality in the game engine later on.
"After the assessment and high-level visionary phases, it is time to get started with some actual audio work. The goal is to translate your vision into something tangible, like a style guide. Starting the mix early on allows for benchmarking and mixing sounds for a few features to demonstrate your aesthetic approach, which is extremely beneficial for you and others. It's all about evaluating your mix vision on a controlled scope, taking the time in pre-production to test different ways to achieve your goals, and to develop further ideas based on your preliminary assumptions in a 'non-stressful' controlled environment." p. 21
What to include in a benchmark?
But how many benchmarks should we do as a sound designer for a game? I believe it depends on the project size and scope. Our last game at Amythica and Stoutheart Games, Pick-a-Pocket had a much shorter development life cycle due to the game being born within a fast-paced game jam. This meant I had less time to build up mixes and guides for the game, and instead, I was endlessly tinkering around in the game engine and within the audio middle-ware finding the right mix and styles. This is where the audio vision unveils itself a bit later in the development which isn't desirable but sometimes can't be avoided.
If anything, Pick-a-Pocket taught me how important it is to build up a vision well in advance so that each sound is intentionally crafted with the vision in mind. For my current project with Amythica, we have been quite ambitious with these early stages of development, and with that said, we need a clear vision of how everything will look, sound and feel for each environment we intend to build. Our new game Project JEL is an online co-op extraction looter with detailed lore and story elements. As a sound designer for this project, I need to make sure that these elements are the primary focus during these conceptualisation stages. This is why, for me, using at least one benchmark for each environment or game state will be a huge benefit.
"When deciding what type of sounds to benchmark, think about what are the most important types of sounds for your game. Do you have any audio signature features? What sounds are essential to deliver your vision? Do you have any aesthetic or technical risks, or uncertainties that you would like to flesh out early on? Being early in the preproduction phase, this step is about conceptualizing audio outside of the game engine environment." p.22
Setting Guidelines For Further Development
Let's think for a moment about what happens when you work on benchmarks for a game. You are bringing the audio vision to life, while also testing the visions and guides that you set beforehand. This means that the more time you work in this phase, the clearer and more defined the vision will become.
It also turns out that once we have all of these benchmarks in place, we can use these for mix and implementation guidelines. Riviere says that this can be done by measuring each sound within a benchmark so that they can be implemented in the engine with the same measurements in mind.
"As you benchmark mix should be a representation of the final quality level you're aiming for, the contents and the way they are mixed can be used to prepare for larger chunks of the game to be produced later on. Extracting implementation guidelines and contents mastering rules from your benchmark mix in a form of a spreadsheet is an important aspect of collaborative mixing work. It can also be convenient for a one-person audio department. It's all about drafting a plan of action to keep the mix under control at any given moment in time."
Now that we've talked about benchmarks, let's hear some in action! I want to share some real-world examples from Project JEL. These are benchmarks I created, and they're currently helping us define the sound for each area in the game.
3D Player Footsteps
Not so long ago we had a prototype which allowed us to all jump into a co-op game. It got me thinking about how players should perceive other players in the multiplayer experience. This is where I first started working on the player footsteps. Most of us don't really notice the footsteps around us while playing a game, but once they are taken away, it becomes much more obvious that they play a big role in feeling like you are a part of the environment around you. Adding onto this, I wanted the players to have positional awareness of other players, but I didn't want these sounds to be too over the top. In fact I wanted other players to sound light on their feet by matching the hopping animation style that our artist Daniel created for the player model.
Have a listen to how the footsteps sound. What else can you hear in terms of the materials and mechanical elements?
If you picked up metallic materials and small servo motors, you're correct! This benchmark combines a few elements I wanted the player's footsteps to have such as the soft metallic impact, and the feeling of an electrical, mechanical motor for a robotic aesthetic.
While these footsteps will sound consistent throughout the game loop, the environment will change regularly. This is why I have been creating multiple benchmarks for the different soundscapes the player will be in at each given time within the game.
Spaceship Landing Sequence
This next benchmark is another experiment that requires a different kind of approach while thinking about sound. How can we indicate to the player exactly what is happening just with audio cues? This is where thinking about audio as a way to form the narrative comes into play.
Before giving this next one away, grab those headphones again and have a listen to this audio clip.
Where do you think you currently are while you listen to this? What gives it away?
This is a benchmark for the spaceship landing sequence when the player is waiting inside the ship before it lands on a planet. You might have heard a few sounds that gave this away such as the ship’s engines engaging, the RCS thrusters before touchdown, and the impact and spool-down of the engines. But if you heard each of these layers individually, it would become a lot more vague. Just like piecing together a narrative, carefully combining sounds in a specific sequence like this is what creates a recognisable and memorable experience.
But what can make this even more blatantly obvious? Dialogue of course! This might be a bit cliché but it's used for a very good reason — it's immersive for the player.
Now let's hear the same clip again but with some added voices.
This voice was intended to be an AI for the ship. The player not only receives useful indicators, it also adds an extra element to the lore of the game. Interesting elements like these are what add to the hype in the team because it opens doors to other gameplay, accessibility and narrative possibilities.
With this in mind, I finally have one more benchmark to share.
“The Hub”
Where do you think this took place? What sort of environment would you describe this as?
It doesn't sound hostile, however it still sounds ominous...
This is a busy city that is known for its business and trade. It's set up as a central hub for players and NPCs to exchange their loot from different planets. However, while moving through the city you may notice a few in-game announcements.
In short, this is because this hub is ruled by a corporation that all players must work for. Some districts are out of bounds if you don't have enough social credits or not a high enough rank.
The best way to indicate this through sound is again, using dialogue and environmental design. I've done this through the hub’s promotional advertisements and loudspeaker announcements. When you listen closer to the ambience, I've used field recordings of construction and street markets that add extra context to where the player is.
Benchmarks in the Development Stage
Designing these benchmarks has proven to be a massive unlock for us as a team because it offers a clear form of communication that wouldn't be possible any other way. Adding small details such as the engine exhaust sounds in the spaceship sequence helped us understand what ship designs we need to make sure the experience feels as authentic as possible. Bringing in the ship announcements has given us more options on how we should use the ship’s built-in AI to indicate game events to the player. From here, once we are happy with each benchmark then we would need to start bringing them into the game which involves probably a ridiculous amount of iteration and refinement.
Oh well, that's a future problem!
Thank you so much for reading through this Dev Log. If you are interested in knowing more about what I have been working on then please feel free to check out my YouTube channel where I do all sorts of things like breaking down popular game sound effects, creating alien sounds, sharing tutorials on how to use Unreal Engine 5 Metasounds, and more!
If you would like to stay in the loop on how we are progressing with Project JEL and Pick-a-Pocket, you can join our public discord channel here: https://discord.gg/aHpVMZ6d
We would love to hear from you if you have any ideas or feedback.
Cheers,
Alex
