On top of the new content such as the Hardcore mode, Title Update 6.1 brought several fixes to bugs and glitches. These fixes also included improvements to several audio issues you’ve been experiencing since the very beginning of your journey in The Division 2, such as delayed and/or no gun sound despite shooting from your weapons, audio distortion or no game sound at all.
We wanted to take some time to give you some insights on how the Audio-tech team approached these very complicated problems throughout these months.
Data patching on PS4
One of the clever features of the PS4 is the ability to download a “base” data image the first time and then overlay it with successive “delta” data patches. This greatly reduces the download size of data patches, but for the game to stream in audio assets efficiently they all need to be next to each other in the data image.
What we discovered happening as we updated the PS4 version of the game over a few months was that several groups of audio assets were drifting apart from each other as they got updated, and this impacted disk I/O performance for audio in a negative way. At first, we couldn’t see this from our workspaces because in development we generally make a completely new disk image as we work, so this drift was not apparent to developers - just to our internal testers and players.
After some debate we eventually did some straight A/B tests in the lab, and it got the irrefutable proof that the delta patching itself was creating problems for audio asset streaming. Since then we’ve updated our data patch authoring system so that it re-patches all the audio assets whether they have changed or not. It does make those data patches slightly larger, but it ensures the audio assets are all grouped together no matter what.
Memory Fragmentation
Let’s highlight a very important concept: in The Division 2 we have a much greater number of audio assets compared to the first game. One of the challenges the team had to face was to use the same amount of memory on the current gen consoles, while also having a smaller memory budget to work with due to other factors such as, for example, characters or world animations and other background systems taking memory allocations overtime.
To best leverage the memory budget that we had, we had to be very aggressive with what we load and unload while the game is playing. As soon as that memory budget is full, to load a new audio asset in you need to unload an older one, and with The Division 2 we had do most of that twice as quickly as compared to first The Division game.
To give you a better idea on how memory allocation works, just imagine when you start the game, everything looks like a nice open car park with all the used spaces tightly packed. A bus-sized vehicle might leave its place, and then a car could move into that empty spot. What happens in that case is that the new bus won’t be able to park in that same spot as that car has filled some of that area up. The new bus will have to park further on, thus making the car park less tightly packed than it was earlier, and therefore lose a little of its capacity overall. We call this type of degradation “memory fragmentation”.
For any memory allocated and freed directly from Snowdrop, we have a system that keeps track of it all and prevents fragmentation occurring. However, our audio asset management is done through middleware and works somewhat differently. How it works is fair if our game engine was reserving dedicated “parking sections” in memory for all our different data types, but it doesn’t play nicely with our more modern, dynamic memory allocator. For example, what was happening was that a player could be running the game for a couple of hours and eventually get to the point where switching to a different gun would result in no weapons fire - the audio assets for that weapon could not be loaded in anymore. Fragmentation had set in, and so the only solution was to reserve a dedicated “parking section” for audio assets at start up and hand this over to the middleware’s asset management system.
Microsoft Spatial Audio and Dolby Atmos
We were one of the very first companies to work with Microsoft Spatial Audio and while there was nothing wrong with the technology itself, a lot of sound peripheral companies on Windows 10 weren’t ready for us. For instance, we had a couple of cases where a whole range of headsets that were going full silent while using Dolby Atmos through Spatial Audio, despite functioning as intended while working in stereo mode.
Unfortunately, despite having tried many headsets, no one could have known that with that combination, some headsets would have not worked properly. This happened because some companies didn’t have our game and we didn’t have their headsets.
Despite all the efforts made to fix these issues we noticed coming up during the Private Beta, there were still a few headsets that were still not playing nicely with Microsoft Spatial Audio. We did a large amount of mitigation to resolve these issues, such as publishing a workaround and allowing the game to reboot its audio mix when in and out the client’s window.
In August we received a firmware update for Xbox One that made the Microsoft Spatial Audio playback system slightly heavier on the CPU, and since The Division 2 is running right on the edge of this console’s maximum performance, this small change upset smooth playback for anything that was spatialized by it. It was a tough decision, but for the sake of the players it was decided to deprecate Microsoft Spatial Audio on this platform and revert to standard 7.1 surround until further notice.
Pre-Emptive asset preloading
In The Division 2, there is a constant stream of content loaded in and out, which stress the capabilities of all the platforms we support, especially the consoles. As a result, the file I/O gets easily swamped. We need to ensure an optimal data transmission flow with the hard drives, since we’re not just talking about audio assets, but also animations and other game processes that are running in the background. A slight delay in loading assets, sometimes just milliseconds, and you could experience frame rates problems, freezing animations, audio de-synchronization and other issues.
We’ve done adjustments but those didn’t always work due to different factors such as what system you were playing on and how old your console or hard drive was. These are issues that didn’t come up with our Development Kits, since they run in different conditions compared to, for example, retail game consoles. At the end of the day, we shouldn’t consider these as traditional bugs, since there isn’t code at fault, rather an issue of hardware robustness and longevity.
Sometimes there are peak moments in terms of data traffic, and we have to consider the level of importance of some assets over others. For the loading system to work properly, we need to set priorities. We worked hard with the balancing team trying to rebalance the priorities because there were not just audio issues, but also textures problems where, for instance, a player could walk into a basement and no textures were loaded in. The player could be left standing in an empty void.
Part of the work with assets priorities went through preloading assets pre-emptively, allowing that specific chunk of data to be already available before it’s needed. An example of pre-emptive assets preloading can be experienced in the first tutorial mission while hearing the first mortar strike. Without that preloading, you would have heard the big mortar explosion with a quite significant delay of several seconds. Of course, when the player is funnelled in by the game, or watching a cut-scene, it’s easy to predict what needs to be cached, but in emergent, open world play this is absolutely not straightforward.
Moving forward
In the 6.1 update, thanks to the continued effort with pre-emptive asset preloading, internal testing and your reports, we mitigated different audio issues. However, we’re aware the work is not over, as more potential scenarios require specific tuning since there are many variables to take into account. As a result, you still might experience missing game sounds from time to time.
We appreciate your reports as they help us identifying more scenarios, which is why we look for every possible detail you share with us, such as when and where that specific audio issue happened. If you encounter more instances where you are experiencing audio issues, please let us know through our Player Support forums or the Ubisoft Support website.
/The Division Team
The insight into performance tuning for consoles is fascinating
I wonder if the Playstation 4's hard drive being connected via a USB interface contributes to the data streaming issues.
Internal PS4 hard drives are connected via SATA.Originally Posted by dagrommit Go to original post
I use SATA SSD and I still have issues with performance and sound.
It's going to be interesting to see how the new consoles are going to perform with this game.
You might want to read the article:Originally Posted by DeMizeFPS Go to original post
Granted, I should have differentiated between the physical interface (SATA) and the communication protocol (USB) they use.Originally Posted by eurogamer
Originally Posted by dagrommit Go to original post
Lol.
TIL-- Degradation specifically "Memory fragmentation" = a D.C DZ extraction zone.. or.. say.. the streets of NYC Manhattan shortly after the dollar flu outbreak.
Also..
might i suggest getting a hold of Pied Piper's middleout compression tech?
J/K.. ofc but thank you for the read up.. it's interesting indeed the challenges you gals and dudes face. Development "pvp".~!!
How this impacts your games.
I loved the Division 1 but felt you abandoned it in favor of a redo instead of actually getting it to a polished state. Because of that, I decided to hold on Division 2 until it was available on sale. So I bought it a day or two ago deeply on sale. The game is nearly unplayable with sound bugs and crashes. I doubt I will finish the game due to the sound and crashes.
I have gone from a die hard fan to just not interested due to these problems and I can understand that you are working with difficult to solve problems. My advice is you focus on getting a solid playable game and then adding bells and whistles and if you can't do both, you just get it working. Even if a lowered sound quality is the ultimate outcome, at least it works. Currently, we all know it doesn't work and I'd rather spend my time and money on something that does.