Streaming music in Castle Game Engine

I think this is the biggest functionality I’ve worked on at CGE so far. Particularly important for mobile games.

Why use music streaming?

Long audio files due to compression such as Vorbis or MP3 seem to be not very demanding. One 60 sec long song encoded in a 96 Kbps OGG file takes about 720 KB on the disk. Unfortunately, the whole decoded file can take up to 17 MB of RAM. The approximate usage of RAM can be easily calculated using the audio size calculator. Even for desktop games, loading many long music files can effectively reduce the size of available RAM.

In case of mobile games, the problem is much more serious. Each time the activity is changed, the OpenAL context is closed and sound files unloaded and after returning to the game, the music files need to be loaded again. In addition, sound files (buffers) reduce the memory available for textures. Decoding many audio files will significantly extend the game start time and will have a negative effect on the battery.

When not to use streaming?

Each solution also has its drawbacks. Starting streaming takes some time. In cases where access time counts, it is better to load the whole sound. These are primarily all sound effects such as sounds of explosions, jumps etc. Sounds of this type are usually played repeatedly, so reading them from the disk during the game would cause a significant decrease in performance/ make noticeable delay.

How do you use streaming in Castle Game Engine?

The LoadBuffer function has now an additional argument of type TSoundLoading:

function LoadBuffer(const URL: string; const SoundLoading: TSoundLoading; const ExceptionOnError: Boolean = true): TSoundBuffer; overload;

When we set SoundLoading to slComplete, the sound clip will be fully loaded when the game starts. When we set SoundLoading to slStreaming, the audio file will be streamed during playback.

If you use the XML sound repository, just add the stream attribute:

<?xml version="1.0"?>
<sounds>
  <sound name="track1" url="track1.ogg" stream="true"/>
</sounds>

Thanks to the corrections introduced by Michalis, streaming is not only available on the OpenAL backend (as I originally implemented it), but also on FMOD. Michalis has also made many other improvements and optimizations.

Logging used memory in CGE

If you want to know the amount of RAM used by sounds loaded without streaming from the engine level, just enable logs:

SoundEngine.LogSoundLoading := true;

Then you can find entries similar to the following in the console or log file:

Sound: Loaded "castle-data:/sounds/track1.ogg": audio/ogg, stereo 16, size: 44107776, frequency: 44100, duration: 250.04
Sound: Loaded "castle-data:/sounds/sfx_jump.wav": audio/x-wav, mono 16, size: 20964, frequency: 44100, duration: 0.24
Sound: Loaded "castle-data:/sounds/sfx_death.wav": audio/x-wav, mono 16, size: 59204, frequency: 44100, duration: 0.67

In the example above, the track1.ogg file (3.0MB size on disk) loaded without streaming will take 42.1MB (44107776) of RAM.