SDLGL: Audio system upgrade and Animalese PoC

SDLGL

As you may have seen from my projects overview post, SDLGL is my pet-project game engine written in C++ for 2D game development.

As a re-cap, it has the following features and is a fun way for me to explore modern C++ programming and game design patterns.

• A simple and extensible Entity + Scene organization
• A simple 'Update and Render' game loop
• Audio mixing for sound effects and music
• Easily configurable (JSON-defined) resources supporting:
  • Animated sprites
  • Static textures
  • Sound effects
  • Music tracks
• Collision detection system for rotatable rectangles
• Font renderer
• Debugging UI elements
• Access to the SDL2 rendering context
• Multiple example programs demonstrating each engine feature

You can read more about it and see example programs I've made on the GitHub page.

Idea: Animalese-style dialogue animation

A focus of my game development ambitions is the genre of "cozy games". Some examples of cozy include Animal Crossing, Stardew Valley, and Stray.

A common feature of these games is a dialogue system that substitutes real speech with babbling sounds, giving a "cute" effect when characters are speaking.

For example, see how speech is handled for robot characters in Stray:

Animal Crossing also takes this approach. Their language is even given an official name: Animalese:

I wanted to try my hand at implementing a similar dialogue system that I could use for one of my games, modeled off of the Animal Crossing's Animalese.

The challenge: SDL_Mixer

Creating proof-of-concepts like this one are great for pushing the limitations of your game engine. In my case, I found that SDL_Mixer, the simple mixing library that I used for mixing audio samples in SDLGL, was missing some features that I would need to build my Animalese prototype.

Specifically, I needed to dynamically set the pitch of audio samples. It would also be nice to have access to an unlimited number of software-defined channels as opposed to the 16 channels that SDL_Mixer allows for.

I searched for the best replacement to SDL_Mixer and came across OpenAL. OpenAL is a popular choice for indie developers given it's lack of license fees and decent feature set. However, OpenAL seemed almost too heavy-weight for the scope of my game engine. Moreover, the most popular open-source implementation of OpenAL, openal-soft, is licensed as LGPLv2, which does not allow for static linking without releasing your own program under LGPL.

Instead, I opted for the lighter-weight and MIT-no-attribution-licensed miniaudio project by David Reid. It offers 3D spatialization of audio, as well as build-in pitching effects and an API for custom effects. Additionally, it allows me to use the SDL audio device for playback, which reduces compatibility issues for my games. (Specifically, I know that SDL's audio device is supported by WebAssembly.)

The result: Animalese demo using SDLGL with new miniaudio backend

GitHub Repository Link

Using Animalese as my testbed, I re-wrote the audio system of SDLGL to use miniaudio in place of SDL_Mixer. It was a relatively easy process, with the key difference that miniadio has its own resource manager, allowing me to remove sound file management from SDLGL's resource manager.

While I kept the typewriter effect, I opted for a simple face-card in place of Animal Crossing's 3D cutscenes. My approach is inspired by more dialogue-driven games like Ace Attorney:

To create the effect, I used a simple approach that I've adapted from GitHub user Acedio's AnimaleseJS project. Each letter of the alphabet is given a short speaking sound (such as "a", "be", "ki"). Then the sentence being spoken is shortened to just the first letter of each word and puctuation is removed.

So for example, "Hello, World!" becomes "h w".

Then, the sound for each character in the string is played with a random variation in pitch give a natural speech-like pattern.

You can see a screen recording of the dialogue here. (to be uploaded)

A screenshot of the Animalese demo

You can also try interacting with the demo yourself by building it from source. :)