How to Create a 3D Audio Effect Generator

3D Audio Overview

An immersive experience is much talked about in the current mobile app world, given how it evokes emotions from users to merge the virtual world with reality.

3D audio is a fantastic gimmick that is capable of delivering such an experience. This tech provides listeners with an audio experience that mimics how they hear sounds in real life, mostly by using the binaural sound systems to capture, process, and playback audio waves. 3D audio allows the listener to know where audio sources are from, thereby delivering a richer experience.

The global 3D audio market, according to a report released by ReportLinker, is expected to reach 13.7 billion dollars by 2027 — which marks an immense financial opportunity, as long as this kind of audio effects can be enjoyed by as many users as possible.

The evolution of mobile app technology has made this a reality, making 3D audio more accessible than ever with no need for a bulky headset or a pair of fancy (but expensive) headphones. Truth be told, I just lost one of my Bluetooth earphones down the drain a few weeks ago, and I was struggling to manage without 3D audio. This made me realize that the built-in 3D audio feature is paramount for an app.

Well, in an earlier post, I created a demo audio player with the 3D audio feature, thanks to the spatial audio capability of the UI SDK from HMS Core Audio Editor Kit. And in that post, I mentioned that after verifying the capability's functionality, I'd like to create my own UI rather than the preset one of the SDK. Therefore, I turned to the fundamental capability SDK from the kit, which provides an even more powerful spatial audio capability for implementing 3D audio and allows for UI customization.

Check out what I've created.

4. Declare necessary permissions in the AndroidManifest.xml file, involving the vibration permission, microphone permission, storage write permission, storage read permission, Internet permission, network status access permission, and permission to obtain the changed network connectivity state.

<uses-permission android:name="android.permission.VIBRATE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />

SDK Integration

1. Set the app authentication information via:

• An access token (which is obtained by calling this API). Call setAccessToken to set an access token during app initialization.

HAEApplication.getInstance().setAccessToken("access token");

• An API key (which is allocated to the app during app registration in AppGallery Connect). Call setApiKey to set the key during app initialization.

HAEApplication.getInstance().setApiKey("API key");

2. Call applyAudioFile to apply the spatial audio effect.

// Apply spatial audio.
// Fixed position mode.
HAESpaceRenderFile haeSpaceRenderFile = new HAESpaceRenderFile(SpaceRenderMode.POSITION);
haeSpaceRenderFile.setSpacePositionParams(
                            new SpaceRenderPositionParams(x, y, z));
// Dynamic rendering mode.
HAESpaceRenderFile haeSpaceRenderFile = new HAESpaceRenderFile(SpaceRenderMode.ROTATION);
haeSpaceRenderFile.setRotationParams(new SpaceRenderRotationParams(
                                    x, y, z, circling_time, circling_direction));
// Extension.
HAESpaceRenderFile haeSpaceRenderFile = new HAESpaceRenderFile(SpaceRenderMode.EXTENSION);
haeSpaceRenderFile.setExtensionParams(new SpaceRenderExtensionParams(radian, angle));
// Call the API.
haeSpaceRenderFile.applyAudioFile(inAudioPath, outAudioDir, outAudioName, callBack);
// Cancel applying spatial audio.
haeSpaceRenderFile.cancel();

The whole development procedure closes here, giving birth to an app that works like the GIF above.

Use Cases Beyond Music Playback

Music playback is just one of the basic use cases of spatial audio capability. I believe that it can be adopted in many other scenarios, in navigation, for example. Spatial audio can help users navigate from A to B even easier. It could, for example, tell users to "Turn left" with the sound coming from the left side of a listener, taking immersion to a new level. 

Karaoke apps, on the other hand, can count on spatial audio and audio source separation (a capability I've also used for my demo) for generating accompaniments with even better effects: The audio source separation capability first abstracts the accompaniment a user needs from a song, and the spatial audio capability then works its magic to turn the accompaniment into 3D audio, which mimics how an accompaniment would really sound like in a concert or recording studio.

Takeaway

3D audio contributes heavily to the immersive experience of a mobile app, as it can digitally imitate how sounds are perceived in the real world. Such an effect, coupled with the huge financial benefits of the 3D audio market and its expansive application scenarios, has thrown 3D audio into the spotlight for app developers.

What's more, devices such as headsets and headphones are no longer necessary for enjoying 3D audio, thanks to advancements in mobile app technology. A solution to implementing the feature comes from Audio Editor Kit, which is known as spatial audio and is available in two SDKs: UI SDK and fundamental capability SDK. The former has a preset UI featuring basic functions, while the latter allows for UI customization and offers more functions (including three rendering modes applicable to different use cases and music genres). Either way, with the spatial audio capability, users of an app can have an audio experience that resembles how sounds are perceived in the real world.