Content »
Kartaverse »
Immersive Pipeline Integration Guide »

Spatial Audio Tools

Scrivener Export - Reformatting Needed!

This article is an export of a Scrivener document. It will definitely need at least some reformatting to work in Obsidian and MkDocs. Delete this note once the article's formatting has been fixed to some extent.

Impulse Response Simulation¶

Impulse Response Simulation

Extensive research is underway to explore the feasibility of analytically generating sound impulse response files from the volumetric scan data.

This approach helps increase the realism of the immersive XR experiences by allowing for a more accurate sound playback of field-recorded sound elements. These techniques can model sound sources and the audio level "fall off" when they are hidden by occlusions (large blocking objects) such as natural caverns, and interior spaces like hallways and tunnels.

Reaper DAW¶

Reaper DAW

Reaper is a free open-source DAW software package that is excellent for spatial audio workflows. It doesn't force any preconceptions onto the audio so you can flexibly work with multi-track sound information.

https://www.reaper.fm/

Logic Pro¶

Logic Pro

Apple's Logic Pro DAW software can be used to trim spatial audio based recordings down to the sound clip's final edited duration. The program runs on macOS systems.

Kartaverse/Immersive Pipeline Integration Guide/img/image195.png

Resolve Fairlight DAW¶

Resolve Fairlight DAW

Blackmagic Design's NLE software Resolve (free) and Resolve Studio (Paid) include a copy of the Fairlight DAW.

https://www.blackmagicdesign.com/products/davinciresolve/fairlight

Kartaverse/Immersive Pipeline Integration Guide/img/image280.png

About Fairlight¶

About Fairlight

Fairlight has its own dedicated page environment inside the Resolve software:

Kartaverse/Immersive Pipeline Integration Guide/img/image276.png

Fairlight makes it possible for advanced audio post-production workflows to be completed inside the same Resolve project database that holds Media pool based footage, Fusion page composites, and Resolve Edit page video editing timelines.

Fairlight Sound Library¶

Fairlight Sound Library

When you first open the Fairlight page Sound Library tab an option is presented to "Download DaVinci Resolve's Free Sound Library".

Kartaverse/Immersive Pipeline Integration Guide/img/image23.png

Clicking the "Download" button takes you to the BMD Support Center webpage in your default web browser. After completing the 2-part form on the webpage you can then download a macOS or Windows based install package.

Kartaverse/Immersive Pipeline Integration Guide/img/image162.png

Once you press the "Register & Download" button a new download link will be generated for your session.

Kartaverse/Immersive Pipeline Integration Guide/img/image343.png

On macOS systems you would open up and mount the downloaded DMG (Disk Image) file named:

Blackmagic_Fairlight_Sound_Library_Mac.dmg

The disk image holds a "Install Fairlight Sound Library 1.0.pkg" installer.

Kartaverse/Immersive Pipeline Integration Guide/img/image53.png

The PKG installer then adds the 1.12 GB of sound resources to a folder that Resolve's Fairlight page can access.

Kartaverse/Immersive Pipeline Integration Guide/img/image397.png

Unreal Engine MetaSounds¶

Unreal Engine MetaSounds

Epic Game's Unreal Engine 5 includes the MetaSounds DSP technology for ambisonic spatial audio playback on PC desktops, Consoles and VR HMDs.

For more information about Unreal MetaSounds:

Cycling '74 Max¶

Cycling '74 Max

Max is a node based creation environment used to create audio and interface with hardware like MIDI devices.

https://cycling74.com/products/max

For more information about Max:

Spat Revolution¶

Spat Revolution

Spat allows artists, sound-designers, and sound-engineers to create and mix audio that is targeted at creating outstanding immersive experiences. The toolset provides options for WFS (Wave Field Synthesis). Spat Revolution comes with three integration plugins for use with DAW packages.

https://www.flux.audio/project/spat-revolution/

Fraunhofer SpatialSound Wave¶

Fraunhofer SpatialSound Wave

https://www.idmt.fraunhofer.de/en/institute/projects-products/spatialsound-wave.html

Morrow Sound¶

Morrow Sound

https://www.morrowsound.com/

Dolby Atmos¶

Dolby Atmos

https://www.dolby.com/technologies/dolby-atmos/

Spatial Audio Designer¶

Spatial Audio Designer

https://newaudiotechnology.com/products/spatial-audio-designer/

NVIDIA VR Works | Audio Tools¶

NVIDIA VR Works | Audio Tools

https://developer.nvidia.com/vrworks/vrworks-audio

Noisemakers Ambi Plugin¶

Noisemakers Ambi Plugin

The Noisemakers Ambi tools help create 3D audio scenes which can be used in the production of 360VR/180VR videos, and immersive experiences.

https://www.noisemakers.fr/product/ambi-bundle-hd/

Noisemakers creates ambisonic plugins including:

AMBI PAN HD -- Position input sounds in the 3D scene
AMBI VERB HD -- Add 360 reverberation
AMBI LIMITER HD -- Control the ambisonic bus peak level
AMBI HEAD HD -- Render the 3D scene to binaural audio
AMBI EYES - Monitor 360 videos while authoring audio
AMBI CONVERTER - Convert B-format signals between FuMa and ambiX

Comsol Multiphysics Simulations¶

Comsol Multiphysics Simulations

A friend of the Kartaverse project, Jared Sandrew (from InfiniteWorld), introduced our audio research group to Jason Riggs from Harman International. Jason volunteered his time over multiple sessions, and shared his insights via Zoom on advanced audio workflows, sound impulse response creation, and an overview of acoustic simulation technologies. Part of that effort by Jason was to explain how the Comsol Multiphysics software is used to pre-visualize detailed and accurate audio environments based upon measured real-world locations for HiFi home audio and automotive audio needs.

Comsol Multiphysics, when combined with the Comsol acoustics simulation module, dramatically expands on what is possible for digitally simulating acoustic environments using hybrid FEM (Finite Element) solvers.

This is a sample Comsol website image of simulating sound emission from a source like a speaker element:

Kartaverse/Immersive Pipeline Integration Guide/img/image155.png

These techniques potentially take the volumetric research further by allowing for a greater understanding of the acoustic properties of the environments such as caves, and visualize how sounds in this space propagate.

Kartaverse/Immersive Pipeline Integration Guide/img/image223.png

Ultrasonic Sounds in Caves¶

Ultrasonic Sounds in Caves

Through the use of software like Comsol, it is possible to fully visualize and simulate the acoustic properties of how ultrasonic avoidance is used by bats for navigation and hunting insects using sound recordings of bat vocalizations. One can see plots in 2D and in 3D of the sound waves propagating outwards as bats fly through a large underground cavern hunting for insects.

This image shows a real-world ultrasonic audio recording of a pulse train of bat chirps from a cave in Puerto Rico:

Kartaverse/Immersive Pipeline Integration Guide/img/image102.png

Microsoft Triton | Project Acoustics¶

Microsoft Triton | Project Acoustics

The Microsoft Triton / Project Acoustics learning resources have been enormously helpful to Kartaverse development efforts. Project Acoustics ships with extensive background reference material that clarifies what is currently possible with real-time acoustic simulation-driven spatial audio playback and for insight into game engine-based uses of acoustic impulse responses.

For more information:

Project Triton | Unity Plugin

Project Triton | Unreal Plugin

Project Triton | Talks

Kartaverse/Immersive Pipeline Integration Guide/img/image4.png