Audio

From waveform to model.
One GPU pipeline.

Decode, capture, process, analyze, play, and encode audio through the same matrix runtime that powers OA machine learning.

Pipeline

Audio is not a side channel. It is first-class GPU data.

Decode and capture

Load WAV, FLAC, or MP3, capture a live device, or read the audio track from a video into an explicit Float32 contract.

Process on GPU

Mix, normalize, resample, filter, fade, and transform channels through reusable matrix operations and audio kernels.

Extract features

STFT, mel spectrograms, and MFCCs produce ordinary OaMatrix values ready for the ML stack.

Stream and encode

Play, seek, loop, or encode through the same stream contract used by Vision for synchronized media.

Unified media

One timeline from input to output.

OA keeps sample metadata, media timestamps, and GPU completion explicit. A decoded video frame and its audio block can be processed independently, synchronized precisely, and written back through the same media layer.

Audio_to_model.cpp

auto audio = OaAudioDecoder::LoadFile("speech.flac").Unwrap();
auto clean = OaFnAudio::Normalize(audio.Buffer, -3.0F);
auto mel = OaFnAudio::MelSpectrogram(clean, audio.Meta());
auto output = model.Forward(mel);

Capabilities

One tensor contract

Audio is Float32 OaMatrix data shaped [channels, samples], with sample rate and channel layout carried explicitly.

GPU-first DSP

Fifteen verified operations cover the practical path from waveform processing to ML-ready features, with explicit codec and synchronized metric boundaries.

Real media I/O

File decode, device capture, playback, video audio tracks, WAV output, and AAC recording share one timeline.

Cross-module composition

Audio features feed OaMl directly; audio and video share timestamps, completion, and container boundaries.

C++ and Python

The same module structure and operation names are exposed natively and through the oa.audio package.

Cross-vendor Vulkan

The compute path uses the same capability-driven Vulkan runtime as Core, ML, and Vision.

From waveform to model.One GPU pipeline.