Audio

From waveform to model.
One GPU pipeline.

Decode, capture, process, analyze, play, and encode audio through the same matrix runtime that powers OA machine learning.

Pipeline

Audio is not a side channel. It is first-class GPU data.

Decode and capture

Load WAV, FLAC, or MP3, capture a live device, or read the audio track from a video into an explicit Float32 contract.

Process on GPU

Mix, normalize, resample, saturate, and filter with block-parallel Biquad and 1–64-section SOS processing on the GPU.

Extract features

STFT, mel spectrograms, and MFCCs produce ordinary Matrix values ready for the Ml stack.

Stream and encode

Play, seek, loop, or encode through the same stream contract used by Vision for synchronized media.

Unified media

One timeline from input to output.

OA keeps sample metadata, media timestamps, and GPU completion explicit. A decoded video frame and its audio block can be processed independently, synchronized precisely, and written back through the same media layer.

Audio_to_model.rs

1use oa::{audio, Engine};
2
3let engine = Engine::new()?;
4let decoded = audio::decode_file(&engine, "speech.flac")?;
5let clean = audio::normalize(
6    &decoded,
7    audio::NormalizeAudioConfig::default(),
8)?;
9let mel = audio::mel_spectrogram(&clean, audio::MelConfig::default())?;
10let output = model.forward(&mel)?;

Capabilities

One signal. Every stage. Same contract.

Decode, transform, synchronize, and deliver without breaking the data model between media, compute, and machine learning.

One tensor contract

Audio is Float32 Matrix data shaped [channels, samples], with sample rate and channel layout carried explicitly.

GPU-first DSP

Schema-owned waveform operations, saturation, block-parallel Biquad, SOS filtering, and Ml-ready features share explicit graph ownership and independent host oracles.

Real media I/O

File decode, device capture, playback, video audio tracks, WAV output, and AAC recording share one timeline.

Cross-module composition

Audio features feed Ml directly; audio and video share timestamps, completion, and container boundaries.

Rust-native composition

PascalCase values and snake_case operations live directly in oa::audio and compose with the same Matrix and Engine contracts.

Capability-gated Vulkan

The compute path uses the same vendor-neutral runtime as Core, Ml, and Vision; each device pack remains independently qualified.

From waveform to model.One GPU pipeline.