GPU Image Transforms
Resize, normalize, crop, flip, rotate, and blur — each available as a standalone call or chained in a pipeline. All operations run on the accelerator.
Vision
Images and video.
Straight to the model.
A complete image and video runtime for machine learning. Load, decode, transform, and normalise through one GPU-first media runtime. Every frame arrives as a tensor, ready for inference or training.
What it covers
From raw media to model-ready tensors.
Image Processing
Comprehensive image transformations for machine learning. Resize, normalize, crop, and more — all optimized for GPU workloads.
Video Decode
Vulkan Video decoding for H.264, H.265, AV1, and VP9 when the device exposes the required profiles and formats.
Ml Bridge
Seamless integration with machine learning pipelines. Video frames become model-ready tensors in a single operation.
Capability-Gated
One vendor-neutral API admits only the codec, format, queue, and kernel routes exposed by the selected device.
Capabilities
Everything between your media and your model.
Semantic images and video frames remain device-resident through ingest, transformation, model preparation, and preview. Stateful decode sessions exist only where media lifecycle requires them.
API
Clean surface. No ceremony.
Each operation is a direct call. No pipeline objects to configure before you can resize an image. No session lifecycle for a normalisation pass. Stateful resources — decoders, encoders — exist where they earn their keep.
Image_ingest.rs
1use oa::{image, Engine, ImageFormat};23let engine = Engine::new()?;4let decoded = image::decode_file(&engine, path, ImageFormat::Rgb)?;5let resized = image::resize(6 &decoded,7 224,8 224,9 image::InterpolationMode::Bilinear,10)?;11let input = image::normalize(&resized, image::NormalizationParams::default())?;12let tensor = input.as_matrix();
Load and preprocess an image
One call from file path to normalised GPU tensor. Resize and ImageNet normalisation happen on the device.
Video_decode.rs
1use oa::{video, Engine};23let engine = Engine::new()?;4let mut source = video::VideoDemuxer::open("clip.mp4")?;5let mut decoder = video::VideoDecoder::create(&engine, source.info())?;67while let Some(packet) = source.read_next_packet()? {8 if let Some(frame) = decoder.decode(&packet)? {9 consume(frame);10 }11}
Decode video to Ml tensor
Open a decoder session, submit a compressed access unit, receive a normalised tensor in the same call.
Media Support
Standard formats. Hardware decode where available.
Vision queries the selected device before opening a session, then admits only the codec, format, queue, conversion, and resolution routes it can support. Hardware-specific decisions stay private while availability remains explicit.
H.264 / AVC
Vulkan Video hardware decode when the selected device exposes the required profile, queue, and formats.
H.265 / HEVC
Capability-gated hardware decode with explicit admission rather than an unconditional platform claim.
AV1
Hardware decode where the device and driver expose the complete codec contract required by the stream.
VP9
A queried hardware path where available, with unsupported devices remaining explicit.
Zero-copy decode path
Decoded frames stay device-resident as they transition from video output to shader-readable work.
Hardware colour conversion
YCbCr-to-RGB uses sampler conversion where admitted and a measured compute route where required.
Ml and preview
One decoded frame can feed a normalized model input and an RGBA preview without a second decode.
Capability admission
Codec, resolution, format, queue, and conversion support are queried before a stateful session opens.
Built on the same stack
Vision is one layer of the Realm platform.
Model execution and the compute substrate remain first-party neighbors, so semantic frames move into learning workloads without changing ownership systems.
