Dataset and Sample
Datasets expose indexed retrieval and stable size. Each sample carries an input Matrix and an optional target Matrix without creating another tensor type.
Data · Rust
Data into models.
Contracts intact.
Index, split, collate, and batch semantic samples without inventing a second tensor system or claiming a hidden streaming runtime.
Current surface
Small enough to understand. Complete enough to train.
OA Data owns deterministic dataset composition and batch iteration. Domain transforms stay with Vision, Audio, and Ml; device execution remains with the shared engine.
Boundary
The old streaming experiments are not the product.
Earlier OA trees explored broad data infrastructure. The current public contract keeps only the parts with clear ownership and tests; advanced sources return when a real consumer can prove them.
No hidden I/O
The current Data surface does not claim an implicit filesystem, network, mmap, or io_uring backend.
No phantom workers
There is no invisible prefetch pool, background upload path, or infinite-streaming runtime behind DataLoader.
No second tensor system
Samples and batches use OA Matrix values rather than crossing into a parallel host array or framework contract.
Future work stays bounded
Async sources and prefetch return only as explicit sessions with cancellation, backpressure, completion, and workload evidence.