linumpy.gpu.kvikio_zarr#

kvikio / GPUDirect Storage reader for uncompressed zarr arrays.

This module implements the read_zarr_via_kvikio() backend used by linumpy.gpu.zarr_io. It reads zarr v2 or zarr v3 chunks directly into GPU memory using kvikio.CuFile. The version is auto-detected from the on-disk metadata file (zarr.json for v3, .zarray for v2).

Most callers should use linumpy.gpu.zarr_io.read_zarr_to_gpu(), which dispatches to this backend only when GDS native mode is available and the array is uncompressed; otherwise it falls back to zarr.config.enable_gpu.

Supported on-disk formats#

  • zarr v3: codecs=[{"name": "bytes"}] only (or empty) — raw little/big- endian bytes. Any compression codec (blosc, gzip, zstd, …) is rejected because GDS reads bytes verbatim and on-device decompression would require nvCOMP.

  • zarr v2: compressor=None, filters=None, order='C'.

Notes

  • Requires kvikio and cupy. Both are imported lazily so the rest of linumpy is unaffected if they are not installed.

  • For the GDS fast path the source filesystem must support GDS natively (ext4 on local NVMe, IOMMU disabled or in passthrough, and properties.use_compat_mode=false in /etc/cufile.json). Otherwise kvikio falls back to a posix bounce-buffer path with no speed-up.

Functions#

read_zarr_via_kvikio(array_path)

Load a full uncompressed zarr (v2 or v3) array into a CuPy array via GDS.

Module Contents#

linumpy.gpu.kvikio_zarr.read_zarr_via_kvikio(array_path)[source]#

Load a full uncompressed zarr (v2 or v3) array into a CuPy array via GDS.

Parameters:

array_path (str | pathlib.Path) – Path to the zarr array directory (containing zarr.json for v3 or .zarray for v2).

Returns:

Device-resident array of shape and dtype matching the zarr metadata.

Return type:

cupy.ndarray