smftools.analysis.compute.read_cache

Reader/writer utilities for the per-read modification matrix cache.

Two cache backends are supported:

Zarr cache (recommended)

One Zarr store per run containing all layers. Obs is sorted by (Barcode, Reference_strand) so any barcode × strand combination can be read as a contiguous slice in ~138 ms regardless of which layer is needed. A companion barcode_index.json maps barcode → ref_strand → [start, end] for O(1) slice lookup.

Layout:

<run>.zarr/
    obs/       # all obs columns
    var/       # all var columns (reindexed coords, C_site flags, …)
    layers/
        C_site_binary/
        C_nucleosome_depleted_region_merged/
        …

Example:

from smftools.analysis.compute.read_cache import (
    open_zarr_cache, load_barcode_index, load_zarr_layer,
)

z     = open_zarr_cache("cache/260406_run.zarr")
index = load_barcode_index("cache/260406_run_index.json")
mat   = load_zarr_layer(z, index, "NB01", "6B6_top", "C_site_binary")

Parquet cache (legacy)

One parquet file per barcode × ref_strand × layer.

Layout:

var_info/<ref_strand>_var_info.parquet
<barcode>_<ref_strand>/obs_metadata.parquet
<barcode>_<ref_strand>/<layer_name>.parquet

Parquet columns are str(int(TSS_coord)), e.g. "-1690". Cast back to int with np.array(df.columns, dtype=int).

Functions

build_barcode_index(zarr_path, barcodes, ...)	Build a barcode × ref_strand → [start, end] slice index.
cache_dir(cache_root, barcode, ref_strand)
cache_key(barcode, ref_strand)
is_cached(cache_root, barcode, ref_strand, ...)
load_barcode_index(index_path)	Load a barcode index previously saved by build_barcode_index().
load_layer(cache_root, barcode, ref_strand, ...)	Load a modification matrix layer from the parquet cache.
load_obs_metadata(cache_root, barcode, ...)	Load per-read metadata for a barcode × ref_strand pair.
load_var_info(cache_root, ref_strand)	Load var_info for a reference strand.
load_zarr_layer(z, index, barcode, ...[, ...])	Load one layer for a barcode × ref_strand pair from a Zarr cache.
load_zarr_obs(zarr_path, index, barcode, ...)	Load obs metadata for a barcode × ref_strand pair.
load_zarr_var(zarr_path)	Load the full var DataFrame from a Zarr cache using anndata.
load_zarr_var_fast(z, zarr_path)	Load var columns directly from the Zarr store — avoids full anndata load.
open_zarr_cache(zarr_path)	Open a Zarr store in read-only mode and return the root group.
resolve_barcode_str(barcode_int, index)	Resolve an integer barcode number to its string key in a Zarr barcode index.
write_zarr_cache(adata, zarr_path[, ...])	Write an AnnData to a Zarr cache, sorting obs and optionally subsetting layers.
zarr_cache_exists(zarr_path)	Return True if both the Zarr store and its index file exist.
zarr_index_path(zarr_path)	Return the conventional index path for a given Zarr store path.

smftools.analysis.compute.read_cache.write_zarr_cache(adata, zarr_path, sort_cols=None, layers=None, obs_chunk=512)

Write an AnnData to a Zarr cache, sorting obs and optionally subsetting layers.

Return type:

None

Parameters

adataAnnData

In-memory AnnData to write (backed mode not supported — materialise first).

zarr_pathPath

Destination Zarr store path. Created if it does not exist.

sort_colslist of str, optional

obs column names to sort by before writing, e.g. ["Barcode", "Reference_strand"]. Sorting ensures contiguous slices for fast barcode-level access. Defaults to no sorting.

layerslist of str, optional

Layer names to include. None writes all layers. Non-existent layer names are silently skipped.

obs_chunkint

Number of obs (reads) per Zarr chunk along the obs axis.

smftools.analysis.compute.read_cache.build_barcode_index(zarr_path, barcodes, ref_strands, index_path=None)

Build a barcode × ref_strand → [start, end] slice index.

Call this immediately after write_zarr_cache(), passing the sorted obs arrays from the in-memory AnnData (before it goes out of scope). This avoids decoding the obs from the Zarr store, which requires navigating AnnData's categorical encoding.

Return type:

dict

Parameters

zarr_pathPath

Path to the Zarr store (used only to determine the default index path).

barcodesnp.ndarray

Barcode values in sorted order, matching the Zarr obs rows.

ref_strandsnp.ndarray

Reference strand values in sorted order, matching the Zarr obs rows.

index_pathPath, optional

Where to save the JSON index. Defaults to zarr_path.parent / (zarr_path.stem + "_index.json").

Returns

dict

{barcode: {ref_strand: [start, end]}}

smftools.analysis.compute.read_cache.load_barcode_index(index_path)

Load a barcode index previously saved by build_barcode_index().

Return type:

dict

smftools.analysis.compute.read_cache.zarr_index_path(zarr_path)

Return the conventional index path for a given Zarr store path.

Return type:

Path

smftools.analysis.compute.read_cache.zarr_cache_exists(zarr_path)

Return True if both the Zarr store and its index file exist.

Return type:

bool

smftools.analysis.compute.read_cache.open_zarr_cache(zarr_path)

Open a Zarr store in read-only mode and return the root group.

smftools.analysis.compute.read_cache.load_zarr_layer(z, index, barcode, ref_strand, layer, pos_mask=None)

Load one layer for a barcode × ref_strand pair from a Zarr cache.

Return type:

ndarray

Parameters

zzarr.Group

Open Zarr root group (from open_zarr_cache()).

indexdict

Barcode index from load_barcode_index().

barcodestr

e.g. "NB01".

ref_strandstr

e.g. "6B6_top".

layerstr

Layer name, e.g. "C_site_binary".

pos_masknp.ndarray of bool, optional

Boolean position mask to apply after slicing (column filter). Must have length equal to n_var.

Returns

np.ndarray

Float array of shape (n_reads, n_positions) or (n_reads, n_masked_positions).

smftools.analysis.compute.read_cache.load_zarr_obs(zarr_path, index, barcode, ref_strand)

Load obs metadata for a barcode × ref_strand pair.

Uses anndata to decode the Zarr obs (handles categorical encoding). This is not on the critical performance path — use load_zarr_layer() for fast repeated layer access.

Return type:

DataFrame

Parameters

zarr_pathPath

Path to the Zarr store.

indexdict

Barcode index from load_barcode_index().

Returns

pd.DataFrame

obs rows for the selected barcode × ref_strand.

smftools.analysis.compute.read_cache.load_zarr_var(zarr_path)

Load the full var DataFrame from a Zarr cache using anndata.

Return type:

DataFrame

smftools.analysis.compute.read_cache.load_zarr_var_fast(z, zarr_path)

Load var columns directly from the Zarr store — avoids full anndata load.

Reads each column array via zarr.open_array. ~10x faster than load_zarr_var() for large stores because it bypasses anndata's categorical / sparse decoding overhead.

Return type:

DataFrame

smftools.analysis.compute.read_cache.resolve_barcode_str(barcode_int, index)

Resolve an integer barcode number to its string key in a Zarr barcode index.

Tries NB{n:02d} first (the standard format for most runs), then scans for keys ending in _barcode{n:02d} to handle SQK-style barcodes used by some earlier runs (e.g. SQK-NBD114-24_barcode07).

Raises KeyError if no match is found.

Return type:

str

smftools.analysis.compute.read_cache.cache_key(barcode, ref_strand)

Return type:

str

smftools.analysis.compute.read_cache.cache_dir(cache_root, barcode, ref_strand)

Return type:

Path

smftools.analysis.compute.read_cache.is_cached(cache_root, barcode, ref_strand, layer_name)

Return type:

bool

smftools.analysis.compute.read_cache.load_var_info(cache_root, ref_strand)

Load var_info for a reference strand.

Returns DataFrame with int TSS-coord index and bool columns C_site, GpC_site.

Return type:

DataFrame

smftools.analysis.compute.read_cache.load_obs_metadata(cache_root, barcode, ref_strand)

Load per-read metadata for a barcode × ref_strand pair.

Returns DataFrame indexed by obs_name with all adata.obs columns plus precomputed max_cigar_del (int).

Return type:

DataFrame

smftools.analysis.compute.read_cache.load_layer(cache_root, barcode, ref_strand, layer_name)

Load a modification matrix layer from the parquet cache.

Return type:

tuple[DataFrame, ndarray]

Returns

tuple of (pd.DataFrame, np.ndarray)

DataFrame of shape (n_reads × n_positions) — index is obs_name, columns are str(int(TSS_coord)), values are float (NaN = no coverage) — and an int array of TSS-centred coordinates matching the DataFrame columns.