cellmap_flow.finetune.crop_loader

YAML manifest schema and helpers for importing externally annotated crops.

Each manifest entry points at a 3D zarr (typically OME-NGFF) of instance labels or class IDs. The dashboard’s YAML loader (see cellmap_flow.dashboard.routes.finetune.yaml_crops) reads these crops, remaps their values into the trainer’s 0 = unannotated, 1 = background, >=2 = foreground instance convention, and writes them into the session’s annotation_volume.zarr at the crops’ physical offsets.

This module owns:

The pydantic schema (CropEntry, CropsConfig).
The label remap function (remap_labels()).
Small zarr-attrs helpers used by the loader to derive a crop’s voxel size, offset, and the array sub-path inside an OME-NGFF group (the array itself is read with io.source.open_array).

Attributes

`logger`
`YAML_SUFFIXES`
`YAML_SIZE_LIMIT`

Exceptions

YamlFileRefused

A path read_yaml_file() will not read; the message says why.

Classes

`CropEntry`	One annotation crop to import.
`CropsConfig`	Top-level YAML schema.

Functions

`read_yaml_file`(→ str)	Return the text of the YAML file at `path`, or raise `YamlFileRefused`.
`parse_crops_yaml`(→ CropsConfig)	Parse YAML text, or the path to a YAML file, into a validated config.
`remap_labels`(→ numpy.ndarray)	Map source label values to `0=unannotated, 1=BG, >=2=FG instance`.

Module Contents

cellmap_flow.finetune.crop_loader.logger

class cellmap_flow.finetune.crop_loader.CropEntry(/, **data: Any)

One annotation crop to import.

Fields other than path are optional with sensible defaults:

fg_ids=None means “every nonzero source value is foreground”
bg_ids=[] means “no explicit BG ids, see mode for what 0 means”
mode='dense' treats unmatched voxels (incl. 0) as background
mode='sparse' treats unmatched voxels as unannotated
connected_components=False keeps source ids as instance ids; set True with a single-id fg_ids to split same-id blobs into per-instance ids for affinity-style training.

model_config: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

path: str

name: str | None = None

fg_ids: List[int] | None = None

bg_ids: List[int] = None

mode: Literal['dense', 'sparse'] = 'dense'

connected_components: bool = False

class cellmap_flow.finetune.crop_loader.CropsConfig(/, **data: Any)

Top-level YAML schema.

patches_per_epoch, jitter_voxels, and seed are passed through to the VirtualPatchDataset manifest the loader writes — they govern epoch length, patch-center jitter (in voxels), and the per-worker RNG base seed for reproducible patch sampling across runs.

patches_per_epoch=None (the default) means “cover every annotated chunk roughly once per epoch” — the dataset substitutes the number of chunks holding a voxel of either pool at index build time. Override with an explicit int to cap the epoch length.

dense_to_sparse_ratio=None (the default) means “auto-balance”: half the patches are centred in the dense pool (foreground inside the imported crops) and half in the sparse one (every annotated voxel outside them, background included: the painted scribbles) when both have voxels, and all in the one that has when only one does (see finetune.data.sampler).

model_config: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

crops: List[CropEntry]

patches_per_epoch: int | None = None

jitter_voxels: List[int] | None = None

seed: int = 0

dense_to_sparse_ratio: float | None = None

cellmap_flow.finetune.crop_loader.YAML_SUFFIXES = ('.yaml', '.yml')

cellmap_flow.finetune.crop_loader.YAML_SIZE_LIMIT = 1000000

exception cellmap_flow.finetune.crop_loader.YamlFileRefused: A path read_yaml_file() will not read; the message says why.

cellmap_flow.finetune.crop_loader.read_yaml_file(path: str) → str

Return the text of the YAML file at path, or raise YamlFileRefused.

The dashboard listens on every interface without authentication, and two of its routes read a file the request names: read-yaml, for the editor, and load-crops. Both used to read any file the dashboard’s user could, so this reads only a regular file that is YAML by name once symlinks are resolved, and at most 1 MB of it.

A file that is missing, is not YAML, is not a regular file or cannot be opened is refused with the same message, so the answer does not tell a caller which paths exist either.

cellmap_flow.finetune.crop_loader.parse_crops_yaml(yaml_text_or_path: str) → CropsConfig

Parse YAML text, or the path to a YAML file, into a validated config.

A single line that ends in .yaml or .yml is a path, read through read_yaml_file(); anything else is YAML text. Deciding by the name rather than by whether the file exists is what keeps load-crops from opening arbitrary files, or answering differently for one that exists.

cellmap_flow.finetune.crop_loader.remap_labels(source: numpy.ndarray, fg_ids: Iterable[int] | None, bg_ids: Iterable[int], mode: Literal['dense', 'sparse'], connected_components: bool) → numpy.ndarray

Map source label values to 0=unannotated, 1=BG, >=2=FG instance.

Mapping rules:

source value in fg_ids (or any nonzero if fg_ids is None) becomes a unique instance id >=2. If connected_components is True, each connected blob within an fg_id class gets its own instance. Otherwise, source ids map to consecutive 2,3,… in order.
source value in bg_ids -> 1 (background).
everything else -> 1 if mode='dense', else 0 (unannotated).

Returns uint8. If the number of distinct instances would overflow uint8, all FG voxels collapse to id=2 and a warning is emitted.