cellmap_flow.finetune.session.instance ====================================== .. py:module:: cellmap_flow.finetune.session.instance .. autoapi-nested-parse:: Instance corrections: annotation volumes seeded from an instance segmentation. They are ordinary annotation volumes, only with instance-id labels (uint16 or uint32) in the ``AffinityTargetTransform`` scheme: 0 unannotated, 1 the background shell around each instance, and instance id + 1 for each instance. So the periodic sync, the pull before a mirror, session listing and the overlay treat them like any other. The routes in ``dashboard/routes/finetune/instance_correction`` document the workflow; the functions here do the work for them and return ``(success, info or error)``. The volume on disk is written with worldarray; MinIO's copy is read and written with zarr-python over s3fs, which takes the session's own credentials (tensorstore takes S3 credentials only from the environment). Attributes ---------- .. autoapisummary:: cellmap_flow.finetune.session.instance.logger Functions --------- .. autoapisummary:: cellmap_flow.finetune.session.instance.seed_instance_volume cellmap_flow.finetune.session.instance.populated_backing_store cellmap_flow.finetune.session.instance.bucket_key cellmap_flow.finetune.session.instance.snapshot_from_minio cellmap_flow.finetune.session.instance.cc3d_relabel Module Contents --------------- .. py:data:: logger .. py:function:: seed_instance_volume(output_zarr_path, instance_zarr_path, dataset_path, model_name, input_size, input_voxel_size, dilation_radius_voxels=5, chunk_size=None, annotation_dtype='uint16', claimed_input_voxel_size=None, claimed_output_voxel_size=None, input_norm_config=None, postprocess_config=None) Seed a paintable annotation volume at ``output_zarr_path`` from an instance zarr. ``instance_zarr_path`` is a zarr group whose ``s0`` holds the instance ids, with OME multiscales or ``resolution``/``offset`` attributes. The volume lies on that array's grid (shape, voxel size and position), so neuroglancer draws it over the segmentation it came from. Each instance gets a background shell ``dilation_radius_voxels`` thick, in the instance array's voxels (5 at 16 nm is an 80 nm shell). ``chunk_size`` defaults to the instance array's chunks; the dashboard passes the model's output shape, since each chunk is one training sample. ``annotation_dtype`` is "uint16" (up to 65534 instances) or "uint32". The model geometry (``input_size`` in voxels, ``input_voxel_size`` in nm, the claimed voxel sizes) and the chains are recorded in the root attrs, as for any volume. Returns ``(success, zarr path or error)``. .. py:function:: populated_backing_store(state, output_dir, zarr_name) Where MinIO's data directory holds painted chunks of ``zarr_name``, or None. The clobber guard of a fresh seed: re-seeding a zarr whose MinIO copy may hold edits not yet pulled would overwrite them with the seed on the first mirror. It looks at the files rather than asking MinIO because the decision comes before MinIO is started. A running MinIO keeps its data where it was first started, which need not be ``output_dir``, so the path is returned for the refusal to name. .. py:function:: bucket_key(volumes, zarr_path) The MinIO key the volume at ``zarr_path`` is served under. Create serves every attach of a ROI under ``_annotation.zarr``, whatever the zarr on disk is called, and answers with the zarr's own path: a dated snapshot (``roi3_annotation_.zarr``) when one was reattached. So the key is that of the registered volume whose zarr this is (its id plus ".zarr"), and the basename only when none is. ``volumes`` is the volume registry, id -> record. .. py:function:: snapshot_from_minio(state, volumes, zarr_path, dst_path=None) Copy the MinIO state of a paintable instance-correction zarr to disk. Unlike ``sync.sync_volume``, which pulls changed chunks into the served volume, this copies the whole MinIO object, metadata included, so it can make a new zarr: a dated snapshot for rollback or audit, or a copy to train from. ``zarr_path`` names the MinIO bucket key (see :func:`bucket_key`; the registry ``volumes`` is read for it); the zarr itself is not opened. ``dst_path`` defaults to ``zarr_path``, but a fresh dated path (``.../roi3_annotation_.zarr``) is better: the volume stays as it was, a point to roll back to. Either way each chunk is written to a temp file and renamed into place (zarr's DirectoryStore), so a hardlinked copy of the files (a ``cp -rl`` seed) keeps its contents. Returns ``(True, {zarr_path, dst_path, keys_copied, keys_skipped, bytes_copied})`` or ``(False, error)``. .. py:function:: cc3d_relabel(state, volumes, zarr_path, target_label, snapshot_dir=None) Split one label of a paintable instance correction into its connected components. cc3d finds the 26-connected components of ``target_label``'s voxels; the largest keeps the label, and every other one gets a fresh id from ``max(existing) + 1``. ``target_label`` must be 2 or more: 0 is unannotated and 1 is background. The zarr is read and written in MinIO, which holds the in-progress brush edits; ``zarr_path`` only names the bucket key (see :func:`bucket_key`) and where the rollback snapshot goes. That snapshot of MinIO's state is taken before anything is written, to ``snapshot_dir`` (default ``/snapshots``). Returns ``(True, {zarr_path, target_label, n_components, kept_voxels, splits: [{new_label, voxels}], snapshot_path})`` or ``(False, error)``.