YAML Configuration

cellmap_flow yaml lets you define and run multiple models from a single YAML file. It is the recommended way to launch inference jobs, and the same YAML format is used by the blockwise processor (cellmap_flow blockwise).

Usage

Run inference; check the YAML without running it; list the model types; and run with more logging:

cellmap_flow yaml config.yaml
cellmap_flow yaml config.yaml --validate-only
cellmap_flow yaml --list-types
cellmap_flow yaml config.yaml --log-level DEBUG

YAML Structure

A configuration file has the following top-level fields:

Field	Required	Description
`data_path`	Yes	Path to the input dataset: zarr, N5 or precomputed, on disk or at an `http(s)://`, `s3://` or `gs://` URL (see Data paths).
`charge_group`	Yes	Project billing group.
`queue`	No	The GPU queue the jobs run on, or a list of them (`[gpu_h100, gpu_h200]`): each job goes to the one with the most free GPUs, and moves to the next if it has not started in 3 minutes (`cycle_gpu_queues: false` keeps it on the first). Default: the dashboard’s saved queues, else every GPU queue.
`models`	Yes	Dict or list of model entries (see below).
`json_data`	No	Input normalizers and postprocessors.
`wrap_raw`	No	Wrap raw data in neuroglancer (default: `true`).
`resample`	No	`true` (the default) resamples the data to each model’s input voxel size when it has no level at that size; `false` reads the nearest level as if it were at that size (see Data at Another Voxel Size).
`extra_layers`	No	More volumes to show in the viewer beside the raw data (see below).
`output_path`	No	Output zarr path (used by blockwise processing).
`task_name`	No	Task name (used by blockwise processing).
`workers`	No	Number of GPU workers (blockwise).
`cpu_workers`	No	Number of CPU workers (blockwise).
`tmp_dir`	No	Temporary directory for intermediate files.
`bounding_boxes`	No	Regions to process instead of the whole volume, each `{offset, shape}` in nm, z, y, x; the output covers only them (blockwise, see Bounding Boxes).
`separate_bounding_boxes_zarrs`	No	Write each bounding box to its own output, `<output_path>/box_<n>` (blockwise).
`output_channels`	No	Which model channels blockwise writes, and to which outputs (see Output Channels).

Model Entries

Each model entry requires a type field and the parameters for that model type. Use cellmap_flow yaml --list-types to see all available types and their required parameters.

Models can be specified as a dict (keys become model names) or a list (each entry must include a name field).

Dict format (recommended):

models:
  my_mito_model:
    type: fly
    checkpoint: /path/to/checkpoint
    resolution: 16
    classes:
      - mito
  my_dacapo_model:
    type: dacapo
    run_name: my_run
    iteration: 100

List format:

models:
  - name: my_mito_model
    type: fly
    checkpoint: /path/to/checkpoint
    resolution: 16
    classes:
      - mito

Available Model Types

Type	Class	Key Parameters
`script`	ScriptModelConfig	`script_path` (required)
`dacapo`	DaCapoModelConfig	`run_name` (required), `iteration` (required)
`fly`	FlyModelConfig	`checkpoint` (required); `classes`, `resolution`, `input_size` and `output_size` when the run’s folder does not say them; `sigmoid`. See fly_organelles checkpoints.
`bioimage`	BioModelConfig	`model_name` or `model_path` (required), `voxel_size` (required)
`cellmap`	CellMapModelConfig	`config_folder` (required)
`huggingface`	HuggingFaceModelConfig	`repo` (required), `revision` (optional). See HuggingFace Models.
`cellpose`	CellposeModelConfig	`voxel_size` (required), `pretrained_model`, `output`. See Cellpose.
`onnx`	OnnxModelConfig	`path` (an `.onnx`) or `folder` (of `repo`, cellmap/onnx-web-models by default), with `file` or `variant`; `input_voxel_size` for a file without a manifest. See ONNX exports and models finetuned in the browser.

Common optional parameters: name, scale, env (see Running a model in its own environment), run_voxel_size (see Running a model at another voxel size).

An entry from what you have

cellmap_flow add REF writes the entry for you. REF is whatever you have in hand, and the entry is printed as YAML to paste under models:, with comments saying what REF was taken to be, where it runs, and what it still needs:

$ cellmap_flow add cpsam
# cellpose: a Cellpose 4 pretrained model
# runs in the cellpose4 environment (its type's default)
# still needs: voxel_size (add to the entry)
models:
  cpsam:
    type: cellpose
    pretrained_model: cpsam

-n names the model and -v 16,8,8 gives the voxel size of a model that does not say its own (Cellpose, a bioimage.io model cellmap-flow has no trained voxel size for); a zoo EM model’s entry gets the one it was trained at, with a note saying where it comes from. --run -d DATA (with -q, -P and --resample as infer takes them) serves the model right away, as cellmap_flow yaml would.

In the dashboard, the same REF goes in the Models tab’s Add a model box. Resolve (POST /api/models/resolve) shows what it was taken to be and where it runs, with a box for each value it still needs; Run (POST /api/models/add) starts it, and it joins the models with a ticked box, stopped when unticked like any other or by its ✕ under Loaded models (see Usage on the front page); ticked again, it is started again. A config YAML (one with models:) is refused with how to run it: it is a whole run, for cellmap_flow yaml.

REF is checked against these in order, and the first that matches wins:

A local file or folder: a .py is a script; a folder with metadata.json (and model.ts) a cellmap export; a folder with rdf.yaml or bioimageio.yaml, such a file, or a .zip with one inside a bioimage model; model_checkpoint_<n>, a .ts or a model.pt a fly model (its channels and voxel sizes read from its run’s folder, see fly_organelles checkpoints); a folder with adapter_config.json (a LoRA adapter) or a full finetune’s model_state_dict.pt a finetune, which still needs its base_model; Cellpose-SAM weights a cellpose model; an .onnx an onnx model, which needs input_voxel_size unless it carries a manifest (one finetuned in the browser does).
A prefix that says outright what it is: hf:org/repo[@revision], bioimageio:<id, nickname, URL or path>, cellpose:<name or path>, fly:<path>, dacapo:<run>@<iteration>, script:<path>, onnx:<path, folder or org/repo/folder>.
A Cellpose model name: cpsam_v2, cpsam, cpdino, cpdino-vitb.
A URL: a bioimage.io model page, a Zenodo record, a DOI link or the URL of an rdf.yaml or zip is a bioimage model; huggingface.co/org/repo is as 5, and with a folder after its revision (.../tree/main/setup_54) an onnx model.
org/repo/folder: a folder of a repo of ONNX web exports (one with its manifest.json, as cellmap/onnx-web-models holds them), an onnx model. org/repo: a Hugging Face repo, which must be a cellmap-models export (metadata.json and model.ts). A repo of web exports is refused with its folders, one of which to add; any other model on the Hub is refused: wrap it in a script (Custom Model).
A BioImage Model Zoo nickname or id (affable-shark, 10.5281/zenodo.5764892), looked up in the zoo’s model list, the one the Models tab shows (bioimage.io’s server, else its legacy index).
A folder of cellmap/onnx-web-models by its name (setup_54): an onnx model.

5, 6 and 7 are checked online; with --offline (or when the Hub or zoo cannot be reached) a repo, folder or zoo-shaped name is taken on trust, with a note, and a bare folder name needs its onnx: prefix. Anything else is an error saying what was tried; a prefix settles it.

Running a model in its own environment

A model whose packages conflict with cellmap-flow’s environment (Cellpose 4, transformer models) can run in an environment of its own. Give its entry an env:

models:
  my_model:
    type: script
    script_path: /path/to/my_model.py
    env: cellpose4

env is either

the name of an environment in cellmap-flow’s pixi.toml. The server runs as pixi run --frozen --manifest-path <pixi.toml> -e cellpose4 cellmap_flow serve ..., and pixi installs the environment from the lockfile the first time. The manifest is the one in the checkout cellmap-flow is installed from; set CELLMAP_FLOW_PIXI_MANIFEST to use another. The environments of cellmap-flow’s pixi.toml are listed below; or
the absolute path of a conda environment or virtualenv with cellmap-flow installed. The server runs as <path>/bin/python -P -m cellmap_flow.cli.main serve ....

A value with a / in it, or starting with ~, is a path. An unknown name, or a path without bin/python, is an error when the YAML is read.

The environments of cellmap-flow’s pixi.toml. A model type with a default runs there when its entry gives no env; an explicit env wins, and env: current keeps it in the environment cellmap-flow runs in.

Environment	What it is for	Default for
`default`	The dashboard, the catalog and finetuning; what Fileglancer deploys. `test` and `dev` add pytest and the linters to it.	types without a default of their own, which run where cellmap-flow runs
`fly`	fly_organelles (`mzouink/fly-organelles` at `ab89c10`), whose `StandardUnet` loads a raw training checkpoint and whose classes an eager `model.pt` unpickles. Solved on its own.	`fly`, unless the checkpoint is TorchScript (`.ts`)
`bioimageio`	`bioimageio.core` 0.11 with its ONNX and PyTorch backends, for BioImage Model Zoo models. `example/bioimage_em.yaml` runs one.	`bioimage`
`onnx`	ONNX Runtime: `onnxruntime-gpu` (its CUDA provider, CUDA 13 as the torch wheels) on Linux, `onnxruntime` (CPU) on the Mac, for the browser mode’s ONNX exports and the models finetuned there. Solved on its own.	`onnx`
`cellpose4`	Cellpose 4 (Cellpose-SAM). Solved on its own, so Cellpose’s pins do not move the default environment’s packages.	`cellpose`
`dacapo`	`dacapo-ml`. Its lock does not import DaCapo yet (`dacapo-ml` 0.3.0 with fibsem-tools 7, which dropped `fibsem_tools.metadata`), so a DaCapo model is served from a conda environment that has it.	none
`docs`	Building these docs.	none

Only the model’s inference server and its finetuning job run there. The dashboard reads the model’s geometry from its running server rather than building the model itself; blockwise runs models in its own process, so run pixi run -e cellpose4 cellmap_flow blockwise ... for such a model. A finetuning job needs peft in the environment: a pixi environment that does not install it is refused when the job is submitted.

cellmap_flow infer <type> --env <env> and the dashboard’s model form take it too. The cellpose type runs in cellpose4 without one (see Cellpose).

ONNX exports and models finetuned in the browser

type: onnx serves an ONNX model with ONNX Runtime, fed exactly as the browser mode feeds it: one block of float32 [1, 1, z, y, x] a call (an fp16 export whose graph takes float16 gets float16), [1, C, z', y', x'] out, served as float32. Its server runs in pixi’s onnx environment, on the GPU when the job has one. The model is either

folder: a model of a Hugging Face repo of web exports, repo (cellmap/onnx-web-models by default) at revision. Its manifest.json gives the voxel sizes, the channels and the variants, one .onnx per precision and block; file names the one to serve (what the browser’s Export Config writes: the one the page ran), else variant (fp32 by default, fp16, int8w, int8w_fp16) and input_size choose, by default the largest block. Both are downloaded with huggingface_hub on first use, as a huggingface model is.
path: a local .onnx. The one the browser’s Download finetuned model saves carries its manifest in the file’s metadata_props (cellmap_flow.manifest), so it needs nothing else. Any other file needs input_voxel_size; its blocks and channel count are read from its graph’s input and output shapes, or given as input_size, output_size and channels when those are not fixed. A relative path is taken from the YAML file’s folder.

models:
  mito_aff_setup54_16nm:
    type: onnx
    repo: cellmap/onnx-web-models
    folder: setup_54
    file: setup_54_mito_affinities_16nm_in132_fp16.onnx
  mito_aff_setup54_16nm-ft1:
    type: onnx
    path: mito_aff_setup54_16nm-ft1.onnx   # beside this YAML

The browser’s Export Config (in its header) writes such a YAML: the volume’s URL as data_path, the resample setting, the Processing tab’s steps as json_data, and each model it serves as a type: onnx entry, a Hugging Face export by repo, folder and the file the page ran, a model finetuned there by path: <name>.onnx (the file Download finetuned model saves), with a comment saying to put that file beside the YAML. It writes the entries as a list, each with its name (both forms load, see Model Entries), and leaves out charge_group and queue: add them to run on the cluster.

What the entry gives (input_voxel_size, output_voxel_size, the sizes, channels, output) wins over the manifest; run at another input_voxel_size, the output keeps the manifest’s ratio to it. A 2D student (input and output one slice thick, setup_53_2d) is given one z slice a call, slices_per_chunk (8) slices a chunk.

The exports output logits (their manifests’ output): add a SigmoidPostprocessor, or the affinity chain for affinities; the warmup’s output probe sees the output unbounded, and the dashboard suggests the same. Their manifests also declare the input normalization they were trained behind (input_norm), which the dashboard offers with an Apply button; the YAML’s json_data is what runs, as for every type. An ONNX model cannot be finetuned on the server (the trainer trains torch modules); the browser finetunes an export’s last layer (or, for the 2D exports, its whole network), and the result is served here. example/onnx_web_models.yaml runs two exports.

fly_organelles checkpoints

type: fly serves a network fly_organelles trained, from one checkpoint file:

a training checkpoint (model_checkpoint_<iteration>), loaded into fly_organelles’ StandardUnet; its feature maps, levels and kernel sizes are read from the weights;
a TorchScript file (.ts);
a whole pickled model (model.pt), unpickled only when CELLMAP_FLOW_ALLOW_PICKLE=1 is set.

A training checkpoint or a model.pt runs in the fly pixi environment unless the entry gives an env; a .ts runs in any. A folder that cellmap_models exported (metadata.json and model.ts) is served as exported by type: cellmap, folder_path: <the folder>, and refused here. The model.pt in such a folder can be served here, at a larger input_size than the one it was exported at.

What the entry does not give is read from the checkpoint’s folder. A fly_organelles training run is enough as it is:

models:
  mito_distance_16:
    type: fly
    checkpoint: /groups/cellmap/cellmap/zouinkhim/salevary/train/v2/distance/mito_16_all/model_checkpoint_20000

reads the channel names from the run’s train.py and the voxel sizes and tile from its training snapshots. Each value comes from the first of these files that has it; a key the entry gives always wins:

File	What is read
`metadata.json`	A cellmap_models export’s: `channels_names`, the voxel sizes, `input_shape` and `output_shape`.
`config.yaml`	fly_organelles’ run configuration: `run.labels`, `run.voxel_size`, `checkpoint.input_shape` and `output_shape`.
`snapshots/`	The newest training snapshot’s `raw` and `output` arrays: input and output size, their voxel sizes, and how many channels the network outputs.
`train.py`	Its top-level `labels = [...]` and `voxel_size = ...`, when they are written out as literals. The script is parsed, never run.

Labels are taken as channel names only when the snapshots show one channel per label; an affinity or LSD run’s network outputs several per label, and its entry has to name them. The voxel sizes are read only when the entry gives neither, and the input and output size only when it gives neither: an output size belongs to the input size it came with.

Parameter	Default	Meaning
`checkpoint` (`checkpoint_path`)	(required)	The checkpoint file.
`classes` (`channels`)	read from the folder	The name of each output channel, one per channel. An error when neither the entry nor the folder gives them.
`resolution` (`input_voxel_size`), `output_resolution` (`output_voxel_size`)	read from the folder	nm per voxel, one number or one per axis; fractions are kept. One stands for the other, as fly_organelles trains at one voxel size. An error when neither the entry nor the folder gives one.
`input_size`, `output_size`	read from the folder, else 178 and computed	Voxels a side of a tile in and out. Without either, 178 goes in (fly_organelles’ training tile) and what comes out is computed from the network. `input_size` alone is enough; `output_size` alone is an error. Larger tiles waste less context: a StandardUnet takes 178 + 16k (194, 338, …).
`sigmoid`	`true`	Pass the output through a sigmoid, which is added unless the network ends in one already (a cellmap_models export does). fly_organelles trains on logits, so a training checkpoint gets one. `false` serves the network’s output as it is.

BioImage Model Zoo

type: bioimage runs a BioImage Model Zoo model through bioimageio.core, with the model’s own preprocessing (its normalization) and postprocessing (a sigmoid, say). It runs in the bioimageio pixi environment unless the entry gives an env. example/bioimage_em.yaml serves “conscientious-dromedary” (a 3D mitochondria U-Net) on jrc_mus-salivary-1.

models:
  mito_bioimageio:
    type: bioimage
    model: conscientious-dromedary
    voxel_size: 16     # without it: 30x8x8, the voxel size it was trained at

model is anything bioimageio.core loads: a zoo id or nickname (“conscientious-dromedary”, “affable-shark”), a DOI or URL, or the path of a model’s rdf.yaml (bioimageio.yaml) or packaged .zip. The model is downloaded to bioimageio’s cache ($BIOIMAGEIO_CACHE_PATH, by default ~/.cache/bioimageio) when its server starts. Before 0.3.0 the key was model_name; it, and model_path, are still read.

The model’s description says the rest, unless the entry does:

Parameter	Default	Meaning
`voxel_size`	the description’s, else the one it was trained at	nm per input voxel, one number or z, y, x: the level the model reads. The description gives one when its input’s space axes have a unit (`scale` and `unit`). No zoo model does, so for the zoo’s EM models cellmap-flow uses the voxel size each was trained at, looked up by hand (`cellmap_flow/models/bioimage_voxel_sizes.yaml`, with each number’s source; some were trained on downsampled data, the Platynereis nuclei model at 100x80x80). Any other model needs it. A 2D model’s z is the spacing of its slices, its y by default.
`input_size`	256 (2D), 128 (3D)	Voxels a side of the tile the model is given, one number or one per space axis of the model (z, y, x; y, x for a 2D model). An axis of fixed size keeps it; a parameterized one (`min + n * step`) gets the smallest size it takes of at least this.
`context`	the description’s halo, else 0	Voxels cut off each side of the model’s output, one number or one per space axis, so that chunks meet where the model saw both sides of the seam. The input is read that much larger. The zoo’s EM U-Nets give no halo, and their chunks show seams; a `context` of 16 or so hides them at the cost of computing the overlap twice.
`slices_per_chunk`	8	z slices in a chunk of a 2D model, each segmented on its own (in one call when the model’s batch axis takes any size). A 3D model ignores it.
`weight_format`	bioimageio.core’s choice	The weights to run, of those the model has: `torchscript`, `pytorch_state_dict`, `onnx`, `tensorflow_saved_model_bundle`, `keras_hdf5` or `keras_v3`.

Every output tensor is served, its channels one after another, as the model’s postprocessing leaves it: float32, not clipped or rescaled, unless the description says the output is integers (labels), which keep their type. A 2D model’s labels are its own, slice by slice and chunk by chunk: MortonSegmentationRelabeling makes them unique across chunks, as for Cellpose’s masks, but not from one slice to the next.

It cannot run a model with more than one required input, an input with more than one channel (RGB), or an output that is not a map over the input’s space, such as micro-SAM’s (“noisy-ox”, “humorous-crab”), whose masks come one per prompted object. A model whose code needs packages the bioimageio environment lacks (its weights’ dependencies name them) fails when its server builds it; the environment has timm, which the zoo’s BiaPy transformers need. Empanada’s MitoNet (“stupendous-sheep”) needs OpenCV and loads a file from beside its code, where bioimageio.core 0.11 does not put it, so it does not run.

The zoo’s 2D EM U-Nets (“gleeful-skunk”, “jolly-duck”, “good-microbe”) find jrc_mus-salivary-1’s mitochondria at 8 nm; “conscientious-dromedary” at 16 nm.

Cellpose

type: cellpose runs Cellpose 4 (Cellpose-SAM) on each z slice of a chunk, in 2D, or on the whole chunk in 3D (do_3d, below), and serves its cell probability, its flows or its instance masks. It runs in the cellpose4 pixi environment unless the entry gives an env: the default environment has no Cellpose. example/cellpose_sam.yaml serves it on jrc_mus-salivary-1.

models:
  cellpose_sam:
    type: cellpose
    voxel_size: 64
    output: flows      # the default: all three channels

Parameter	Default	Meaning
`voxel_size`	(required)	nm per voxel, read and written; one number or one per axis. Cellpose-SAM sees objects best about 30 voxels across, so pick the voxel size at which yours are roughly that (or give `diameter`).
`pretrained_model`	`cpsam_v2`	`cpsam_v2`, `cpsam`, `cpdino`, `cpdino-vitb`, or the path of finetuned Cellpose weights. The named ones are downloaded from the Hugging Face Hub (about 1 GB) to `~/.cellpose/models`, or `$CELLPOSE_LOCAL_MODELS_PATH`, when the server starts. The DINO models also need facebookresearch’s `dinov3` package, which the `cellpose4` environment does not install.
`output`	`flows`	`flows`, `probability` or `masks`: see the table below.
`slices_per_chunk`	8	z slices in a chunk.
`slice_size`	512	Voxels a side, in y and x, of each chunk’s slices.
`context`	32	Voxels read beyond them on each side in y and x, so that objects at a chunk’s edge are seen whole, and cut off again. None in z.
`batch_size`	16	Tiles per GPU pass. Cellpose cuts each slice into tiles (256 px for `cpsam*`, 384 for the DINO models, overlapping by 10%), and runs this many at a time, from any of the chunk’s slices. More is no faster: Cellpose-SAM took about as long per tile at 16 as at 72 or 128 on an L4 and an H100, and 2.4 GB at 16 against 9 GB at 72. A network finetuned in the trainer’s live server runs half as many, as it is float32 there.
`diameter`	none	Object diameter in voxels; Cellpose resizes each slice by 30 / `diameter`. None keeps the model’s own scale.
`flow_threshold`, `cellprob_threshold`	0.4, 0.0	Cellpose’s mask thresholds; `masks` only.
`stitch_threshold`	0 (off)	`masks` only: Cellpose’s own linking of masks from slice to slice. A mask takes the id of the mask in the slice before that it overlaps by at least this IoU (0 to 1), so an object keeps one id through the chunk’s slices. Within a chunk only, and Cellpose then normalizes the chunk’s slices together rather than each on its own. Another output refuses it, and so does 3D.
`do_3d`	false	Segment each chunk in 3D (see In 3D below). `slice_size` and `context` then default to 192 and 24, in z too.
`flow3d_smooth`	0 (off)	In 3D, the sigma in voxels of a Gaussian smoothing of the flows, if objects break up along z.

What each output serves:

`output`	Channels	dtype	What it is
`flows` (the default)	`flow_y`, `flow_x`, `cell`	float32	“Flows + probability” on the Models tab, and not masks: all the network predicts, the flows towards each object’s centre, in y and in x (Cellpose’s `dP`, about -5 to 5), then the cell probability, 0 to 1 (the sigmoid of its logit). The layer shows them together: each voxel coloured by its flow’s direction, dimmed by `cell` (the model’s `display_channel`); untick the shader’s `flows` box to see `cell` alone. `CellposeMasksPostprocessor` makes masks of them on the server, with thresholds that can be changed from the dashboard.
`probability`	`cell`	float32	The cell probability alone.
`masks`	`cell`	uint64	Instance ids, unique within a chunk.

The probability and the flows are computed voxel by voxel, so they join up across chunks, and they skip the mask dynamics, which makes them the faster outputs. Masks are made per chunk: an object that crosses a chunk’s edge is cut there, with an id on each side, and objects are joined from slice to slice only by stitch_threshold, within the chunk. Add the MortonSegmentationRelabeling postprocessor to make the ids unique across chunks and show the layer as a segmentation:

json_data:
  postprocess:
    - name: MortonSegmentationRelabeling

For masks of a whole volume, write the probability with pixi run -e cellpose4 cellmap_flow blockwise ... (blockwise runs the model in its own process) and segment that, or run Cellpose’s own distributed segmentation (cellpose.contrib.distributed_segmentation), which stitches objects across blocks.

In 3D (do_3d: true, Cellpose’s do_3D), each chunk is segmented as one 3D image:

Cellpose runs its 2D network on every YX, ZY and ZX plane of the chunk and sums the three into 3D flows.
Masks follow those flows in 3D, so an object keeps one id through z, with no slice linking needed.
flows gains a flow_z channel first: flow_z, flow_y, flow_x, cell. The cell logit is the sum of the three orientations’, as in Cellpose.
A chunk is a cube with context on every side. The defaults are 192 voxels (slice_size) with 24 of context (context), so each plane is one 256 px tile.
With anisotropic voxels, z gets as many nm as y and x, and Cellpose is given the anisotropy.
It costs about 3× what slice by slice does for the same voxels: about 5 s for a 192³ chunk on an H100.
flow3d_smooth (a sigma in voxels) smooths the 3D flows first, if objects break up along z. stitch_threshold is 2D only.

models:
  cells_3d:
    type: cellpose
    voxel_size: 16
    do_3d: true
    output: masks

In the dashboard, the Models tab’s Cellpose panel lists Cellpose-SAM v2 (cpsam_v2) and Cellpose-SAM (cpsam); the DINO models are left out, as the cellpose4 environment cannot run them. Tick one, enter its voxel size (required: the scale at which the objects are about 30 voxels across) and pick its output (and for masks, Link slices: stitch_threshold), then Submit Models. Its job and layer are named after the model and any output but the default (cellpose_sam_v2, cellpose_sam_v2_masks), so + output runs another output of the same model beside it. Unticking stops it; changing the voxel size or the linking of a running one restarts it; a reloaded page ticks the running ones with their settings. A row’s 3D box runs it in 3D, as its own job and layer ending in _3d, so a 2D and a 3D row can be compared side by side.

A finetuned Cellpose model is trained on the flows whatever its output, and served after training with its base model’s output.

Finetuned weights, from Cellpose’s GUI or cellpose.train, are a path: pretrained_model: /path/to/models/my_model. Cellpose reads from the weights which network they are, and the tiling follows.

Licence: the Cellpose-SAM weights were trained on data that includes datasets licensed CC-BY-NC, so they are for non-commercial use.

Each model type’s default

Some model types run in an environment of their own unless the entry says otherwise, so their entries need no env:

Type	Default `env`
`cellpose`	`cellpose4`
`bioimage`	`bioimageio`
`dacapo`	`dacapo`
`fly`, with a raw checkpoint	`fly`
`finetune`	its base model type’s
`script`, `cellmap`, `huggingface`	none: this environment

An entry’s own env wins. env: current runs the model in this environment whatever its type’s default (default is not that: it is pixi’s default environment). Exported YAMLs write only an env the entry gave, never the type’s default. A plugin type sets its own as default_env on its ModelConfig subclass.

A default this machine cannot provide does not stop the model: with no pixi.toml, no environment of that name in it, or no pixi at all (a conda-only account), the model runs in this environment, as before its type had a default, with a warning that says how to give it one. A default that is in pixi.toml but not installed yet is used: it is installed before the model’s first job is submitted (several minutes, once; Job Logs shows pixi’s output meanwhile), not in the job, whose time to start a server would run out first. An env the entry names itself is an error when it cannot be used.

Aliases

~/.cellmap_flow/envs.yaml (or the file CELLMAP_FLOW_ENVS_FILE names) maps environment names to conda environments or virtualenvs with cellmap-flow installed. An alias wins over a pixi environment of the same name, both for an entry’s env and for a type’s default, so this is how a machine without pixi runs the types’ defaults:

cellpose4: /groups/lab/home/me/miniconda3/envs/cellpose4
dacapo: /groups/lab/home/me/miniconda3/envs/dacapo

A model whose env is an alias keeps the name in exported YAMLs, so the YAML works on a machine that has the pixi environment instead. An alias to a relative path, or to a directory without bin/python, is an error.

`cellmap_flow envs`

cellmap_flow envs alone lists each environment, as envs list does:

cellmap_flow envs
cellmap_flow envs install cellpose4
cellmap_flow envs check cellpose4

envs list shows each environment of pixi.toml and each alias: where it is, whether it is installed, whether it can run a finetuning job, and which model types default to it (a type that decides per model, as fly does, is listed apart). envs install installs a pixi environment from the lockfile now (pixi install --frozen), rather than on its first job; an alias has nothing to install. envs check imports cellmap-flow with the environment’s python, to show that it works there.

Channel Names

A model names its output channels, in order, with one of:

channels: a list in a model script, or a fly entry’s parameter. In a fly entry, classes is the same parameter under another name, so give one of them; a fly entry may also give its names as one comma-separated string, classes: mito,er.
channels_names: read from the metadata.json of a huggingface or cellmap model.
classes: a list in a model script.

A DaCapo model takes its names from its run’s task. A bio model names none.

When a model gives more than one of these, the first that is not empty wins, in the order channels, channels_names, classes. In a model script a single string is one name: channels = "mito" is one channel called mito.

The inference server reports the names with the model’s geometry, and blockwise names its outputs by them (see Output Channels). A model that names no channels still serves; blockwise then needs output_channels as a mapping of channel indices.

Normalizers and Postprocessors

Define input normalization and output postprocessing under json_data:

json_data:
  input_norm:
    MinMaxNormalizer:
      min_value: 0
      max_value: 250
      invert: false
    LambdaNormalizer:
      expression: "x*2-1"
  postprocess:
    DefaultPostprocessor:
      clip_min: 0
      clip_max: 1.0
      bias: 0.0
      multiplier: 127.5
    ThresholdPostprocessor:
      threshold: 0.5

Normalizers are applied in order before inference. Postprocessors are applied in order after inference.

A model reads one channel of a raw that has several (light microscopy’s OME-Zarr with a c axis, an N5 or precomputed volume of more than one channel): channel 0, or the one a ChannelSelector step picks, first in input_norm:

json_data:
  input_norm:
    ChannelSelector:
      channel: 1

In the dashboard this is the Channel at the top of the Processing tab’s input normalization, shown for such a raw and named by its OME-Zarr omero channel labels when it has them. A pick applies at once, and the models, the raw layer, the finetune trainer, the ranking, Label it and AI annotation all read that channel; a model’s start and Use this model keep it in front of the normalization they set.

Data at Another Voxel Size

A model reads the level of data_path at its input voxel size. When there is no such level, what happens depends on resample:

resample: false: the level closest to the model’s voxel size that is not coarser on any axis (the finest level, when every one is) is read as if it were at the model’s voxel size, voxel for voxel, with a warning. The model then sees data at the wrong scale, and its predictions are drawn where that level really is, at a proportionally different voxel size.
Left out, or resample: true: a level is resampled to the model’s voxel size, axis by axis, and the model sees the data at the size it was trained at. Its predictions are at its declared output voxel size.

data_path: /nrs/cellmap/data/my_dataset/my_dataset.zarr/recon-1/em/fibsem-uint8
charge_group: cellmap
# levels 8x8x40, 16x16x80 ... nm; the model wants 16x16x16, so a level is
# resampled to it (resample: false would read 8x8x40 as if it were)

models:
  my_model:
    type: dacapo
    run_name: my_run
    iteration: 50000

How the data is resampled:

Which level. The coarsest level that is no coarser than the model’s voxel size on any axis, so every axis is downsampled from as close as the pyramid allows; when every level is coarser on some axis, the finest level. A level at exactly the model’s voxel size is read as it is.
Each axis by its own factor. By a whole number of voxels (8 nm to 16 nm), each voxel is the mean of the voxels it covers. Otherwise (40 nm to 16 nm, or 12 nm to 16 nm), it is a linear interpolation between the two nearest voxels. Label data (bool, or integers of 32 bits or more) takes the nearest voxel instead, so no label is invented. Intensities stay in their dtype (uint8 stays uint8, rounded), and go through json_data’s normalizers after resampling, as a stored level would.
Where. The resampled grid starts at the level’s own corner, so the predictions lie over the data. Chunks are resampled on their own, and each gives exactly the voxels a read of the whole volume would.

cellmap_flow yaml starts each server with --resample (--no-resample for resample: false), and cellmap_flow blockwise reads the data the same way. cellmap_flow infer <type> resamples too, unless given --no-resample (Command line).

In the dashboard it is the Models tab’s Resample if no scale matches the model box, ticked unless resample: false (or cellmap_flow view --no-resample) starts it unticked. It applies to the models submitted from then on, to blockwise runs, and to annotation volumes made while it is on: those are at the model’s own voxel sizes, and their finetunes train on the data resampled and are served resampled. When a running model reads a level as if it were at its voxel size, the banner above the tabs says so.

Running a model at another voxel size

run_voxel_size runs a model at a voxel size of your choosing instead of its own: one number for every axis, or [z, y, x] in nm. Each axis is scaled by the run voxel size over the model’s input voxel size: the model reads the data at the run voxel size, writes at its output voxel size scaled alike, and its read and write shapes in nm scale with them, while its block sizes in voxels stay its own. A 16 nm mitochondria model run on the 32 nm level:

models:
  mito_at_32nm:
    type: huggingface
    repo: cellmap/mito-aff-unet-setup-16
    run_voxel_size: 32        # or [32, 16, 16]

The level read is then the one at 32 nm, as for a 32 nm model (and Data at Another Voxel Size when there is none), and everything downstream takes the model as a 32 nm one: its layer, a finetune’s annotation volume and training data, the ranking, blockwise, and what a finetune of it declares, which is a 32 nm model. Left out, the model runs at its own voxel size.

A model that reads its input itself, through its own process_chunk or predict (a TensorFlow script, a Cellpose or BioImage Model Zoo model), works out what to read in nm at its own voxel size, and refuses run_voxel_size; give Cellpose and zoo models their voxel_size instead. In the dashboard it is the Run at (nm) field under a ticked Local Catalog or Hugging Face model on the Models tab, and cellmap_flow add REF -v 32 writes it for a model that has no voxel size argument of its own.

Extra Layers

cellmap_flow yaml can show more volumes in the viewer beside the raw data and the predictions, for instance an earlier prediction or an instance segmentation. Each is read as stored, without the input normalizers:

extra_layers:
  - name: base_mito
    path: /nrs/cellmap/predictions/mito.zarr/mito
    shader: |                      # optional, image layers only
      void main() { emitRGB(vec3(0, toNormalized(getDataValue()), 0)); }
    blend: additive                # optional, image layers only
  - name: instances
    path: /nrs/cellmap/predictions/instances.zarr/s0
    layer_type: segmentation       # default: image
    disable_meshes: true           # optional; no meshes computed when a segment is picked

A volume that cannot be opened is logged and left out. --validate-only checks that every entry has a unique name and a path, and a known layer_type.

Behind a Reverse Proxy

When the dashboard is reached through a reverse proxy that sets X-Forwarded-Host, the page loads the neuroglancer viewer from the same path on the proxy’s host, so the proxy must route /v/ on to the viewer. Inference servers are addressed as they report themselves (http://<node>:<port>) unless CELLMAP_FLOW_SERVER_URL_TEMPLATE is set in the dashboard’s environment, for example to https://proxy.example.org/inf-{port}; {url}, {host} and {port} are the reported address and its parts.

Bounding Boxes

Blockwise processing can run over regions of interest instead of the whole volume. Each box is an offset, its lower corner, and a shape, both in nm, z, y, x (not voxels):

bounding_boxes:
  - offset: [59611, 52237, 5627]
    shape: [4674, 11566, 10067]
  - offset: [64285, 26408, 15695]
    shape: [11626, 12405, 26847]

The output covers only the box, not the whole volume. A box is widened to every output voxel it touches (snapped outward onto the output voxel grid, which starts at the raw data’s corner) and cut to the raw data’s extent. The output’s offset is the box’s corner, and its OME translation the centre of the voxel there, half a voxel in. Its blocks start there and the last block along each axis is cut at the box’s end, so nothing is written past it. For example, on data whose corner is at 0, with 8 nm output voxels, a box like the first above, 4674 nm deep from 59611 nm, covers 59608 nm to 64288 nm in z: 585 voxels.

With several boxes, one output covers their bounding box, on each axis from the lowest of their corners to the highest of their far ends; only the boxes’ blocks are run, and what lies between the boxes stays 0. Set separate_bounding_boxes_zarrs: true to give each box its own output instead, the size of the box: <output_path>/box_1, box_2 and so on, in the list’s order.

Label ids a postprocessor makes unique by block (Morton, affinity segmentation) stay unique: a block’s index is counted from the raw data’s corner, so a box on the volume’s block grid gets the ids the inference server gives. The precheck refuses a box whose offset (0, 0, 0 when left out) or shape is not three numbers, or whose shape has a side not above 0.

A run resumes into an output only if it covers the same region: one written for other boxes, or for boxes by an older cellmap-flow (which wrote an array the size of the whole volume), is refused. Write to a new output path.

Output Channels

Blockwise writes each output to its own group, <output_path>/<output name>/s0. output_channels says which of the model’s channels go to which output:

Left out: one output per model channel, named by the model’s channel names (see Channel Names).
A list of channel names (or one name): one output for each, holding that model channel.
```
output_channels: [mito, er]
```
A mapping of output names to channel indices, counted from 0. An index, or a list of one, gives an output holding that channel. A list of several gives one output with those channels stacked, in the list’s order, on a leading channel axis c: its axes are c, z, y, x.
```
output_channels:
  affinities: [0, 1, 2]   # c, z, y, x: channels 0, 1 and 2
  mito: 3                 # z, y, x: channel 3
```
A mapping names the outputs itself and picks channels by index, so it also works for a model that names no channels.

Outputs are named uniquely: a list naming a channel twice is refused.

Examples

Minimal configuration

data_path: /nrs/cellmap/data/my_dataset/my_dataset.zarr/recon-1/em/fibsem-uint8
charge_group: cellmap
queue: gpu_h100

models:
  my_model:
    type: dacapo
    run_name: my_run
    iteration: 50000

Full configuration with normalizers

data_path: /nrs/cellmap/data/jrc_mus-salivary-1/jrc_mus-salivary-1.zarr/recon-1/em/fibsem-uint8
queue: gpu_h100
charge_group: cellmap

json_data:
  input_norm:
    MinMaxNormalizer:
      min_value: 0
      max_value: 250
      invert: false
    LambdaNormalizer:
      expression: "x*2-1"
  postprocess:
    DefaultPostprocessor:
      clip_min: 0
      clip_max: 1.0
      bias: 0.0
      multiplier: 127.5
    ThresholdPostprocessor:
      threshold: 127.5

models:
  model_tmp1:
    type: fly
    checkpoint: /path/to/model_checkpoint_362000
    resolution: 16
    classes:
      - mito

Blockwise processing

data_path: /nrs/cellmap/data/jrc_mus-salivary-1/jrc_mus-salivary-1.zarr/recon-1/em/fibsem-uint8
output_path: /path/to/output.zarr
task_name: cellmap_flow_mito_task
charge_group: cellmap
queue: gpu_h100
workers: 14
cpu_workers: 12
tmp_dir: /path/to/tmp

models:
  - name: model_tmp1
    type: fly
    channels:
      - mito
    checkpoint_path: /path/to/model_checkpoint_362000
    input_size: [178, 178, 178]
    input_voxel_size: [16, 16, 16]
    output_size: [56, 56, 56]
    output_voxel_size: [16, 16, 16]

bounding_boxes:
  - offset: [59611, 52237, 5627]
    shape: [4674, 11566, 10067]
  - offset: [64285, 26408, 15695]
    shape: [11626, 12405, 26847]

json_data:
  input_norm:
    MinMaxNormalizer:
      invert: false
      max_value: 250
      min_value: 0
    LambdaNormalizer:
      expression: "x*2-1"
  postprocess:
    ThresholdPostprocessor:
      threshold: 0.5

Run blockwise processing with:

cellmap_flow blockwise config.yaml
cellmap_flow blockwise config.yaml --log-level DEBUG