cellmap_flow.cli.yaml_cli

YAML-based CLI for running multiple models. Uses YAML configuration files for batch processing.

This dynamically discovers ModelConfig subclasses just like cellmap_flow, making it easy to add new model types without modifying this file.

Attributes

logger

EXTRA_LAYER_TYPES

Functions

extra_layer_entries(config) → list

The YAML's extra_layers, checked; [] when there are none.

build_extra_layers(entries) → dict

{name: neuroglancer layer} for extra_layers entries.

run_multiple(models, dataset_path, charge_group, queue) → None

Submit multiple model inference jobs, then serve the dashboard.

main(ctx, config_path, list_types, validate_only)

Run multiple model inference jobs from a YAML configuration file.

Module Contents

cellmap_flow.cli.yaml_cli.logger
cellmap_flow.cli.yaml_cli.EXTRA_LAYER_TYPES = ('image', 'segmentation')
cellmap_flow.cli.yaml_cli.extra_layer_entries(config) → list

The YAML’s extra_layers, checked; [] when there are none.

Each entry is {name, path, layer_type?, shader?, blend?, disable_meshes?}: a volume shown beside the raw data under name.

Raises:

ConfigError – an entry lacks a name or path, repeats a name or uses the raw layer’s (“data”), or has an unknown layer_type.

cellmap_flow.cli.yaml_cli.build_extra_layers(entries) → dict

{name: neuroglancer layer} for extra_layers entries.

Read as stored, without the input normalizers. An image entry’s shader and blend are applied to its layer; a segmentation colours its ids itself, and has no blend. An entry whose volume cannot be opened is logged and left out, rather than stopping the dashboard.

cellmap_flow.cli.yaml_cli.run_multiple(models: List[cellmap_flow.models.models_config.ModelConfig], dataset_path: str, charge_group: str, queue: List[str], wrap_raw: bool = True, resample: bool = True) → None

Submit multiple model inference jobs, then serve the dashboard.

Parameters:

  • models – List of ModelConfig instances to run, each through serving.launch.launch (which applies its scale)

  • dataset_path – Base path to the dataset

  • charge_group – Billing/chargeback group

  • queue – The GPU queues each job may run on (a list)

  • resample – start each server with –resample, else –no-resample (the YAML’s resample)

cellmap_flow.cli.yaml_cli.main(ctx, config_path: str, list_types: bool, validate_only: bool)

Run multiple model inference jobs from a YAML configuration file.

The YAML file should have the following structure:

 data_path: /path/to/data charge_group: my_group # on the cluster; not needed without bsub queue: gpu_h100 # optional; one GPU queue, or a list of them

# ([gpu_h100, gpu_h200]): each job goes to the one # with the most free GPUs, and moves to the next # if it has not started in 3 minutes. Defaults to # the saved queues, else every GPU queue.

walltime: “08:00” # optional; LSF run limit, “HH:MM” or minutes.

# Without it the queue’s own default applies, # which is 2 hours on the Janelia GPU queues.

cycle_gpu_queues: true # optional; false pins each job to the first of

# queue above instead of moving it on.

wrap_raw: true # optional; false serves raw straight from the file resample: true # optional, the default: resample the data to each

# model’s input voxel size when it has no level at # it; false reads the nearest level as if it were.

extra_layers: # optional; more volumes to show beside the raw data

  • name: mito_pred path: /path/to/pred.zarr/mito shader: “…” # optional, image layers only blend: additive # optional, image layers only

  • name: instances path: /path/to/instances.zarr/s0 layer_type: segmentation # default: image disable_meshes: true # optional; no meshes computed on a pick

json_data: # optional; normalization and postprocessing
input_norm:

MinMaxNormalizer: {min_value: 0, max_value: 255} LambdaNormalizer: {expression: “x*2-1”}

postprocess:

SigmoidPostprocessor: {}

models:

  • type: dacapo name: my_model run_name: my_run iteration: 100

  • type: fly name: fly_model checkpoint: /path/to/checkpoint.ts classes: [mito, er, nucleus] resolution: [4, 4, 4]

Models may also be given as a mapping, where each key is the model name:

 models:

my_model:

type: dacapo run_name: my_run

json_data is an inline mapping (or a JSON string), not a path to a file.

Model types are automatically discovered from ModelConfig subclasses. Use –list-types to see all available types.

Examples:



cellmap_flow yaml config.yaml cellmap_flow yaml config.yaml –log-level DEBUG cellmap_flow yaml –list-types cellmap_flow yaml config.yaml –validate-only