cellmap_flow.dashboard.routes.finetune.annotation_core

The models to annotate for, a new annotation volume, and the user’s settings.

Routes: GET /api/finetune/models, POST /api/finetune/create-volume, and GET and POST /api/finetune/user-prefs.

A model’s entry says, with its geometry, whether it trains one channel at a time (channel_choice, common.channel_choice): training’s submit refuses such a model without the channel the paint is for (channel_pick_refusal).

A volume is made for a model, on the grid its prediction is served on, or before any model (a “paint-first” volume, make_paint_volume) on the raw’s grid at a voxel size of the user’s, the raw level the viewer shows by default: the models answer lists the raw’s levels for New Volume to offer. A paint-first volume is not trained on (paint_first_refusal, which training’s submit asks): Which model to start from’s Use this model carries its paint onto a model’s grid (rank_routes, through carry_onto_model). So does training’s submit for a model whose grid is not the session volume’s (grid_change says whether it is, and refuses a model whose grid is not known yet).

Attributes

logger

Functions

serve_new_volume(geometry, corrections_dir, ...[, ...])	Write a new volume with geometry into corrections_dir and serve it.
make_volume(model_name, model_config, config, ...)	A new annotation volume for model_name (config its geometry,
make_paint_volume(voxel_size, corrections_dir)	A new volume painted before any model (no model_name), over the
session_raw_voxel_sizes()	The voxel sizes (nm, z, y, x) of the session's raw levels, finest
shown_voxel_size()	The voxel size of the raw level the viewer shows now
paint_first_refusal(corrections_dir)	Training's submit refused for a session whose volume
listed_geometry(name, model_config)	name's geometry as the models list has it: its server's, as it
channel_pick_refusal(model_name, model_config, ...)	Training's submit refused for a model that trains one channel at a time
volume_grid(volume) → dict	A volume record's grid (portable/volume.py GRID_FIELDS, the record's
grid_change(model_name, model_config, corrections_dir)	Whether training model_name on the session volume of
carry_onto_model(model_name, model_config, geometry, ...)	The paint of the session's volume from_id (from_volume its
get_finetune_models()	The models to finetune, and the raw's levels: ``{"models",
create_annotation_volume()
get_user_prefs()
set_user_prefs()

Module Contents

cellmap_flow.dashboard.routes.finetune.annotation_core.logger

cellmap_flow.dashboard.routes.finetune.annotation_core.serve_new_volume(geometry, corrections_dir, dataset_path, model_name, annotation_dtype='uint8')

Write a new volume with geometry into corrections_dir and serve it.

The volume records the dashboard’s current normalization and postprocessing chains, so the trainer can reproduce the inference-side normalization and the finetuned model’s YAML the postprocessing. Returns (volume_id, zarr_path, url), the URL as the browser reaches MinIO. Registering the volume is the caller’s.

cellmap_flow.dashboard.routes.finetune.annotation_core.make_volume(model_name, model_config, config, corrections_dir)

A new annotation volume for model_name (config its geometry, resolve_model_geometry’s), over the session’s dataset in corrections_dir: written, served, registered as the session’s and in its manifest. Returns (volume_id, zarr_path, minio_url, geometry). Create Volume makes one; so does Which model to start from’s Use this model (rank_routes).

cellmap_flow.dashboard.routes.finetune.annotation_core.make_paint_volume(voxel_size, corrections_dir)

A new volume painted before any model (no model_name), over the session’s dataset in corrections_dir, on the raw’s grid at voxel_size (nm, z, y, x; None: the level the viewer shows, shown_voxel_size): written, served and registered as the session’s, as make_volume’s, and returned as it returns one. It gets no training manifest, as it is not trained on (paint_first_refusal), and the session’s manifest is removed (common.write_volume_manifest), so that it, not a volume before it, is the session’s.

cellmap_flow.dashboard.routes.finetune.annotation_core.session_raw_voxel_sizes()

The voxel sizes (nm, z, y, x) of the session’s raw levels, finest first (session/volume.py raw_voxel_sizes), kept per dataset: the models answer carries them, every two seconds while the tab waits for a model. Empty without a dataset, or when its levels cannot be read (logged).

cellmap_flow.dashboard.routes.finetune.annotation_core.shown_voxel_size()

The voxel size of the raw level the viewer shows now (portable/volume.py shown_level at the viewer’s zoom, viewer/raw.py view_of), the finest without a viewer or a zoom: a paint-first volume’s when New Volume is given none. A ValueError when the raw’s levels cannot be read.

cellmap_flow.dashboard.routes.finetune.annotation_core.paint_first_refusal(corrections_dir)

Training’s submit refused for a session whose volume (session_store’s session_volume of corrections_dir) was painted before any model: (response, 400), whose error is portable/volume.py training_refusal’s sentence (the browser’s submit says the same) and whose paint_first the tab can key on. None when the volume can be trained on, or there is none (submit says so itself).

cellmap_flow.dashboard.routes.finetune.annotation_core.listed_geometry(name, model_config)

name’s geometry as the models list has it: its server’s, as it serves the model; else what the pipeline builder saved; else what the model declares (model_config its config, None for a model a job serves without one). None when none of them knows it. Never the model built here (see get_finetune_models).

cellmap_flow.dashboard.routes.finetune.annotation_core.channel_pick_refusal(model_name, model_config, output_type, select_channel)

Training’s submit refused for a model that trains one channel at a time when no channel is picked: (response, 400), with the sentence the browser’s submit says too.

Such a model is one with more than one output channel, trained on a target that is not one per channel (common.trains_one_channel, of the output_type submit resolved): the models list says channel_choice for it, and the training form then asks for the channel: without one, the job would train every channel on the paint. None when a channel is picked, or the model trains every channel at once, or the list has no geometry for it (the form shows no pick then, and the job trains as the command line does).

cellmap_flow.dashboard.routes.finetune.annotation_core.volume_grid(volume) → dict

A volume record’s grid (portable/volume.py GRID_FIELDS, the record’s own keys), or None when the record lacks one of them.

cellmap_flow.dashboard.routes.finetune.annotation_core.grid_change(model_name, model_config, corrections_dir)

Whether training model_name on the session volume of corrections_dir takes its paint carried onto the model’s grid first: (carry, refusal), with nothing changed either way.

carry is None when the model trains on the volume as it is: the volume is the model’s own, or its grid is the model’s too (portable/volume.py same_grid: a finetune of its model, any model of the same sizes), or there is no volume with a grid to compare (one a dashboard before this one registered, which trains as it did). Else it is what carry_onto_model takes after the model’s name and config, (geometry, from_id, from_volume). The model’s grid is the volume a Create Volume for it would plan now, on the session’s dataset; its sizes are its server’s or the geometry cache’s (geometry_cache.known_geometry), never the model built here (see rank_routes._served_geometry). refusal is (response, 400) when neither knows them yet: its server has not answered.

cellmap_flow.dashboard.routes.finetune.annotation_core.carry_onto_model(model_name, model_config, geometry, from_id, from_volume)

The paint of the session’s volume from_id (from_volume its record) carried onto model_name’s grid, geometry its sizes (resolve_model_geometry’s): a volume made for the model (make_volume) beside it in its corrections dir, the old one’s last strokes pulled from MinIO first, the paint carried onto the new one (session/volume.py carry_paint) and mirrored to MinIO, and its paint layer in the viewer in the old one’s place. The old volume is retired (finetune_utils retire_annotation_volume: out of the session, kept on disk in its corrections dir’s superseded/), so the session has the one volume, the new one, which its manifest (make_volume wrote it) trains on.

Which model to start from’s Use this model (rank_routes) and training’s submit for a model on another grid (grid_change) both carry through here; neither the chain nor the channel shown changes here (Use this model sets those itself, submit leaves them as the user set them). Returns {"volume_id", "volume" (its record), "layer_name", "painted", "source", "conflicts", "chunks", "superseded"}: carry_paint’s counts, and where the old volume went.

cellmap_flow.dashboard.routes.finetune.annotation_core.get_finetune_models()

The models to finetune, and the raw’s levels: {"models", "selected_model", "raw_voxel_sizes"}.

Each model is {"name", "write_shape" (nm), "output_voxel_size", "output_channels", "channels", "channel_choice", "finetune_modes"}: channels the output channels’ names (None when nothing names them), channel_choice whether training takes one of them at a time (common.channel_choice). One whose geometry nothing knows yet is {"name", "status", "finetune_modes"} instead, its status “waiting for its server” or, stopped on the Models tab, “not running”.

cellmap_flow.dashboard.routes.finetune.annotation_core.create_annotation_volume()

cellmap_flow.dashboard.routes.finetune.annotation_core.get_user_prefs()

cellmap_flow.dashboard.routes.finetune.annotation_core.set_user_prefs()