cellmap_flow.portable.volume

Where a new annotation volume lies and what holds its labels: the planning of finetune/session/volume.py (its module docstring says what a volume is), on numbers alone, so that the server and the browser plan a volume the same way. The server reads the raw level and the model’s geometry (open_raw, resolve_model_geometry) and the browser its DatasetInfo and model manifest (web/src/finetune/volume.ts, through volume_calls.py); both hand what they found to plan_volume here.

The grid. A model reads one level of the raw, the one select_level’s “floor” picks for its input voxel size (or, with Resample, the raw resampled to that size), and both servers serve its prediction on that level’s grid: output voxel 0 starts at the level’s corner, and an output voxel is the level’s voxel times the model’s ratio of output to input voxel size. The volume lies on that grid, one chunk per model output block, so that a chunk of paint is a block of the prediction it is painted over, and it covers the level’s whole extent, a partial output voxel at the far end rounded up, then padded to whole chunks.

A volume painted before any model (plan_paint_volume, a “paint-first” volume) lies on the raw’s own grid instead, at a voxel size the user picks (shown_level gives the default: the level the viewer shows), in chunks PAINT_FIRST_CHUNK voxels a side. It is for no model (its model_name is None), so it is not trained on (training_refusal): Which model to start from ranks the models on its paint, and Use this model carries the paint onto the chosen model’s grid (carry.py), into a volume that is trained on.

Attributes

`INSTANCE_TARGETS`
`WHOLE_VOXEL_TOLERANCE`
`PAINT_FIRST_CHUNK`
`PAINT_FIRST_TRAINING`
`SHOWN_LEVEL_SLACK`
`GRID_FIELDS`

Functions

`annotation_dtype`(output_type) → str	The labels' dtype of a new volume for a model that trains on `output_type`.
`volume_corner_nm`(dataset_offset_nm, output_voxel_size) → numpy.ndarray	The world position of an annotation volume's voxel-0 lower corner, in nm.
`whole_voxels`(values, what) → list	`values` (a block size in voxels, as floats) as whole numbers.
`block_shapes`(model) → tuple	`(input_size, output_size)`, a model's blocks in voxels.
`plan_volume`(level, model) → dict	Where a new volume lies for `model` over the raw `level` it reads.
`shown_level`(voxel_sizes[, pixel_nm]) → int	The index in `voxel_sizes` (each a raw level's voxel size in nm,
`plan_paint_volume`(level, voxel_size) → dict	Where a new paint-first volume lies: over the raw `level` it is
`training_refusal`(volume)	Why `volume` (its record or root attrs: a mapping, or an object
`same_grid`(a, b) → bool	Whether grids `a` and `b` (mappings of GRID_FIELDS) are one: the
`grid_text`(grid) → str	A grid (GRID_FIELDS) as the sentences name it: its input block to its
`carried_note`(from_model, from_grid, to_model, to_grid, ...) → str	What training's submit says, in both modes, when the session volume

Module Contents

cellmap_flow.portable.volume.INSTANCE_TARGETS = ('affinities', 'flows')

cellmap_flow.portable.volume.WHOLE_VOXEL_TOLERANCE = 1e-06

cellmap_flow.portable.volume.PAINT_FIRST_CHUNK = 64

cellmap_flow.portable.volume.PAINT_FIRST_TRAINING = "This volume was painted before a model was chosen, so it is on no model's grid: under Which...

cellmap_flow.portable.volume.annotation_dtype(output_type) → str

The labels’ dtype of a new volume for a model that trains on output_type.

uint8 (ids up to 255) for a model trained on foreground and background; uint16 (up to 65535) for one trained on instances (affinities, Cellpose’s flows), whose seeds number each object: 282 of a Cellpose model’s masks in one box did not fit a uint8 volume.

cellmap_flow.portable.volume.volume_corner_nm(dataset_offset_nm, output_voxel_size) → numpy.ndarray

The world position of an annotation volume’s voxel-0 lower corner, in nm.

dataset_offset_nm (a root attr of every volume) is also written as the volume’s OME-NGFF translation, and a translation is voxel 0’s centre. Neuroglancer drew the volume that way while it was painted, so that is where the labels are: read as a corner, the value puts every label half an annotation voxel away from where it was drawn. None is an offset of 0.

cellmap_flow.portable.volume.whole_voxels(values, what: str) → list

values (a block size in voxels, as floats) as whole numbers.

A ValueError names what when one is not whole: truncating a block of 177.6 voxels to 177 would make a volume whose chunks are not the model’s blocks.

cellmap_flow.portable.volume.block_shapes(model) → tuple

(input_size, output_size), a model’s blocks in voxels.

model (a mapping, or an object with these attributes) has input_voxel_size and output_voxel_size, and either input_shape/output_shape in voxels or read_shape/ write_shape in nm, which is what a server reports.

cellmap_flow.portable.volume.plan_volume(level, model) → dict

Where a new volume lies for model over the raw level it reads.

level is that level as the model reads it, z, y, x: voxel its voxel size in nm, corner its voxel-0 lower corner in nm (not the OME translation, which is the centre), shape its voxels. On the server it is the level select_level picks for the model’s input voxel size, at its own voxel size, or the raw resampled to the model’s input voxel size when Resample is on; in the browser, the level chooseRawLevel picks. model is as block_shapes takes it.

Answers the plan’s fields of VolumeGeometry, as JSON-ready lists: input_voxel_size the level’s own, output_voxel_size the level’s times the model’s ratio of output to input voxel size (where both servers serve the prediction), the model’s own as the claimed sizes, chunk_size the model’s output block and input_size its input block, dataset_offset_nm voxel 0’s centre (the level’s corner plus half an output voxel) and dataset_shape_voxels the level’s extent in output voxels, rounded up, then padded to whole chunks.

The ratio is the model’s: snapping the output voxel size to a raw level of its own could break it (on levels of 10, 20 and 40 nm, a model reading 8 nm and writing 16 read the 10 nm level and wrote a 10 nm volume, while its server serves the prediction at 20), and the volume’s chunks were then not the model’s blocks.

cellmap_flow.portable.volume.SHOWN_LEVEL_SLACK = 1.01

cellmap_flow.portable.volume.shown_level(voxel_sizes, pixel_nm=None) → int: The index in voxel_sizes (each a raw level’s voxel size in nm, finest first) of the level the viewer shows when one screen pixel spans pixel_nm: the coarsest whose finest axis is no coarser than a pixel, and the finest when zoomed in past it, as Neuroglancer picks the level to draw (viewer/raw.py’s _view_contrast_range reads it so). The finest when the zoom is not known (None: no viewer, or none that has reported one yet). A paint-first volume’s voxel size is this level’s unless the user picks another. A ValueError when there is no level.

cellmap_flow.portable.volume.plan_paint_volume(level, voxel_size) → dict

Where a new paint-first volume lies: over the raw level it is read from, on a grid of voxel_size nm (z, y, x), for no model.

level is as plan_volume takes it: on the server the level raw_level opens for voxel_size (the level at that size, else the finest not coarser than it, at its own voxel size; with Resample on, the raw resampled to it), in the browser the one chooseRawLevel picks. The grid starts at the level’s corner: at a level’s own voxel size its voxels are the level’s (a size within float noise of it is taken as it), and at another (one the user typed) the level’s extent is covered in voxels of that size, a partial voxel at the far end rounded up. The chunks are PAINT_FIRST_CHUNK voxels a side, as thin as the volume along an axis it is thinner along, and the extent is padded to whole chunks.

Answers plan_volume’s fields: output_voxel_size the grid’s, input_voxel_size the level’s (what the raw under the paint is read at: Which model to start from’s window of it), no claimed sizes (there is no model to claim any), chunk_size, input_size a chunk in the level’s voxels, rounded up, dataset_offset_nm voxel 0’s centre and dataset_shape_voxels the padded extent. A ValueError says what is wrong with level or voxel_size.

cellmap_flow.portable.volume.training_refusal(volume): Why volume (its record or root attrs: a mapping, or an object with model_name) is not trained on, or None when it can be: PAINT_FIRST_TRAINING for a paint-first volume, which names no model, so no model’s blocks are its chunks to train on.

cellmap_flow.portable.volume.GRID_FIELDS = ('input_size', 'output_size', 'input_voxel_size', 'output_voxel_size')

cellmap_flow.portable.volume.same_grid(a, b) → bool: Whether grids a and b (mappings of GRID_FIELDS) are one: the same blocks, voxel for voxel, at the same voxel sizes (float noise of nm aside). A volume made for a model is on the grid of that model, of its finetunes and of any model of the same sizes; a model on another grid took the volume’s 178^3 blocks at 8 nm for its own 1 x 556 x 556 at 16 and torch.export refused them.

cellmap_flow.portable.volume.grid_text(grid) → str: A grid (GRID_FIELDS) as the sentences name it: its input block to its output block, “178×178×178 → 56×56×56 at 8 nm”, each at its own voxel size when the two differ (“1×556×556 at 8 nm → 1×468×468 at 16 nm”).

cellmap_flow.portable.volume.carried_note(from_model, from_grid, to_model, to_grid, kept) → str: What training’s submit says, in both modes, when the session volume was on from_model’s grid and the model to train, to_model, is on another: the paint carried onto that model’s grid, as Use this model carries it, both grids named (grid_text), and where the old volume is kept (kept, the mode’s words). The input normalization, the postprocessing and the channel stay what the user set: only the volume changed.