cellmap_flow.portable.rank

How well a model’s prediction already matches the paint: the score the Finetune tab ranks models by, to pick the one to start finetuning from.

The one implementation both modes run: the server dashboard (dashboard/routes/finetune/rank_routes.py) in its process, the browser (web/src/backend/routes/rank.ts) in chunkmirage’s Pyodide workers through portable/calls.py; numpy and the standard library only. In short:

A model is scored on the targets its own finetune trains on, over the painted voxels: an object or not on each channel of a binary model, the painted pairs of an affinity model.
The score is the geometric mean of the share of object targets called object and of background targets called background: each share is of its own class, so the paint’s mix of the two does not tilt it, and a model must get both right, as one that calls everything background scores 0 however much of the paint is background (their plain mean, the balanced accuracy, gave that model 0.5, read as half right). With one class painted, it is that class’s share. A binary model’s painted voxels next to one of the other class are not counted (rough strokes are forgiven along their edges).
It is read at the model’s own cut-off: the logit, among CUTOFFS, with the best Dice of its calls against the painted objects (the nearest to 0 among equals; 0 when only one class is painted). The score at 0 is given beside it.
A binary model answers with its best channel, by name and index (channel, channel_index): the one the Finetune tab’s Use this model makes the training form’s Channel (select_channel) for a model of several, and Fit to Paint when none is picked. An affinity model is scored on its channels together, and answers neither.
A model is read through its own input normalization (what its start sets: its declaration, or its kind’s convention) and through the session’s when that is another (norm_tries), and the better score is kept (best_try, its own among equals). Read through none, or another model’s, a model’s score is noise. Each is read on the raw channel the session’s chain picks (on_channel).
Where the raw under the paint is narrower than NARROW_WINDOW of the model’s MinMaxNormalizer’s range, the model is scored again with the normalizer stretched to the raw’s 1st to 99th percentiles (windowed), kept only when it scores WINDOW_MARGIN better.
The row of the ranking (score_row) is the score kept, with what it was read at in the units its controls take: the cut-off as Seed from Prediction’s threshold (a probability) and the raw’s range read as 0 to 1 (the stretched window, or the normalizer’s own range), and the normalizer steps it was read through (input_norm), which Use this model and Fit to Paint make the session’s, with what the other channels and normalizations matched (channel_matches, input_norm_others): why this one. Both modes answer it as it is, so their tables agree.
Beside it, the postprocessing the model’s layer needs to show a good prediction (needed_postprocess): the sigmoid its output needs and the threshold fitted to the paint, which they set too, as a model’s start sets the sigmoid alone.

The class-balanced BCE with label smoothing RANK_SMOOTHING, the finetune’s default loss, is given beside the score: about 0.69 is guessing, about 0.20 the paint fitted exactly.

Attributes

`RANK_BLOCKS`
`RANK_SMOOTHING`
`CUTOFFS`
`NARROW_WINDOW`
`WINDOW_MARGIN`
`CHANNEL_STEP`
`SIGMOID`
`THRESHOLD`

Functions

`spread`(items, count) → list	`count` of `items`, spread evenly over them in their order; all of them when there are no more.
`empty_tallies`(channels) → List[dict]	One channel's agreement with the paint per channel: the targets
`paint_edges`(labels) → numpy.ndarray	The painted voxels of `labels` (z, y, x; 0 unpainted, 1 background,
`tally`(tallies, target, mask, logits[, skip]) → None	Add one painted block to `tallies`: `target` and `mask` (C, z, y, x)
`rank_score`(tallies, channels, affinity) → Optional[dict]	The score of `tallies`: an affinity model's over its channels
`windowed`(norm, lo, hi) → Optional[List[dict]]	`norm` (normalizer specs, `{"name": ..., ...}`) with its
`same_steps`(a, b) → bool	Whether two normalizer chains are the same steps with the same
`normalizers`(steps) → List[dict]	`steps` (normalizer specs) without those that pick the raw's channel.
`on_channel`(steps, chain) → List[dict]	`steps`' normalizers read on the raw channel `chain` picks: its
`norm_tries`(own, own_from, session) → List[dict]	The input normalizations a model is scored through, each once, its own
`best_try`(scores) → Optional[int]	The index of the score (`rank_score`'s, one per `norm_tries`
`needed_postprocess`(output_class, postprocess[, threshold]) → Optional[List[dict]]	The postprocessing a model's layer needs instead of `postprocess`
`score_row`(base, stretched, norm, raw[, norm_from, others]) → Optional[dict]	A model's row of the ranking, as both modes answer it beside the

Module Contents

cellmap_flow.portable.rank.RANK_BLOCKS = 32

cellmap_flow.portable.rank.RANK_SMOOTHING = 0.1

cellmap_flow.portable.rank.CUTOFFS

cellmap_flow.portable.rank.NARROW_WINDOW = 0.25

cellmap_flow.portable.rank.WINDOW_MARGIN = 0.03

cellmap_flow.portable.rank.spread(items: Sequence, count: int) → list: count of items, spread evenly over them in their order; all of them when there are no more.

cellmap_flow.portable.rank.empty_tallies(channels: int) → List[dict]: One channel’s agreement with the paint per channel: the targets counted, their logits binned against CUTOFFS (bin b: above b of them), and the loss summed, apart for object and background targets.

cellmap_flow.portable.rank.paint_edges(labels: numpy.ndarray) → numpy.ndarray: The painted voxels of labels (z, y, x; 0 unpainted, 1 background, 2 and up objects) that touch a painted voxel of the other class along z, y or x: the strokes’ edges, which a binary model’s score leaves out.

cellmap_flow.portable.rank.tally(tallies: List[dict], target: numpy.ndarray, mask: numpy.ndarray, logits: numpy.ndarray, skip: numpy.ndarray | None = None) → None: Add one painted block to tallies: target and mask (C, z, y, x) as the finetune’s target transform makes them, logits the model’s on the same voxels (NaN where it was not read), and skip (z, y, x) the voxels not to count (paint_edges), or None.

cellmap_flow.portable.rank.rank_score(tallies: List[dict], channels: Sequence[str], affinity: bool) → dict | None: The score of tallies: an affinity model’s over its channels together (channel and channel_index None), a binary model’s best channel’s (by the score, then the lower loss), channel its name in channels (“channel <i>” past them) and channel_index its index, as select_channel takes it; with channel_matches, every channel’s {channel, channel_index, matches}, the best first, which says why that one (organelle close behind mito, say). None when nothing was counted.

cellmap_flow.portable.rank.windowed(norm: Sequence[dict], lo: float, hi: float) → List[dict] | None: norm (normalizer specs, {"name": ..., ...}) with its MinMaxNormalizer set to the raw’s window [lo, hi], or None: no MinMaxNormalizer, no window (lo = hi), or one not narrower than NARROW_WINDOW of the normalizer’s own range.

cellmap_flow.portable.rank.same_steps(a: Sequence[dict], b: Sequence[dict]) → bool: Whether two normalizer chains are the same steps with the same parameters, read as values (_value: “0” is 0).

cellmap_flow.portable.rank.CHANNEL_STEP = 'ChannelSelector'

cellmap_flow.portable.rank.normalizers(steps: Sequence[dict]) → List[dict]: steps (normalizer specs) without those that pick the raw’s channel.

cellmap_flow.portable.rank.on_channel(steps: Sequence[dict], chain: Sequence[dict]) → List[dict]: steps’ normalizers read on the raw channel chain picks: its first ChannelSelector, then them. The channel is the raw’s, picked by the user, so a model’s start, its advice and the ranking, which put a model’s normalization in the chain, keep it.

cellmap_flow.portable.rank.norm_tries(own: Sequence[dict], own_from: str, session: Sequence[dict]) → List[dict]

The input normalizations a model is scored through, each once, its own first: {"input_norm": steps, "from": where}.

own is what the model’s start sets (its declaration, own_from “declared”; or its kind’s convention, “convention”), else what the model advice assumes for a model nothing is known of (“assumed”; [] when there is nothing to assume), session the session’s input normalization (“session”). The model’s own is always tried: read through another’s, or none, a model scores as noise (fly_organelles fed raw 0 to 255 matched 51%, its best channel nucleus for paint of mitochondria). The session’s is tried too, as one the user set may fit this raw better. With neither, the raw as it is (“none”). Every one reads the channel the session’s chain picks (on_channel), which alone is no normalization of the session’s.

cellmap_flow.portable.rank.best_try(scores: Sequence[dict | None]) → int | None: The index of the score (rank_score’s, one per norm_tries entry, None where it counted nothing) that matches best, the first among equals (the model’s own); None when none counted.

cellmap_flow.portable.rank.SIGMOID = 'SigmoidPostprocessor'

cellmap_flow.portable.rank.THRESHOLD = 'ThresholdPostprocessor'

cellmap_flow.portable.rank.needed_postprocess(output_class: str | None, postprocess: Sequence[dict], threshold: float | None = None) → List[dict] | None

The postprocessing a model’s layer needs instead of postprocess (the session’s steps) to show a good prediction, or None when it shows one as it is: what a model’s start sets on a chain with none (no threshold), and what Use this model, Fit to Paint and Use these settings set beside the input normalization (the ranking row’s threshold, an affinity model’s too: the cut-off its painted pairs were scored at). output_class is the model’s output activation (portable/label_it.py, as its server’s probe answers it); with None, not known, nothing changes.

Two steps are this rule’s, on a chain of nothing else:

The activation: a sigmoid for an output with none (UNBOUNDED, logits) when the chain has none, and is empty or is given a threshold. A threshold alone on logits shows them cut at a logit (0, the model’s own boundary, as a rule); a sigmoid put ahead of it would cut the probabilities there instead, everything object at 0. An affinity model’s logits need it as much as any model’s: the chain into a segmentation (serving/probe.py suggest_affinity_chain) stays the model advice’s optional offer, which replaces the threshold and cuts the objects at it.
threshold, a probability as the ranking’s row gives it (0.5 at the model’s own boundary): a ThresholdPostprocessor at it, in place of one the chain has, in the units of the output after the activation: the probability itself after the sigmoid and on an output in [0, 1] (UNIT); 2p - 1 on one in [-1, 1] (SIGNED_UNIT, a tanh), whose probability label_it.probability reads as (x + 1) / 2.

A chain with any other step (a watershed, DefaultPostprocessor, a ChannelSelection) is the user’s and left as it is, but for one thing: a sigmoid on an output that has one already (UNIT, SIGNED_UNIT) is taken away from any chain, as a second sigmoid flattens the contrast (the model advice warns of it: serving/probe.py review_postprocess).

cellmap_flow.portable.rank.score_row(base: dict | None, stretched: dict | None, norm: Sequence[dict], raw: Sequence[float] | None, norm_from: str = 'session', others: Sequence[dict] = ()) → dict | None

A model’s row of the ranking, as both modes answer it beside the model’s name, the time taken and the notes; None when base is None (the paint gave the model nothing to score).

base is the model’s score (rank_score) read through norm, the normalizer specs it was served with (the best of norm_tries, norm_from its from); stretched its score read through windowed(norm, *raw), raw the raw’s 1st and 99th percentiles under the paint, or None when that was not tried (the raw is not narrow) or counted nothing. The stretched score is kept when it matches WINDOW_MARGIN better than base. The row is the score kept, with:

window: raw when the stretched score was kept, else None; and then matches_without_window, base’s match;
threshold: the cut-off as a probability, as Seed from Prediction’s threshold takes it, to three decimals: 0.5 at the model’s own boundary (cut-off 0), else the probability the paint is fitted best above;
input_window: the raw’s range the model read as 0 to 1, as the Processing tab’s MinMaxNormalizer takes it: window when kept, else norm’s own range; None when norm has no MinMaxNormalizer;
input_norm: the normalizer steps the score kept was read through (norm, stretched when the stretch was kept), which Use this model and Fit to Paint make the session’s; and input_norm_from, where they come from (norm_from);
input_norm_others: what the other normalizations tried matched, others ({from, matches} each), which says why this one.