cellmap_flow.serving.probe

Infer what postprocessing a model’s raw output needs, from what its output is.

A model’s output activation is read off its network’s graph where that says it (portable/activation.py: the op that ends it), and is otherwise observed: feed it extreme inputs and see whether the result stays bounded, a guess, as a network can answer noise with one value everywhere. Either tells us what activation is already baked into the model, which is enough to catch the common misconfigurations (a SigmoidPostprocessor stacked on a model that already ends in a sigmoid, or a tanh-range model served without the rescale it needs).

Deliberately does NOT try to decide what unbounded output means. Raw logits and signed-distance predictions are both unbounded and want opposite treatment, and nothing in the tensor distinguishes them – see UNBOUNDED below.

Attributes

`logger`
`NEIGHBORHOOD_OFFSETS`
`DACAPO_FRAMEWORKS`
`CELLMAP_HF_ORG`
`CELLMAP_MODEL_DIR`
`SHIFT_TO_SIGNED`
`EXPRESSION_SAMPLES`

Functions

`looks_like_affinities`([out_channels, model_name, ...]) → bool	Heuristic: does this model emit affinities rather than a single map?
`suggest_affinity_chain`(output_class) → list	The postprocessing an affinity model needs, in order.
`suggest_postprocess_params`(chain, output_class[, ...]) → dict	Parameter values for a suggested chain, from the range flowing into each step.
`output_display_range`(postprocess_steps, output_class)	The value range a configured chain produces, for the viewer's shader.
`activation_evidence`([output_class_from, output_op, ...]) → str	What says a model's output is what it is, in words for the advice
`review_postprocess`(output_class, postprocess_names[, ...]) → dict	Compare a model's output class against the configured postprocessors.
`metadata_input_norm`(input_normalization, data_range)	The input normalization a cellmap-models export's metadata.json
`same_value`(configured, declared) → bool	Whether a configured parameter value is the declared one: numbers by
`input_norm_differences`(configured_steps, declared) → list	Where the configured input normalization departs from what a model
`suggest_input_norm`([framework, raw_dtype, declared, ...]) → dict	Propose an input_norm config for a model.

Module Contents

cellmap_flow.serving.probe.logger

cellmap_flow.serving.probe.looks_like_affinities(out_channels=None, model_name='', channels_names=None) → bool

Heuristic: does this model emit affinities rather than a single map?

Affinity models have one output channel per neighborhood offset (3 for nearest-neighbour, 9 with longer range), so a channel count of 3+ is the structural tell. Require a naming hint as well, since a multi-class semantic model also has several channels and wants entirely different handling.

cellmap_flow.serving.probe.suggest_affinity_chain(output_class: str) → list

The postprocessing an affinity model needs, in order.

AffinityPostprocessor takes affinities in [0, 1], or the uint8 0-255 that DefaultPostprocessor produces (integer input is divided by 255). The chain suggested here goes through DefaultPostprocessor, the long-standing convention; a sigmoid feeding AffinityPostprocessor directly works too.

cellmap_flow.serving.probe.NEIGHBORHOOD_OFFSETS = [[1, 0, 0], [0, 1, 0], [0, 0, 1], [3, 0, 0], [0, 3, 0], [0, 0, 3], [9, 0, 0], [0, 9, 0], [0, 0, 9]]

cellmap_flow.serving.probe.suggest_postprocess_params(chain, output_class, out_channels=None, bias=None) → dict

Parameter values for a suggested chain, from the range flowing into each step.

The default parameters assume each step sees the range its most common predecessor produces, which stops being true as soon as the chain differs. The costly case is DefaultPostprocessor after a sigmoid: its defaults clip to [-1, 1] and map that to 0-255, so probabilities in [0, 1] land in [127.5, 255] – half the range. AffinityPostprocessor then divides by 255 and subtracts its bias, and a probability of 0.0003 (a strongly repulsive edge) comes out at -0.002 instead of -0.5. The mutex watershed is left with essentially no repulsive edges and merges everything.

bias is the AffinityPostprocessor’s: the cut-off the ranking fitted to the paint for the model (its row’s threshold, a probability), or None for the midpoint, 0.5.

Returns {postprocessor_name: {param: value}}, containing only the steps whose defaults are wrong for this chain, and the AffinityPostprocessor’s bias and neighborhood always.

cellmap_flow.serving.probe.output_display_range(postprocess_steps, output_class)

The value range a configured chain produces, for the viewer’s shader.

The raw layer has to sample percentiles because its distribution is unknown. A postprocessed output does not: the last step’s arithmetic fixes the range exactly, so this is both cheaper and more accurate than sampling would be – and sampling would mean running inference purely to choose a display range.

postprocess_steps is a list of to_dict() dicts, so the real configured parameters are used rather than assumed defaults.

Returns (lo, hi), or None when the chain leaves the range undetermined – an unbounded model output with nothing applied, where logits could be any scale. Callers should leave neuroglancer’s own range controls to it there.

cellmap_flow.serving.probe.activation_evidence(output_class_from=None, output_op=None, output_range=None) → str: What says a model’s output is what it is, in words for the advice (the server’s model_info: output_class_from, output_op and the probe’s output_range): its network’s last op, which says it for certain, or its output on noise, a guess; “” when nothing is known of either.

cellmap_flow.serving.probe.review_postprocess(output_class: str | None, postprocess_names, out_channels=None, model_name='', channels_names=None, evidence='', threshold=None) → dict

Compare a model’s output class against the configured postprocessors.

Returns {"level", "message", "suggest", "params"} where level is one of “ok” / “optional” / “suggest” / “warn” / “unknown”, suggest is a list of postprocessor names to apply (empty when the current configuration is already sensible), and params their values (suggest_postprocess_params).

level == "warn" means the current chain is actively wrong, not merely unconventional. "optional" offers a chain the model’s output shows right without: an affinity model’s into a segmentation, once its affinities show as probabilities. The page shows it as a quiet line with its Apply, not a suggestion’s banner, which over the sigmoid a model’s start or the ranking set (portable/rank.py needed_postprocess, an affinity model’s logits’ too) reads as though that were wrong. A threshold is no fault either: the ranking sets one at the probability the paint is fitted best at, an affinity model’s too, so its layer shows the painted pairs’ decision the score read. Ahead of an AffinityPostprocessor it is a “warn”: the watershed then has no affinities to join objects by.

evidence (activation_evidence’s) is what says the class, which each message gives. output_class None, when neither the network nor its output on noise says what the output is, is “unknown”, with what the probe saw.

threshold is the one the ranking fitted to the paint for this model (its row’s, Session.rank_fits), or None when it was not ranked: an affinity chain suggested cuts its objects there (the watershed’s bias), where the score called a painted pair joined, and says so; at 0.5 without one.

cellmap_flow.serving.probe.DACAPO_FRAMEWORKS = ('dacapo',)

cellmap_flow.serving.probe.CELLMAP_HF_ORG = 'cellmap/'

cellmap_flow.serving.probe.CELLMAP_MODEL_DIR = '/models/cellmap/'

cellmap_flow.serving.probe.SHIFT_TO_SIGNED

cellmap_flow.serving.probe.metadata_input_norm(input_normalization, data_range)

The input normalization a cellmap-models export’s metadata.json declares (its input_normalization), as chain steps ([{"name", **params}]), for raw data spanning data_range (lo, hi); None when it declares none.

{"method": "min_max", "min": a, "max": b} says the model reads its input scaled onto [a, b]: a MinMaxNormalizer from the raw data’s range onto [0, 1], as the convention’s (_to_unit_range), then a LambdaNormalizer onto [a, b], none for [0, 1]. [-1, 1] gives SHIFT_TO_SIGNED’s x*2-1, so the declaration is the convention’s chain step for step, and a chain set from either matches the other. Another method, or a min or max that is not a number, is not guessed at: it is logged once and declares nothing, as a declaration does without the data’s range (no dataset open).

cellmap_flow.serving.probe.EXPRESSION_SAMPLES

cellmap_flow.serving.probe.same_value(configured, declared) → bool: Whether a configured parameter value is the declared one: numbers by value (the form sends “255.0” for a manifest’s 255), anything else as text, trimmed and without regard to case (a form’s “False”, Python’s False and JSON’s false are one value), and two expressions by what they give at EXPRESSION_SAMPLES, each through the whitelist LambdaNormalizer runs it through (compile_expression).

cellmap_flow.serving.probe.input_norm_differences(configured_steps, declared) → list

Where the configured input normalization departs from what a model declares, when it has the declared steps in the declared order: one {"step", "param", "configured", "declared"} per declared parameter whose value is not the configured one (same_value), in step order. [] when every value is the declared one, and when the steps themselves differ, which the step names already say. A parameter the declaration does not name (a MinMaxNormalizer’s invert) is not compared.

configured_steps are the chain’s steps as {"name", **params} dicts (to_dict() of the live normalizers); declared is the model’s declaration in suggest_input_norm’s declared form, {name: params} in order. The browser’s twin is web/src/backend/routes/models.ts inputNormDifferences.

cellmap_flow.serving.probe.suggest_input_norm(framework=None, raw_dtype='uint8', declared=None, source=None, data_range=None) → dict

Propose an input_norm config for a model.

Priority:

declared – an explicit input_norm in the model’s own metadata: an onnx model’s manifest (the browser’s exports declare theirs).
The training framework’s convention. DaCapo trains on [-1, 1], hence the extra x*2-1 on top of the 0-1 rescale.
source – where the model came from. A CellMap model follows the same [-1, 1] convention whatever its metadata calls the framework.
Nothing known about the model: assume [-1, 1] anyway, because it is what everything else in this repo trains on, and mark it low confidence so callers can present it as the assumption it is. Getting the scale wrong is not subtle – the model sees inputs far outside its training range and predicts noise – so a stated assumption beats silently feeding it raw uint8.

data_range is the actual (min, max) of the raw data. Prefer it over raw_dtype: uint8 data spans 0-255, but float data can span anything and its dtype says nothing useful.

Returns {"input_norm": {...}, "order": [...], "reason", "confidence"}.

order is redundant with the dict’s insertion order but not with what survives transport: Flask’s jsonify sorts keys, which would silently turn MinMax-then-Lambda into Lambda-then-MinMax – i.e. apply x*2-1 to raw uint8 before the rescale. Callers must use order.