cellmap_flow.serving.probe

Infer what postprocessing a model’s raw output needs, from the output itself.

A model’s output activation is observable: feed it extreme inputs and see whether the result stays bounded. That tells us what activation is already baked into the weights, which is enough to catch the common misconfigurations (a SigmoidPostprocessor stacked on a model that already ends in a sigmoid, or a tanh-range model served without the rescale it needs).

Deliberately does NOT try to decide what unbounded output means. Raw logits and signed-distance predictions are both unbounded and want opposite treatment, and nothing in the tensor distinguishes them – see UNBOUNDED below.

Attributes

logger
UNIT
SIGNED_UNIT
UNBOUNDED
NEIGHBORHOOD_OFFSETS
DACAPO_FRAMEWORKS
CELLMAP_HF_ORG
CELLMAP_MODEL_DIR
SHIFT_TO_SIGNED

Functions

classify_output_range(→ str)	Bucket an observed (min, max) output range into an activation class.
looks_like_affinities(→ bool)	Heuristic: does this model emit affinities rather than a single map?
suggest_affinity_chain(→ list)	The postprocessing an affinity model needs, in order.
suggest_postprocess_params(→ dict)	Parameter values for a suggested chain, from the range flowing into each step.
output_display_range(postprocess_steps, output_class)	The value range a configured chain produces, for the viewer's shader.
review_postprocess(→ dict)	Compare an observed output class against the configured postprocessors.
suggest_input_norm(→ dict)	Propose an input_norm config for a model.

Module Contents

cellmap_flow.serving.probe.logger

cellmap_flow.serving.probe.UNIT = 'unit'

cellmap_flow.serving.probe.SIGNED_UNIT = 'signed_unit'

cellmap_flow.serving.probe.UNBOUNDED = 'unbounded'

cellmap_flow.serving.probe.classify_output_range(lo: float, hi: float) → str

Bucket an observed (min, max) output range into an activation class.

cellmap_flow.serving.probe.looks_like_affinities(out_channels=None, model_name='', channels_names=None) → bool

Heuristic: does this model emit affinities rather than a single map?

Affinity models have one output channel per neighborhood offset (3 for nearest-neighbour, 9 with longer range), so a channel count of 3+ is the structural tell. Require a naming hint as well, since a multi-class semantic model also has several channels and wants entirely different handling.

cellmap_flow.serving.probe.suggest_affinity_chain(output_class: str) → list

The postprocessing an affinity model needs, in order.

AffinityPostprocessor takes affinities in [0, 1], or the uint8 0-255 that DefaultPostprocessor produces (integer input is divided by 255). The chain suggested here goes through DefaultPostprocessor, the long-standing convention; a sigmoid feeding AffinityPostprocessor directly works too.

cellmap_flow.serving.probe.NEIGHBORHOOD_OFFSETS = [[1, 0, 0], [0, 1, 0], [0, 0, 1], [3, 0, 0], [0, 3, 0], [0, 0, 3], [9, 0, 0], [0, 9, 0], [0, 0, 9]]

cellmap_flow.serving.probe.suggest_postprocess_params(chain, output_class, out_channels=None) → dict

Parameter values for a suggested chain, from the range flowing into each step.

The default parameters assume each step sees the range its most common predecessor produces, which stops being true as soon as the chain differs. The costly case is DefaultPostprocessor after a sigmoid: its defaults clip to [-1, 1] and map that to 0-255, so probabilities in [0, 1] land in [127.5, 255] – half the range. AffinityPostprocessor then divides by 255 and subtracts its bias, and a probability of 0.0003 (a strongly repulsive edge) comes out at -0.002 instead of -0.5. The mutex watershed is left with essentially no repulsive edges and merges everything.

Returns {postprocessor_name: {param: value}}, containing only the steps whose defaults are wrong for this chain.

cellmap_flow.serving.probe.output_display_range(postprocess_steps, output_class)

The value range a configured chain produces, for the viewer’s shader.

The raw layer has to sample percentiles because its distribution is unknown. A postprocessed output does not: the last step’s arithmetic fixes the range exactly, so this is both cheaper and more accurate than sampling would be – and sampling would mean running inference purely to choose a display range.

postprocess_steps is a list of to_dict() dicts, so the real configured parameters are used rather than assumed defaults.

Returns (lo, hi), or None when the chain leaves the range undetermined – an unbounded model output with nothing applied, where logits could be any scale. Callers should leave neuroglancer’s own range controls to it there.

cellmap_flow.serving.probe.review_postprocess(output_class: str, postprocess_names, out_channels=None, model_name='', channels_names=None) → dict

Compare an observed output class against the configured postprocessors.

Returns {"level", "message", "suggest"} where level is one of “ok” / “suggest” / “warn”, and suggest is a list of postprocessor names to apply (empty when the current configuration is already sensible).

level == "warn" means the current chain is actively wrong, not merely unconventional.

cellmap_flow.serving.probe.DACAPO_FRAMEWORKS = ('dacapo',)

cellmap_flow.serving.probe.CELLMAP_HF_ORG = 'cellmap/'

cellmap_flow.serving.probe.CELLMAP_MODEL_DIR = '/models/cellmap/'

cellmap_flow.serving.probe.SHIFT_TO_SIGNED

cellmap_flow.serving.probe.suggest_input_norm(framework=None, raw_dtype='uint8', declared=None, source=None, data_range=None) → dict

Propose an input_norm config for a model.

Priority:

1. declared – an explicit input_norm in the model’s own metadata. Nothing writes this today, but honour it when it appears.

2. The training framework’s convention. DaCapo trains on [-1, 1], hence the extra x*2-1 on top of the 0-1 rescale.

3. source – where the model came from. A CellMap model follows the same [-1, 1] convention whatever its metadata calls the framework.

4. Nothing known about the model: assume [-1, 1] anyway, because it is what everything else in this repo trains on, and mark it low confidence so callers can present it as the assumption it is. Getting the scale wrong is not subtle – the model sees inputs far outside its training range and predicts noise – so a stated assumption beats silently feeding it raw uint8.

data_range is the actual (min, max) of the raw data. Prefer it over raw_dtype: uint8 data spans 0-255, but float data can span anything and its dtype says nothing useful.

Returns {"input_norm": {...}, "order": [...], "reason", "confidence"}.

order is redundant with the dict’s insertion order but not with what survives transport: Flask’s jsonify sorts keys, which would silently turn MinMax-then-Lambda into Lambda-then-MinMax – i.e. apply x*2-1 to raw uint8 before the rescale. Callers must use order.