cellmap_flow.finetune.trainer.tensorboard

A run’s TensorBoard event files: the config card, a patch image strip, and the per-step and per-epoch scalars.

File-based, so it works from GPU nodes with no network and needs no service; one tensorboard --logdir over the training/ tree overlays every run ever made. Silently off if unavailable or not asked for.

Attributes

logger

Classes

TensorBoard

The writer, and the step and epoch it is at.

Module Contents

cellmap_flow.finetune.trainer.tensorboard.logger

class cellmap_flow.finetune.trainer.tensorboard.TensorBoard(log_dir, enabled=True, start_step=0, start_epoch=0)

The writer, and the step and epoch it is at.

Monotonic across restarts: the CLI makes a new trainer for every iteration, writing to the same tensorboard/ directory, and passes the previous one’s position on (start_step, start_epoch). Starting each at 0 drew every iteration’s curves on top of each other.

image_every = 5

dir

step = 0

epoch = 0

writer = None

property on: bool

config_card(rows): The run’s settings as a table under the “config” text tag.

images(raw, target, pred, mask, channel, output_class)

Mid-Z slice of one sample as one strip: raw (centre-cropped to the output) | target | prediction | mask, at the current epoch.

This is the picture that would have shown augmentation doing nothing for five months, and that shows raw and labels moving together once it does something. Never lets a display problem stop training.

log_step(*, loss, supervised, distillation, other_channels, bce_metrics, interval_terms, lr, step_s, data_wait_s): One optimizer step’s scalars, at the next step; distillation and other_channels (the hold on the channels not trained) None when off.

log_epoch(*, loss, supervised, best_supervised, bce_metrics, other_channels, data_wait_s, compute_s, peak_memory_gb): One epoch’s scalars, at the next epoch; supervised None when nothing was supervised, other_channels (how far the channels not trained moved) None without the hold.

flush()

close(): Close the writer; the CLI calls this when an iteration is done.