cellmap_flow.jobs.local

Running a job as a process on this machine, for when there is no LSF.

Attributes

logger

Classes

LocalJob

Job running as a local subprocess.

Functions

run(→ LocalJob)

Run command locally as a subprocess (fallback when bsub unavailable).

Module Contents

cellmap_flow.jobs.local.logger

class cellmap_flow.jobs.local.LocalJob(process: subprocess.Popen, model_name: str | None = None, log_file: pathlib.Path | None = None)

Job running as a local subprocess.

The process leads its own session (see run), so kill() can take down everything under it – a bash -c wrapper, the python it runs, a tee – rather than only the direct child. Its output goes to log_file, which is where the host marker is read from.

process

log_file

kill() → None

Terminate the local process and everything it started.

get_status() → cellmap_flow.jobs.spec.JobStatus

Get current status by checking process state.

peek(max_chars: int = 4000) → str | None

The tail of the job’s log file.

wait_for_host(timeout: int = 180) → str | None

Watch the process’s log file for host information.

Parameters:

timeout – Maximum time to wait in seconds (default 180s for model loading)

Returns:

Host URL if found, None otherwise

cellmap_flow.jobs.local.run(command, name: str, log_file=None, env: Mapping[str, str] | None = None) → LocalJob

Run command locally as a subprocess (fallback when bsub unavailable).

The child writes stdout and stderr to log_file – by default a new file under the log directory (spec.default_log_dir) – rather than to pipes. Nothing reads a pipe once the host is known, so a child on pipes blocks for good as soon as it has logged a pipe buffer’s worth (about 64 KB). It also starts in its own session, so LocalJob.kill() can signal everything under it.

Parameters:

• command – Shell-free command, as a string (split with shlex) or argv list

• name – Job name for tracking

• log_file – Where the output goes; pass os.devnull when the command already writes its own log

• env – Added to this process’s environment for the child

Returns:

LocalJob object with process information