Execution

How a job’s body is executed: in this process, or in one of its own.

Running each job in a fresh interpreter is a reproducibility measure before it is a performance one. Nothing a job leaves behind — a seeded global generator, an imported module’s mutable state, an initialised accelerator context — can reach the next job, so a sweep’s results do not depend on the order its jobs happened to run in.

It costs two requirements, which is why it is opted into rather than assumed:

• The experiment and its parameters must be picklable, since they are sent to the child. A function or class is pickled by name, so it has to be reachable at module level rather than defined inside another function.

• A sweep script’s top level must be guarded by if __name__ == “__main__”:. A spawned child re-imports the module it came from, and an unguarded sweep would start itself again there.

See notes/adr-2026-10-02-orchestrator-v2-a-design.md.

class krum.orchestration.execution.Executed(error: str | None = None, exception: BaseException | None = None, called: dict[str, str] | None = None)

Bases: object

The outcome of running one job’s body.

property called: dict[str, str] | None

The functions the job entered, or None if it was not traced.

property error: str | None

The traceback, if the body raised.

property exception: BaseException | None

The exception itself, for a job that ran in this process.

property ok: bool

Whether the body ran to completion.

class krum.orchestration.execution.InlineRunner(owned: Iterable[str] = (), trace: bool = False)

Bases: object

Runs each job in the orchestrator’s own process.

Parameters:

• owned – Module prefixes whose functions are worth recording.

• trace – Record which owned functions each job entered.

close() → None

Nothing to release.

run(writer: JobWriter, callable: Callable[[...], Any], params: Mapping[str, Any]) → Executed

Execute a job’s body here and now.

An interrupt is left to propagate, so that the orchestrator can discard the staged directory rather than promote a marker for a job that never finished.

class krum.orchestration.execution.SubprocessRunner(owned: Iterable[str] = (), trace: bool = False, workers: int = 1)

Bases: object

Runs each job in a fresh interpreter of its own.

The pool is spawned rather than forked, and retires each worker after one job, so no two jobs share an interpreter. Raising workers above one is all that stands between this and running a sweep in parallel; it is left at one for now, the orchestrator still being fail-fast and sequential.

close() → None

Shut the pool down, abandoning anything still queued.

executor() → ProcessPoolExecutor

The pool, started on first use.

run(writer: JobWriter, callable: Callable[[...], Any], params: Mapping[str, Any]) → Executed

Execute a job’s body in a child process and collect what it recorded.

A child that dies outright, rather than raising, leaves the pool unusable; it is discarded so that the next job starts from a fresh one, and the death is reported as that job failing.

property workers: int

How many jobs may run at once.

krum.orchestration.execution.execute_job(path: str | PathLike[str], callable: Callable[[...], Any], params: Mapping[str, Any], owned: tuple[str, ...] = (), trace: bool = False) → dict[str, Any]

Run one job’s body, recording its metrics into an already staged directory.

This is the child process entry point, so it is a module-level function that returns rather than raises: an exception’s traceback does not survive being sent back to the parent, so the traceback is formatted here, where the frames still exist.

Parameters:

• path – The staged directory to record into.

• callable – The user-defined function to execute.

• params – The parameters to execute it on.

• owned – Module prefixes whose functions are worth recording.

• trace – Record which owned functions the job entered.

Returns:

• The formatted traceback if any, the metrics that were registered, and

• the functions entered when tracing was asked for.