Tracing

Recording what a job actually called, to catch what a static read cannot see.

A job key covers the code reachable from the experiment by name: its own bytecode, its closure, and the globals it mentions, transitively. What a key cannot cover is a dependency reached only while running — a module imported inside a function body, or a helper pulled out of a dict of handlers, or anything found by getattr. Those are invisible to a static read of the bytecode, so a change to one of them would otherwise go unnoticed, which is the one failure the orchestrator design rules out.

So a job records which owned functions it actually entered, and the hash of each one’s own code. A later pass fetches them again and re-hashes: anything renamed, removed or edited makes the stored result stale.

sys.monitoring reports the first call to each function and then, told to DISABLE, stops firing for it. The cost is therefore one callback per distinct function rather than one per call, which is what makes watching a hundred-round simulation affordable at all.

See notes/adr-2026-10-02-orchestrator-v2-a-design.md.

class krum.orchestration.tracing.DependencyTracker(owned: Modules | Iterable[str])

Bases: object

Records the owned functions entered while it is active.

property active: bool

Whether monitoring is currently installed.

called() → dict[str, str]

The owned functions that were entered, by location and code hash.

A location is hashed through the object fetched back by name, not through the code object that was seen running. The two can differ, a decorator standing between a name and the code it runs, and it is the fetched object that a later pass will re-hash: hashing it on both sides is what keeps the comparison meaningful.

Returns:

Each callee’s encoded location mapped to the hex hash of its code.

exception krum.orchestration.tracing.TracingUnavailable

Bases: RuntimeError

Raised when no monitoring tool id is free.

Python allows only a few tools to watch a process at once, so a debugger, profiler or coverage run already in place can leave none for this. Reported rather than quietly skipped: a job recorded without its callees would look verified when it is not.

krum.orchestration.tracing.verify_called(recorded: Mapping[str, str]) → list[str]

Explain why any function a job called no longer matches.

Parameters:

recorded – Encoded locations mapped to the code hashes a job recorded.

Returns:

• One reason per callee that has changed or gone, empty when they all

• still match.