httk.workflow.protocol

The language-neutral filesystem protocol surface of httk₂ workflows.

This module is the one deliberate public home of the on-disk protocol: the shapes, validators, and primitives an implementation in any language reads and writes to interoperate through a workspace. The normative specification is the filesystem protocol reference in the httk-workflow documentation; everything named here is what that document describes, and an independent inspection or verification tool should be able to work from this namespace and that document alone.

Nothing here is manager bookkeeping, a subprocess wrapper, a CLI handler, or a scheduling pass — those live in their own modules and are not part of the protocol. The implementations are owned by the modules re-exported below (models, journal, transactions, and the runtime builders), which are internal detail from the protocol’s point of view; import the names from here.

Attributes

CARRIED_STATE_MEMBERS
CORE_PROFILE
CORE_STATE_KINDS
QUIESCENT_KINDS
RUNNER_SOURCES
STATE_KINDS
SUPPORTED_EXTENSIONS
TERMINAL_KINDS
JoinCondition
OutcomeAction

Exceptions

FormatError	A workspace, job, journal frame, outcome, or request is malformed.
RunnerResolutionError	A shared runner cannot be resolved, staged, or verified.
TransactionError	A transactional-data manifest cannot be safely replayed.
TransitionLostError	Another actor committed a transition from the expected marker.
UnsupportedExtensionError	A workspace requires an extension this implementation does not support.
WorkflowError	Base class for workflow protocol failures.
WorkspaceCorruptionError	The authoritative filesystem state is internally inconsistent.
WorkspaceUnavailableError	The workspace cannot currently provide a coherent protocol view.

Classes

JournalFrame	Describe one intact frame found by walking a segment from its header.
RecordVerification	Report the outcome of reading one referenced frame without raising.
Failure	One canonical structured failure record.
JobDefinition	The immutable declaration and execution settings of one job.
Marker	The state marker locating one job transition in a workspace.
RetentionPolicy	How long a workspace keeps the history it is allowed to collect.
RetryPolicy	The attempt budgets of one job and the failures it retries within them.
StateFrame	The members of one state frame, typed for the manager that uses them.
WorkspacePolicy	The tunables every implementation attaching to one workspace shares.
AttemptContext	Describe the immutable identity and restart evidence for one running attempt.
ChildReference	Identify one child in a native join.
JobSpec	Values needed to create an immutable native job definition.
OutcomeDraft	One unpublished outcome bundle below an attempt control directory.
ReplayableWorkdirBatch	Build a sealed, idempotently replayable set of workdir changes.
RunLog	Append structured application evidence in a workdir.
TransactionBuilder	Build a validated replayable transaction manifest.
MarkerFault	Describe one state entry shaped like a marker that cannot be interpreted.

Functions

encode_record_ref(writer_id, segment, offset, length, ...)	Encode one canonical hwref-v1 reference.
iter_journal_frames(control_dir)	Yield every intact frame of every segment of every writer.
iter_segment_frames(path, writer_id, segment)	Yield every intact frame of one segment.
parse_record_ref(record_ref)	Parse one canonical hwref-v1 reference.
read_record(control_dir, record_ref, *[, deadline_seconds])	Read and verify a journal record, retrying visibility-short reads.
segment_path(control_dir, writer_id, segment)	Return the segment file one record reference names.
verify_record(control_dir, record_ref, *[, ...])	Read one referenced frame, reporting damage rather than raising.
canonical_uuid(value[, name])	Validate and return a canonical lowercase UUID.
is_payload_private(name)	Report whether one payload entry name is runner-private scratch.
job_digest(data)	Return the normative immutable job digest of stored job.json bytes.
make_job_key(job_id, tag)	Compose the stable job key from an identifier and optional tag.
marker_basename(job_key, priority, generation, record_ref)	Build one bounded state-marker basename.
normalize_placement(value)	Validate and normalize one relative POSIX placement.
parse_job_key(value)	Split a job key into its optional tag and job identifier.
parse_package_runner(value)	Split the reserved pkg:<module>/<resource> installed runner form.
to_base36(value)	Encode a nonnegative integer in lowercase base 36.
validate_attempt_control(value[, name])	Validate one attempt-control directory name read from a state frame.
validate_declaration_name(value[, name])	Validate one declaration name of a job.
validate_declarations(value[, name])	Validate the optional declarations object of a job.
validate_failure(value[, name])	Validate one published failure object.
validate_label(value, name)	Validate and return one protocol label.
validate_parameters(value[, name])	Validate the optional application-defined parameters object of a job.
validate_runner_path(value, source)	Validate runner.path against the root implied by runner.source.
validate_sha256(value, name)	Validate one lowercase hexadecimal SHA-256 digest string.
validate_step(value[, name])	Validate and return one workflow step name.
join_mapping(children[, condition, count, ...])	Return the validated join member of one waiting outcome.
prepare_job_payload(destination, spec, *[, parent, ...])	Create and validate job.json in an existing prepared payload.
replay_transaction(transaction_dir, data_dir, *, ...)	Idempotently apply one published transaction.

Module Contents

exception httk.workflow.protocol.FormatError

Bases: WorkflowError, ValueError

A workspace, job, journal frame, outcome, or request is malformed.

exception httk.workflow.protocol.RunnerResolutionError(code, message)

Bases: WorkflowError

A shared runner cannot be resolved, staged, or verified.

The failure carries the exact protocol failure code the manager records, so an unresolvable runner (runner_unavailable), a runner whose staged bytes disagree with the digest the job pinned (runner_mismatch), a missing registration (runner_not_built), and a failed foreground build (runner_build_failed) stay distinguishable to an operator.

Parameters:

• code (str) – Protocol failure code recorded by the manager.

• message (str) – Human-readable failure description.

code

exception httk.workflow.protocol.TransactionError

Bases: WorkflowError

A transactional-data manifest cannot be safely replayed.

exception httk.workflow.protocol.TransitionLostError

Bases: WorkflowError

Another actor committed a transition from the expected marker.

exception httk.workflow.protocol.UnsupportedExtensionError

Bases: WorkflowError

A workspace requires an extension this implementation does not support.

exception httk.workflow.protocol.WorkflowError

Bases: Exception

Base class for workflow protocol failures.

exception httk.workflow.protocol.WorkspaceCorruptionError

Bases: WorkflowError

The authoritative filesystem state is internally inconsistent.

exception httk.workflow.protocol.WorkspaceUnavailableError

Bases: WorkflowError

The workspace cannot currently provide a coherent protocol view.

class httk.workflow.protocol.JournalFrame

Describe one intact frame found by walking a segment from its header.

Parameters:

• record_ref – Identify the canonical record reference.

• writer_id – Identify the journal writer.

• segment – Identify the journal segment.

• offset – Locate the frame within the segment.

• frame – Hold the decoded journal record.

record_ref: str

writer_id: str

segment: int

offset: int

frame: dict[str, Any]

class httk.workflow.protocol.RecordVerification

Report the outcome of reading one referenced frame without raising.

Parameters:

• record_ref – Identify the record that was checked.

• frame – Hold the verified frame when reading succeeded.

• problem – Name the verification problem when reading failed.

• detail – Explain the verification result.

record_ref: str

frame: dict[str, Any] | None

problem: str | None

detail: str

property ok: bool

Report whether the referenced frame was read and verified.

Returns:

True when the frame is present and valid.

Return type:

bool

httk.workflow.protocol.encode_record_ref(writer_id, segment, offset, length, checksum)

Encode one canonical hwref-v1 reference.

Parameters:

• writer_id (str) – Identify the journal writer.

• segment (int) – Identify the journal segment.

• offset (int) – Locate the frame within the segment.

• length (int) – Record the frame payload length.

• checksum (bytes) – Supply the frame checksum.

Returns:

The canonical record reference.

Return type:

str

httk.workflow.protocol.iter_journal_frames(control_dir)

Yield every intact frame of every segment of every writer.

Parameters:

control_dir (pathlib.Path) – Locate the workspace control directory.

Yield:

Each intact journal frame.

httk.workflow.protocol.iter_segment_frames(path, writer_id, segment)

Yield every intact frame of one segment.

The walk is deliberately forgiving. A damaged frame whose framing is still intact is skipped, because the frames behind it remain locatable and are exactly what a repair is looking for; a torn or partially visible tail is the normal state of a segment a live writer is appending to and simply ends the walk.

Parameters:

• path (pathlib.Path) – Locate the journal segment.

• writer_id (str) – Identify the journal writer.

• segment (int) – Identify the journal segment number.

Yield:

Each intact frame found in the segment.

httk.workflow.protocol.parse_record_ref(record_ref)

Parse one canonical hwref-v1 reference.

Parameters:

record_ref (str) – Supply the record reference to parse.

Returns:

The writer, segment, offset, length, and checksum components.

Raises:

httk.workflow.errors.FormatError – If the reference is not canonical.

Return type:

tuple[str, int, int, int, str]

httk.workflow.protocol.read_record(control_dir, record_ref, *, deadline_seconds=None)

Read and verify a journal record, retrying visibility-short reads.

A frame that is absent, short, or undecodable may be an extension of a segment that has not reached this client yet, so it is retried with bounded backoff until deadline_seconds — the workspace’s configured visibility deadline — expires. Damage that no amount of waiting can repair is reported at once.

Parameters:

• control_dir (pathlib.Path) – Locate the workspace control directory.

• record_ref (str) – Identify the journal record to read.

• deadline_seconds (float | None) – Bound retries for metadata visibility.

Returns:

The verified journal record.

Raises:

• httk.workflow.errors.FormatError – If the record reference is invalid.

• httk.workflow.errors.WorkspaceCorruptionError – If the record is permanently damaged.

• httk.workflow.errors.WorkspaceUnavailableError – If the record remains incoherently visible.

Return type:

dict[str, Any]

httk.workflow.protocol.segment_path(control_dir, writer_id, segment)

Return the segment file one record reference names.

Parameters:

• control_dir (pathlib.Path) – Locate the workspace control directory.

• writer_id (str) – Identify the journal writer.

• segment (int) – Identify the journal segment.

Returns:

The segment file path.

Return type:

pathlib.Path

httk.workflow.protocol.verify_record(control_dir, record_ref, *, deadline_seconds=None)

Read one referenced frame, reporting damage rather than raising.

This is the reading half of a workspace check: it distinguishes a segment that is gone from one that is truncated, corrupt, or simply not holding the frame the reference names, which is what a repair decision needs.

Parameters:

• control_dir (pathlib.Path) – Locate the workspace control directory.

• record_ref (str) – Identify the journal record to verify.

• deadline_seconds (float | None) – Bound retries for metadata visibility.

Returns:

The verification result.

Return type:

RecordVerification

httk.workflow.protocol.CARRIED_STATE_MEMBERS = ('step', 'activation_id', 'activation_ordinal', 'attempt_id', 'attempt_ordinal',...

httk.workflow.protocol.CORE_PROFILE = 'core-v2'

httk.workflow.protocol.CORE_STATE_KINDS

httk.workflow.protocol.QUIESCENT_KINDS

httk.workflow.protocol.RUNNER_SOURCES

httk.workflow.protocol.STATE_KINDS = ('submitted', 'ready', 'claimed', 'running', 'committing', 'cancelling', 'relocating',...

httk.workflow.protocol.SUPPORTED_EXTENSIONS: frozenset[str]

httk.workflow.protocol.TERMINAL_KINDS

class httk.workflow.protocol.Failure

One canonical structured failure record.

Every failure published by a runner, a bridge, or the manager itself uses exactly this shape: a stable machine code, one human message, optional structured details, and the advisory retryable flag. Retry policy uses retry_on for manager-detected failures, while a runner-declared retryable failure is retry-eligible regardless of retry_on when budget remains.

Parameters:

• code – The stable machine failure code.

• message – The human-readable failure message.

• details – Optional structured failure details.

• retryable – Whether repeating the attempt could help.

code: str

message: str

details: collections.abc.Mapping[str, object] | None = None

retryable: bool = False

as_mapping()

Return the canonical JSON representation of this failure.

Returns:

The JSON failure mapping.

Return type:

dict[str, object]

class httk.workflow.protocol.JobDefinition

The immutable declaration and execution settings of one job.

The mapping carried in parameters contains opaque implementation knobs; declared staged objects belong to the SDK’s input declarations.

id: str

tag: str | None

name: str

workflow: str

runner_executor: str

runner_source: str

runner_path: pathlib.PurePosixPath

runner_sha256: str | None

runner_arguments: tuple[str, Ellipsis]

workdir_mode: str

workdir_path: pathlib.PurePosixPath

data_mode: str

initial_step: str

priority: int

claim_pool: str

required_capabilities: frozenset[str]

retry_policy: RetryPolicy

resources: collections.abc.Mapping[str, object]

parameters: collections.abc.Mapping[str, object]

environment: collections.abc.Mapping[str, object]

declarations: collections.abc.Mapping[str, collections.abc.Mapping[str, object]]

declared: collections.abc.Mapping[str, collections.abc.Mapping[str, collections.abc.Mapping[str, object]]]

parent: collections.abc.Mapping[str, object] | None

raw: collections.abc.Mapping[str, object]

stored_digest: str | None = None

property job_key: str

Return the stable key of this job.

property digest: str

Return the immutable job digest.

Normatively the digest is job_digest() over the stored job.json file bytes exactly as submitted, which is what every definition read through from_bytes() carries. A definition composed in memory has no stored bytes yet, so its canonical serialization is hashed instead; the two agree as soon as that serialization is what gets written.

classmethod from_bytes(data, *, name='job.json')

Parse stored job.json bytes, pinning the normative job digest.

Parameters:

• data (bytes) – The stored job document bytes.

• name (str) – The document name used in validation errors.

Returns:

The parsed job definition.

Raises:

httk.workflow.errors.FormatError – If the bytes do not contain a valid job document.

Return type:

JobDefinition

classmethod from_path(path)

Read one stored job.json, pinning the normative job digest.

Parameters:

path (pathlib.Path) – The path of the stored job document.

Returns:

The parsed job definition.

Raises:

httk.workflow.errors.FormatError – If the file cannot be read or is invalid.

Return type:

JobDefinition

classmethod from_mapping(value)

Parse one job definition mapping.

Parameters:

value (collections.abc.Mapping[str, object]) – The job document mapping.

Returns:

The parsed job definition.

Raises:

httk.workflow.errors.FormatError – If the mapping does not satisfy the job protocol.

Return type:

JobDefinition

class httk.workflow.protocol.Marker

The state marker locating one job transition in a workspace.

Parameters:

• kind – The state kind encoded by the marker.

• placement – The workspace placement of the job.

• job_key – The stable job key.

• priority – The marker priority.

• generation – The state generation.

• record_ref – The transition record reference.

• path – The marker path.

kind: str

placement: pathlib.PurePosixPath

job_key: str

priority: int

generation: int

record_ref: str

path: pathlib.Path

property job_id: str

Return the job identifier encoded in this marker.

classmethod from_path(state_root, path)

Parse one marker path below a workspace state root.

Parameters:

• state_root (pathlib.Path) – The workspace state directory.

• path (pathlib.Path) – The marker path to parse.

Returns:

The parsed state marker.

Raises:

httk.workflow.errors.FormatError – If the path does not use marker syntax.

Return type:

Marker

class httk.workflow.protocol.RetentionPolicy

How long a workspace keeps the history it is allowed to collect.

Every member is optional and means “no configured limit” when absent. The collector that acts on these numbers is a separate concern; the workspace only carries them so that every implementation attaching to it agrees on what may be removed and when.

Parameters:

• attempt_control_days – The retention period for attempt controls.

• journal_days – The retention period for journal history.

• trash_days – The retention period for discarded workspace entries.

attempt_control_days: float | None = None

journal_days: float | None = None

trash_days: float | None = None

classmethod from_mapping(value, name='policy.retention')

Validate one retention policy mapping.

Parameters:

• value (object) – The policy mapping to validate.

• name (str) – The field name used in validation errors.

Returns:

The validated retention policy.

Raises:

httk.workflow.errors.FormatError – If the mapping contains unsupported or invalid members.

Return type:

RetentionPolicy

as_mapping()

Return the JSON representation, omitting unconfigured limits.

Returns:

The JSON policy mapping.

Return type:

dict[str, object]

class httk.workflow.protocol.RetryPolicy

The attempt budgets of one job and the failures it retries within them.

Two independent rules make a failure retry-eligible, and both are bounded by exactly the same budgets:

• retry_on lists failure codes. A manager-detected failure — a lost lease, a process failure, an unusable outcome — is retried when its code appears in this set.

• A runner-declared failure published with retryable: true is retried whether or not its code appears in retry_on, because the runner that produced the failure is the authority on whether repeating the attempt can help.

Neither rule can exceed maximum_attempts_per_activation or maximum_total_attempts: an exhausted budget always ends the job.

Parameters:

• maximum_attempts_per_activation – The per-activation attempt budget.

• maximum_total_attempts – The total attempt budget.

• maximum_activations – The activation budget.

• retry_on – Failure codes eligible for manager-detected retry.

maximum_attempts_per_activation: int | None

maximum_total_attempts: int | None

maximum_activations: int | None

retry_on: frozenset[str]

classmethod from_mapping(value)

Validate one job retry policy mapping.

Parameters:

value (object) – The retry policy mapping to validate.

Returns:

The validated retry policy.

Raises:

httk.workflow.errors.FormatError – If the mapping contains invalid retry settings.

Return type:

RetryPolicy

class httk.workflow.protocol.StateFrame

The members of one state frame, typed for the manager that uses them.

The frame is held exactly as it is on disk, so every member round-trips verbatim — including one written by a newer implementation, by an enabled extension, or by a workspace older than this code. What this implementation reads and writes goes through the typed accessors and through of(), so a mistyped member name is a type error at the call site rather than a silently defaulted value at runtime.

The envelope members format, workspace_id, job_id, kind, state_generation, and their siblings belong to the transition that publishes a frame and are supplied by the workspace, never here.

Parameters:

members – The state members carried by this frame.

members: collections.abc.Mapping[str, Any]

classmethod from_mapping(value, name='state frame')

Read one stored state frame, keeping every member verbatim.

Parameters:

• value (object) – The state-frame mapping to read.

• name (str) – The field name used in validation errors.

Returns:

The state frame.

Raises:

httk.workflow.errors.FormatError – If the value is not a mapping.

Return type:

StateFrame

as_mapping()

Return the JSON representation, member for member.

Returns:

The state-frame member mapping.

Return type:

dict[str, object]

has(name)

Report whether the frame carries name at all, null included.

Parameters:

name (str) – The member name to check.

Returns:

Whether the member is present.

Return type:

bool

classmethod of(base=None, *, step=_UNSET, activation_id=_UNSET, activation_ordinal=_UNSET, attempt_id=_UNSET, attempt_ordinal=_UNSET, total_attempts=_UNSET, data_generation=_UNSET, join_summary=_UNSET, runner_steps=_UNSET, manager_id=_UNSET, writer_id=_UNSET, claim_id=_UNSET, attempt_control=_UNSET, lease_seconds=_UNSET, matched_pool=_UNSET, matched_capabilities=_UNSET, started_at=_UNSET, workdir=_UNSET, outcome_action=_UNSET, child_digests=_UNSET, child_labels=_UNSET, next_step=_UNSET, join=_UNSET, pause=_UNSET, failure=_UNSET, job_digest=_UNSET, join_unresolved=_UNSET, unclean_restart=_UNSET, unsafe_persistent_takeover=_UNSET, takeover_evidence=_UNSET, cancellation=_UNSET, previous_attempt_id=_UNSET, operator=_UNSET, operator_key=_UNSET, operator_reason=_UNSET, request_id=_UNSET, revival_hazard=_UNSET, reason=_UNSET)

Return base with the named members set, absent ones untouched.

Every member a manager writes is one declared keyword, so the complete vocabulary of a state frame is visible in one signature and no call site can invent a member by misspelling one. Passing None writes the JSON null the protocol distinguishes from an absent member.

Parameters:

• base (StateFrame | None) – The frame to update, or an empty frame when omitted.

• step (str) – The activation step.

• activation_id (str) – The activation identifier.

• activation_ordinal (int) – The activation ordinal.

• attempt_id (str) – The attempt identifier.

• attempt_ordinal (int) – The attempt ordinal.

• total_attempts (int) – The total attempt count.

• data_generation (int | None) – The transactional data generation.

• join_summary (collections.abc.Sequence[object] | None) – The children observed by the activation.

• runner_steps (collections.abc.Sequence[str]) – The runner’s registered step names.

• manager_id (str) – The owning manager identifier.

• writer_id (str) – The writer identifier.

• claim_id (str) – The claim identifier.

• attempt_control (str) – The attempt-control directory name.

• lease_seconds (float) – The claim lease duration.

• matched_pool (str) – The pool selected for the claim.

• matched_capabilities (collections.abc.Sequence[str]) – The capabilities matched by the claim.

• started_at (str) – The attempt start timestamp.

• workdir (str) – The attempt workdir.

• outcome_action (str) – The published outcome action.

• child_digests (collections.abc.Mapping[str, str]) – The child payload digests.

• child_labels (collections.abc.Mapping[str, str]) – The child labels.

• next_step (str) – The next activation step.

• join (collections.abc.Mapping[str, object]) – The child join condition.

• pause (object) – The pause record.

• failure (collections.abc.Mapping[str, object]) – The failure record.

• job_digest (str) – The immutable job digest.

• join_unresolved (collections.abc.Mapping[str, object]) – The persisted first-unresolved child and timestamp of a waiting join.

• unclean_restart (bool) – Whether the previous attempt ended uncleanly.

• unsafe_persistent_takeover (bool) – Whether persistent takeover was unsafe.

• takeover_evidence (collections.abc.Mapping[str, object]) – Evidence for the persistent takeover.

• cancellation (collections.abc.Mapping[str, object]) – The cancellation record.

• previous_attempt_id (str | None) – The previous attempt identifier.

• operator (object) – The operator identity.

• operator_key (object) – The operator key.

• operator_reason (object) – The operator reason.

• request_id (object) – The request identifier.

• revival_hazard (collections.abc.Mapping[str, object]) – Evidence of a revival hazard.

• reason (str) – The transition reason.

Returns:

The updated state frame.

Return type:

StateFrame

carried()

Return only the members every transition of this activation repeats.

Returns:

The carried state frame.

Return type:

StateFrame

select(names)

Return only the named members this frame actually carries.

Parameters:

names (collections.abc.Sequence[str]) – The member names to retain.

Returns:

A frame containing the selected members.

Return type:

StateFrame

property step: str | None

Return the activation step, when present.

property activation_id: str | None

Return the activation identifier, when present.

property activation_ordinal: int | None

Return the activation ordinal, when present.

property attempt_id: str | None

Return the attempt identifier, when present.

property attempt_ordinal: int | None

Return the attempt ordinal, when present.

property total_attempts: int | None

Return the total attempt count, when present.

property data_generation: int | None

Return the transactional data generation, when present.

property join_summary: object

Return the observed child summary, when present.

property join_unresolved: collections.abc.Mapping[str, object] | None

Return the persisted first-unresolved join child and timestamp.

A waiting frame records this once, the first time a manager finds a join child unresolvable, so the grace before the join fails is measured from that instant and survives a manager restart rather than resetting.

property manager_id: str | None

Return the owning manager, refusing anything that is not one.

The value is joined below managers/ to reach a heartbeat, so a frame that does not name a canonical manager UUID is a protocol violation rather than a path to try.

property attempt_control: str | None

Return the validated attempt-control component of this frame.

property lease_seconds: float | None

Return the claim lease duration, when present.

property started_at: str | None

Return the attempt start timestamp, when present.

property workdir: str | None

Return the attempt workdir, when present.

property next_step: str | None

Return the next activation step, when present.

property join: collections.abc.Mapping[str, object] | None

Return the child join condition, when present.

property failure: collections.abc.Mapping[str, object] | None

Return the failure record, when present.

property pause: object

Return the pause record, when present.

property cancellation: collections.abc.Mapping[str, object] | None

Return the cancellation record, when present.

property child_digests: collections.abc.Mapping[str, object] | None

Return the child digests, when present.

property previous_attempt_id: str | None

Return the previous attempt identifier, when present.

property unclean_restart: bool

Report whether the previous attempt ended uncleanly.

property unsafe_persistent_takeover: bool

Report whether persistent takeover was unsafe.

property reason: str | None

Return the transition reason, when present.

class httk.workflow.protocol.WorkspacePolicy

The tunables every implementation attaching to one workspace shares.

These are workspace properties rather than per-process options: two managers on different hosts must agree on how long a marker may take to become visible and on how long an unheartbeaten lease means anything. They live in format.json beside the format and profile declarations, and a workspace written before this section existed simply reads as the defaults.

Parameters:

• visibility_deadline_seconds – The marker visibility deadline.

• lease_seconds – The manager claim lease duration.

• journal_segment_bytes – The journal segment size.

• retention – The workspace retention policy.

visibility_deadline_seconds: float = 5.0

lease_seconds: float = 900.0

journal_segment_bytes: int = 67108864

retention: RetentionPolicy

classmethod from_mapping(value, name='policy')

Validate one complete policy object, filling in absent members.

Parameters:

• value (object) – The policy mapping to validate.

• name (str) – The field name used in validation errors.

Returns:

The validated retention policy.

Raises:

httk.workflow.errors.FormatError – If the mapping contains unsupported or invalid members.

Return type:

WorkspacePolicy

as_mapping()

Return the complete JSON representation stored in format.json.

Returns:

The JSON workspace policy mapping.

Return type:

dict[str, object]

updated(changes, name='policy')

Return this policy with changes applied and revalidated.

Parameters:

• changes (collections.abc.Mapping[str, object]) – Policy members to replace.

• name (str) – The field name used in validation errors.

Returns:

The updated workspace policy.

Raises:

httk.workflow.errors.FormatError – If the changes contain unsupported or invalid members.

Return type:

WorkspacePolicy

httk.workflow.protocol.canonical_uuid(value, name='id')

Validate and return a canonical lowercase UUID.

Parameters:

• value (object) – The value to validate.

• name (str) – The field name used in validation errors.

Returns:

The canonical UUID text.

Raises:

httk.workflow.errors.FormatError – If the value is not canonical UUID text.

Return type:

str

httk.workflow.protocol.is_payload_private(name)

Report whether one payload entry name is runner-private scratch.

A runner-private entry is excluded from every payload digest, so publishing an outcome or writing job state can never change the digest of a payload that a manager, a transfer, or a registration check must still recognize.

Parameters:

name (str) – The payload entry name to classify.

Returns:

Whether the name is runner-private.

Return type:

bool

httk.workflow.protocol.job_digest(data)

Return the normative immutable job digest of stored job.json bytes.

The digest of a job is the SHA-256 over the job.json file bytes exactly as submitted. Nothing rewrites or renormalizes those bytes, so the digest is reproducible by any implementation with only a hash utility.

Parameters:

data (bytes) – The stored job.json bytes.

Returns:

The lowercase SHA-256 digest.

Return type:

str

httk.workflow.protocol.make_job_key(job_id, tag)

Compose the stable job key from an identifier and optional tag.

Parameters:

• job_id (str) – The job UUID text.

• tag (str | None) – The optional job tag.

Returns:

The job key used by workspace markers.

Return type:

str

httk.workflow.protocol.marker_basename(job_key, priority, generation, record_ref)

Build one bounded state-marker basename.

Parameters:

• job_key (str) – The stable job key.

• priority (int) – The marker priority.

• generation (int) – The state generation.

• record_ref (str) – The transition record reference.

Returns:

The marker basename.

Raises:

httk.workflow.errors.FormatError – If a component is invalid or the basename exceeds the profile limit.

Return type:

str

httk.workflow.protocol.normalize_placement(value)

Validate and normalize one relative POSIX placement.

Parameters:

value (str | pathlib.PurePosixPath) – The placement to validate.

Returns:

The normalized relative placement.

Raises:

httk.workflow.errors.FormatError – If the placement is absolute, empty, unsafe, or too long.

Return type:

pathlib.PurePosixPath

httk.workflow.protocol.parse_job_key(value)

Split a job key into its optional tag and job identifier.

Parameters:

value (str) – The job key to parse.

Returns:

The tag and job identifier.

Raises:

httk.workflow.errors.FormatError – If the key does not use the protocol syntax.

Return type:

tuple[str | None, str]

httk.workflow.protocol.parse_package_runner(value)

Split the reserved pkg:<module>/<resource> installed runner form.

Return None when value is an ordinary relative runner path, so callers can treat the reserved form as one alternative spelling of runner.path rather than as a separate protocol member.

Parameters:

value (str) – The runner path to inspect.

Returns:

The package module and resource, or None for an ordinary path.

Raises:

httk.workflow.errors.FormatError – If the reserved package form is malformed.

Return type:

tuple[str, pathlib.PurePosixPath] | None

httk.workflow.protocol.to_base36(value)

Encode a nonnegative integer in lowercase base 36.

Parameters:

value (int) – The integer to encode.

Returns:

The base-36 representation.

Raises:

ValueError – If the value is negative.

Return type:

str

httk.workflow.protocol.validate_attempt_control(value, name='attempt_control')

Validate one attempt-control directory name read from a state frame.

The name is joined below a job payload to reach the control directory of an attempt, so it is exactly one relative component of the canonical .httk-attempt.<attempt-id> shape. Validating it before it is joined is what keeps a hostile or damaged frame from naming a path outside the job.

Parameters:

• value (object) – The attempt-control name to validate.

• name (str) – The field name used in validation errors.

Returns:

The validated attempt-control name.

Raises:

httk.workflow.errors.FormatError – If the value is not a canonical attempt-control name.

Return type:

str

httk.workflow.protocol.validate_declaration_name(value, name='declaration name')

Validate one declaration name of a job.

The name keys the declarations object of job.json and is also the basename of the runtime-refined document below .httk-job/declarations/, so it must be a safe single path component and nothing else.

Parameters:

• value (object) – The declaration name to validate.

• name (str) – The field name used in validation errors.

Returns:

The validated declaration name.

Raises:

httk.workflow.errors.FormatError – If the value is not a safe declaration name.

Return type:

str

httk.workflow.protocol.validate_declarations(value, name='declarations')

Validate the optional declarations object of a job.

Each member is one workflow-declaration document carried verbatim: the protocol checks that a declaration is a JSON object and never looks inside it, because what the members mean is owned by the vocabulary the document names itself — the OPTIMADE workflow-declaration work is standardizing exactly that, and an engine that reinterpreted it would only be able to disagree with it. The bytes live in job.json and are therefore covered by the immutable job digest like every other member.

Parameters:

• value (object) – The declarations mapping to validate.

• name (str) – The field name used in validation errors.

Returns:

The validated declaration documents keyed by name.

Raises:

httk.workflow.errors.FormatError – If a declaration name, document, or size limit is invalid.

Return type:

dict[str, dict[str, object]]

httk.workflow.protocol.validate_failure(value, name='failure')

Validate one published failure object.

Parameters:

• value (object) – The failure mapping to validate.

• name (str) – The field name used in validation errors.

Returns:

The validated failure record.

Raises:

httk.workflow.errors.FormatError – If the failure is missing required members or has invalid members.

Return type:

Failure

httk.workflow.protocol.validate_label(value, name)

Validate and return one protocol label.

Parameters:

• value (object) – The label to validate.

• name (str) – The field name used in validation errors.

Returns:

The validated label.

Raises:

httk.workflow.errors.FormatError – If the value is not valid label text.

Return type:

str

httk.workflow.protocol.validate_parameters(value, name='parameters')

Validate the optional application-defined parameters object of a job.

The member is opaque to the protocol: only its shape, its key syntax, and its serialized size are checked. Its bytes are part of job.json and are therefore covered by the immutable job digest like every other member.

Parameters:

• value (object) – The parameters mapping to validate.

• name (str) – The field name used in validation errors.

Returns:

The validated parameters mapping.

Raises:

httk.workflow.errors.FormatError – If the mapping or its serialized contents are invalid.

Return type:

dict[str, object]

httk.workflow.protocol.validate_runner_path(value, source)

Validate runner.path against the root implied by runner.source.

Every source resolves the same relative path below a different root: the job payload, the workspace runner store, or one configured installed-runner search path. The path must therefore stay below its root under every source, and only an installed runner may use the reserved pkg: form.

Parameters:

• value (object) – The runner path to validate.

• source (str) – The runner source that determines the permitted path form.

Returns:

The validated runner path.

Raises:

httk.workflow.errors.FormatError – If the path is absolute, unsafe, or incompatible with its source.

Return type:

pathlib.PurePosixPath

httk.workflow.protocol.validate_sha256(value, name)

Validate one lowercase hexadecimal SHA-256 digest string.

Parameters:

• value (object) – The digest to validate.

• name (str) – The field name used in validation errors.

Returns:

The validated digest.

Raises:

httk.workflow.errors.FormatError – If the value is not a lowercase hexadecimal digest.

Return type:

str

httk.workflow.protocol.validate_step(value, name='step')

Validate and return one workflow step name.

Parameters:

• value (object) – The step name to validate.

• name (str) – The field name used in validation errors.

Returns:

The validated step name.

Raises:

httk.workflow.errors.FormatError – If the value is not a valid step name.

Return type:

str

class httk.workflow.protocol.AttemptContext

Describe the immutable identity and restart evidence for one running attempt.

Parameters:

• workspace_id – Identify the workspace.

• job_id – Identify the job.

• job_key – Identify the job payload.

• placement – Locate the job placement.

• step – Identify the runner step.

• activation_id – Identify the activation.

• attempt_id – Identify the attempt.

• activation_ordinal – Record the activation sequence position.

• attempt_ordinal – Record the attempt sequence position.

• total_attempts – Record the planned attempt count.

• is_restart – Mark whether this attempt is a restart.

• is_unclean_restart – Mark whether the restart followed an unclean exit.

• attempt_reason – Explain why this attempt was selected.

• previous_attempt_id – Identify the preceding attempt when present.

• activation_reason – Explain why the activation was selected.

• workdir_mode – Describe how the work directory was selected.

• workdir_reused – Mark whether the work directory was reused.

• unsafe_persistent_takeover – Record whether persistent takeover was enabled.

• data_generation – Record the data generation at claim time.

• durable – Record whether storage-crash durability was enabled.

• settings – Record workspace application settings at claim time.

• resources – Record the resources assigned to the attempt.

• join – Record the job’s join description.

• raw – Preserve the complete decoded context.

workspace_id: str

job_id: str

job_key: str

placement: str

step: str

activation_id: str

attempt_id: str

activation_ordinal: int | None

attempt_ordinal: int | None

total_attempts: int | None

is_restart: bool

is_unclean_restart: bool

attempt_reason: str | None

previous_attempt_id: str | None

activation_reason: str | None

workdir_mode: str | None

workdir_reused: bool

unsafe_persistent_takeover: bool

data_generation: int | None

durable: bool

settings: collections.abc.Mapping[str, object]

resources: collections.abc.Mapping[str, object]

join: object

raw: collections.abc.Mapping[str, Any]

classmethod read(path)

Read and validate a manager-written attempt context.

Parameters:

path (str | os.PathLike[str]) – Locate the attempt context file.

Returns:

The validated attempt context.

Raises:

ValueError – If the context format or required values are invalid.

Return type:

Self

class httk.workflow.protocol.ChildReference

Identify one child in a native join.

Parameters:

• workspace_id – Identify the child’s workspace.

• job_id – Identify the child job.

• job_key – Identify the child payload and markers.

• placement_hint – Locate the child within its workspace.

workspace_id: str

job_id: str

job_key: str

placement_hint: str

as_mapping()

Return the protocol mapping for this child.

Returns:

The serialized child reference.

Return type:

dict[str, str]

class httk.workflow.protocol.JobSpec

Values needed to create an immutable native job definition.

A payload runner is a file inside the job payload. A workspace or installed runner lives outside the payload and must therefore pin its own runner_sha256, which is how one published runner serves a whole campaign of jobs without being copied per job.

Parameters:

• name – Set the job display name.

• workflow – Name the workflow.

• runner_path – Locate the runner.

• initial_step – Name the starting step.

• tag – Set the optional job tag.

• job_id – Preserve a job id when resuming or spawning.

• runner_executor – Select the runner executor.

• runner_source – Select where the runner lives.

• runner_sha256 – Pin a runner outside the payload by digest.

• runner_arguments – Supply runner arguments.

• workdir_mode – Select the workdir mode.

• workdir_path – Name the workdir below the job payload.

• data_mode – Select the job data mode.

• priority – Set the scheduling priority.

• claim_pool – Select the claim pool.

• required_capabilities – Require these manager capabilities.

• maximum_attempts_per_activation – Bound attempts in one activation.

• maximum_total_attempts – Bound attempts across the job.

• maximum_activations – Bound job activations.

• retry_on – Name manager-detected failure codes eligible for retry.

• resources – Supply resource requirements.

• parameters – Supply opaque job parameters.

• environment – Supply declared environment metadata and overrides.

• declarations – Supply workflow declarations.

• declared – Supply the declared parameter and input metadata sections.

• compatibility – Supply an optional compatibility profile.

name: str

workflow: str

runner_path: str

initial_step: str = 'start'

tag: str | None = None

job_id: str | None = None

runner_executor: str = 'path'

runner_source: Literal['payload', 'workspace', 'installed'] = 'payload'

runner_sha256: str | None = None

runner_arguments: tuple[str, Ellipsis] = ()

workdir_mode: Literal['persistent', 'isolated'] = 'persistent'

workdir_path: str = 'run'

data_mode: Literal['none', 'transactional'] = 'none'

priority: int = 500

claim_pool: str = 'default'

required_capabilities: tuple[str, Ellipsis] = ()

maximum_attempts_per_activation: int | None = None

maximum_total_attempts: int | None = None

maximum_activations: int | None = None

retry_on: tuple[str, Ellipsis] = ()

resources: collections.abc.Mapping[str, object]

parameters: collections.abc.Mapping[str, object]

environment: collections.abc.Mapping[str, object]

declarations: collections.abc.Mapping[str, collections.abc.Mapping[str, object]]

declared: collections.abc.Mapping[str, object]

compatibility: collections.abc.Mapping[str, object] | None = None

as_mapping(*, parent=None)

Return the validated job-definition mapping.

Parameters:

parent (collections.abc.Mapping[str, object] | None) – Identify the parent job when this is a spawned child.

Returns:

The mapping written to job.json.

Raises:

ValueError – If runner placement or digest settings are invalid.

Return type:

dict[str, object]

type httk.workflow.protocol.JoinCondition = Literal['all_succeeded', 'all_terminal', 'any_succeeded', 'any_terminal', 'at_least']

type httk.workflow.protocol.OutcomeAction = Literal['advance', 'retry', 'wait', 'succeed', 'fail', 'pause']

class httk.workflow.protocol.OutcomeDraft(context, control, root=None, *, durable=False)

One unpublished outcome bundle below an attempt control directory.

The draft is the single place that writes the protocol shapes of an outcome: its transaction, its spawn set, and the atomic rename that publishes it. It is bound to nothing but the attempt identity and the control directory, so the authoring SDK and the Bash bridge publish through exactly one implementation.

Parameters:

• context (httk.workflow.runtime.AttemptContext) – Provide the attempt identity and generation.

• control (pathlib.Path) – Locate the attempt control directory.

• root (pathlib.Path | None) – Locate an existing unpublished draft, when resuming one.

• durable (bool) – Synchronize the draft before publishing it.

context

control

durable = False

root

transaction()

Create the transaction builder for this outcome.

Returns:

The outcome’s transaction builder.

Raises:

• ValueError – If the attempt has no transactional data.

• RuntimeError – If the outcome already has a transaction.

Return type:

TransactionBuilder

add_child(payload, placement, *, label=None)

Register one prepared payload directory as a child of this outcome.

Parameters:

• payload (str | os.PathLike[str]) – Locate the prepared child payload.

• placement (str | pathlib.PurePosixPath) – Place the child within the workspace.

• label (str | None) – Set the child’s unique spawn label.

Returns:

The registered child reference.

Return type:

ChildReference

add_child_job(job, placement, *, label)

Register one synthesized child that needs no prepared payload.

A child whose runner lives outside the payload — a workspace or installed runner — is completely described by its job.json, so a partitioned campaign can spawn children without copying a payload tree per child.

Parameters:

• job (collections.abc.Mapping[str, object]) – Supply the synthesized child job definition.

• placement (str | pathlib.PurePosixPath) – Place the child within the workspace.

• label (str) – Set the child’s unique spawn label.

Returns:

The registered child reference.

Return type:

ChildReference

property children: tuple[ChildReference, Ellipsis]

Return the children registered in this outcome.

Returns:

The child references in registration order.

Return type:

tuple[ChildReference, Ellipsis]

publish(action, *, next_step=None, priority=None, failure=None, retry=None, join=None, pause=None, message=None, expected_data_generation=None, runner_steps=None)

Publish this outcome atomically.

Parameters:

• action (OutcomeAction) – Select the next job action.

• next_step (str | None) – Name the next step for advance or wait.

• priority (int | None) – Override the next marker priority.

• failure (collections.abc.Mapping[str, object] | None) – Supply canonical failure details for fail.

• retry (collections.abc.Mapping[str, object] | None) – Supply retry details for retry.

• join (collections.abc.Mapping[str, object] | None) – Supply join details for wait.

• pause (collections.abc.Mapping[str, object] | None) – Supply pause details for pause.

• message (str | None) – Attach an optional human-readable message.

• expected_data_generation (int | None) – Confirm the transaction generation.

• runner_steps (collections.abc.Sequence[str] | None) – Record the runner steps available to the manager.

Returns:

The authoritative published outcome path.

Raises:

• FileExistsError – If an outcome is already published.

• ValueError – If the action’s required details are invalid.

Return type:

pathlib.Path

class httk.workflow.protocol.ReplayableWorkdirBatch(workdir, root, *, durable=False)

Build a sealed, idempotently replayable set of workdir changes.

Parameters:

• workdir (pathlib.Path) – Locate the workdir receiving the changes.

• root (pathlib.Path) – Locate the batch staging directory.

• durable (bool) – Synchronize staged and applied batch directories.

workdir

root

durable = False

transaction

classmethod create(workdir, *, durable=False)

Create a new workdir batch.

Parameters:

• workdir (str | os.PathLike[str]) – Locate the workdir receiving the changes.

• durable (bool) – Synchronize the batch publications.

Returns:

The new replayable batch.

Return type:

ReplayableWorkdirBatch

seal()

Seal the batch for recovery.

Returns:

The sealed ready-directory path.

Return type:

pathlib.Path

commit()

Replay and retire the sealed batch.

Returns:

The applied batch path.

Return type:

pathlib.Path

static recover(workdir, *, durable=False)

Replay every ready batch found in a workdir.

Parameters:

• workdir (str | os.PathLike[str]) – Locate the workdir containing ready batches.

• durable (bool) – Synchronize replayed publications.

Returns:

The applied batch paths.

Return type:

tuple[pathlib.Path, Ellipsis]

class httk.workflow.protocol.RunLog(workdir)

Append structured application evidence in a workdir.

Parameters:

workdir (str | os.PathLike[str]) – Locate the workdir receiving the run log.

path

append(kind, message, *, files=())

Append one structured run-log event.

Parameters:

• kind (str) – Name the event kind.

• message (str) – Record the event message.

• files (collections.abc.Sequence[str | os.PathLike[str]]) – Attach existing regular files as evidence.

Raises:

ValueError – If kind is empty or contains NUL.

class httk.workflow.protocol.TransactionBuilder(root, *, expected_generation, durable=False)

Build a validated replayable transaction manifest.

Parameters:

• root (pathlib.Path) – Locate the transaction staging directory.

• expected_generation (int) – Pin the data generation the transaction applies to.

• durable (bool) – Synchronize the sealed manifest before returning it.

root

expected_generation

durable = False

classmethod resume(root, *, expected_generation, durable=False)

Reattach to a transaction an earlier process of this attempt sealed.

A Bash runner publishes one outcome through many short-lived processes, so the staged manifest on disk — not any in-memory counter — is what carries the operations of a draft from one call to the next. Resuming reads it back, so appending an operation continues the same sequence and the same overlap checks as the process that staged the first one.

Parameters:

• root (str | os.PathLike[str]) – Locate the sealed transaction directory.

• expected_generation (int) – Require this data generation in the manifest.

• durable (bool) – Preserve the durability setting for later sealing.

Returns:

The resumed transaction builder.

Raises:

ValueError – If the manifest format or generation is invalid.

Return type:

Self

make_dir(operation_id, path)

Stage creation of one directory.

Parameters:

• operation_id (str) – Identify the transaction operation.

• path (str | os.PathLike[str]) – Name the relative destination directory.

put_file(operation_id, source, path)

Stage one regular file for installation.

Parameters:

• operation_id (str) – Identify the transaction operation.

• source (str | os.PathLike[str]) – Locate the regular source file.

• path (str | os.PathLike[str]) – Name the relative destination path.

Raises:

ValueError – If the source is not a regular file.

put_tree(operation_id, source, path, *, replace=False)

Stage one directory tree for installation.

Parameters:

• operation_id (str) – Identify the transaction operation.

• source (str | os.PathLike[str]) – Locate the regular source directory.

• path (str | os.PathLike[str]) – Name the relative destination path.

• replace (bool) – Replace the destination tree instead of merging it.

Raises:

ValueError – If the source tree contains an unsupported entry.

remove(operation_id, path, *, missing_ok=False)

Stage removal of one path.

Parameters:

• operation_id (str) – Identify the transaction operation.

• path (str | os.PathLike[str]) – Name the relative path to remove.

• missing_ok (bool) – Allow the destination to be absent during replay.

seal()

Seal the staged operations into a manifest.

Returns:

The sealed manifest path.

Return type:

pathlib.Path

httk.workflow.protocol.join_mapping(children, condition='all_succeeded', count=None, on_impossible_step=None, additional_children=())

Return the validated join member of one waiting outcome.

Parameters:

• children (collections.abc.Sequence[ChildReference]) – Identify the child jobs to join.

• condition (JoinCondition) – Select the condition that decides the join.

• count (int | None) – Set the success threshold for at_least.

• on_impossible_step (str | None) – Name the step to advance to when the join is impossible.

• additional_children (collections.abc.Sequence[collections.abc.Mapping[str, object]]) – Add already registered child references to the join.

Returns:

The validated join mapping.

Raises:

ValueError – If the child set or condition arguments are invalid.

Return type:

dict[str, object]

httk.workflow.protocol.prepare_job_payload(destination, spec, *, parent=None, durable=False)

Create and validate job.json in an existing prepared payload.

durable synchronizes the written job.json for a caller preparing a payload directly on durable storage; it defaults to False because a payload prepared here is not yet a workspace artifact, and its submission is what makes it authoritative and durable.

Parameters:

• destination (str | os.PathLike[str]) – Locate the prepared payload directory.

• spec (JobSpec) – Supply the immutable job definition values.

• parent (collections.abc.Mapping[str, object] | None) – Identify the parent job when preparing a child.

• durable (bool) – Synchronize job.json before returning.

Returns:

The validated job definition.

Raises:

• FileExistsError – If job.json already exists.

• ValueError – If the job definition or payload runner is invalid.

Return type:

httk.workflow.models.JobDefinition

httk.workflow.protocol.replay_transaction(transaction_dir, data_dir, *, expected_generation, durable=False)

Idempotently apply one published transaction.

Returns whether the manifest contains operations and therefore advances the data generation.

When durable is set, every destination this replay installs and every directory whose entries it changes — including the parents that gained or lost a name, and the trash a removal moved into — is synchronized before this call returns. The manager relies on that ordering: it appends the destination state frame and renames the marker out of committing only after replay returns, so a committed transaction is on storage before the marker that claims it is.

Parameters:

• transaction_dir (pathlib.Path) – Locate the published transaction directory.

• data_dir (pathlib.Path) – Locate the workspace data directory to update.

• expected_generation (int) – Require this current data generation.

• durable (bool) – Synchronize installed data before returning.

Returns:

Whether the manifest contains operations.

Raises:

• httk.workflow.errors.FormatError – If the transaction manifest or an operation is invalid.

• httk.workflow.errors.TransactionError – If the generation, source, or destination is invalid.

• httk.workflow.errors.WorkspaceUnavailableError – If an operation cannot be resolved after retries.

Return type:

bool

class httk.workflow.protocol.MarkerFault

Describe one state entry shaped like a marker that cannot be interpreted.

Parameters:

• path – Identify the unusable state entry.

• reason – Explain why the entry could not be interpreted.

path: pathlib.Path

reason: str