httk-workflow¶

This site documents the httk-workflow module. For the full documentation of httk₂, see docs.httk.org.

The module implements a recoverable, language-neutral workflow protocol whose source of truth is a single atomically renamed state marker per job. The httk-taskmanager executable remains a compatibility alias. It presents three layers, each with its own import home: the filesystem protocol (httk.workflow.protocol), the execution and authoring surface (httk.workflow — Runner, Attempt — with lower-level helpers in httk.workflow.runtime), and orchestration and management (Workspace, TaskManager, collect, and named submodules for transfers, remotes, and compatibility). Installations register the canonical httk workflow command tree and the httk-taskmanager alias.

httk₂ workflows are language-independent: runners, hooks, and postprocess scripts can be written in any language; a workflow is a manifest plus the members it references. Python hooks remain first-class, with an in-process fast path. Successful hook outputs use the same assembly semantics as executable hooks; collector failures differ deliberately: registered .py exceptions abort iteration, while executable-hook errors degrade per job and continue the sweep.

Quick links

New here — start with Quickstart: five commands from an empty directory to a finished relaxation, no runner written and no VASP required.

The filesystem protocol — the language-neutral on-disk contract

The filesystem protocol API — the httk.workflow.protocol namespace
Workflow filesystem API — the normative on-disk specification

The execution API — writing and running workflow steps

Native runner helpers — the Python authoring SDK: Runner, Attempt, steps
Runner SDKs — the same authoring surface in eight more languages
Packaged VASP runners — the packaged runners and relaxation report, for campaigns that write none
Workflow packages — authoring directory packages and their manifest
Workflow declarations — saying what a workflow is, for a data layer
Run provenance — turning one JobRecord into one httk.core.Run
Collecting results — collecting provider-produced outputs and products
Workflow-language runner realizations — CWL, PWD, jobflow, and httk-v1 runner realizations
Examples — worked examples as a notebook

Orchestration and management — driving and inspecting a workspace

Task-manager usage — workspaces, submission, managers, inspection, repair
Project and workflow command line — the whole httk workflow tree: projects, config, remotes
Campaigns — partitioning a very large campaign across many workspaces
Scale benchmarks — measured local scale snapshot and benchmark methodology
Collecting results — reading finished jobs back out as records and collected outputs
Writing a remote adapter — reaching a machine the packaged adapters do not cover
Reference — the generated API reference

Migration

Install¶

Preferably work in a Python virtual environment:

git clone https://github.com/httk/httk-workflow
cd httk-workflow
python -m pip install -e .

Minimal setup¶

One workspace, one job of a packaged runner, and one manager that runs it:

httk project init --name quickstart
httk workflow workspace init . --name default
httk workflow job new --workflow vasp-relax --input structure=POSCAR --tag silicon
httk workflow workspace settings set vasp.command "$PWD/examples/mock_vasp.py"
httk workflow run
httk workflow collect

Quickstart walks through exactly those commands, including how to run them without VASP installed. A complete payload prepared some other way is still submitted directly:

httk workflow job submit workflow-workspace prepared-job --placement project/00

Documentation

Details