Scale benchmarks

This page is the measured scale snapshot for the opt-in runner in benchmarks/run_benchmarks.py. It is evidence about one local configuration, not a capacity guarantee. Run it with:

PYTHONPATH=src python benchmarks/run_benchmarks.py --profile quick
PYTHONPATH=src python benchmarks/run_benchmarks.py --profile snapshot --json snapshot.json

Method

The runner uses a TemporaryDirectory, a single Python process, no network, durable=False, and the local filesystem. Jobs use a minimal shell runner and are submitted with bounded-fan-out placements of the form bench/<i%64>/<batch>. The flat-placement heartbeat case uses one flat directory. The snapshot profile measures N=2,500 and N=10,000; quick uses N=500. The join case always uses one waiting parent and C=500 terminal children.

The nine rows mean:

• streamed_submission measures prepare_job_payload plus workspace submission and reports jobs per second.

• cold_scheduling_tick measures the first tick of a fresh manager over ready jobs. maximum_workers=1 remains required by the API, so the benchmark replaces the launch method with an eligibility-only no-op; no process is started.

• warm_scheduling_tick measures the immediately following tick, with the manager’s in-memory scan cursors warm.

• time_to_first_claim measures from the first tick call until the wrapped transition observes the first claimed marker, with one worker and the minimal no-op runner.

• memory_per_active_marker uses tracemalloc around a fresh marker-index rebuild and eligibility scan. It is an approximation of Python index and bookkeeping allocations, not resident set size.

• heartbeat_under_scan scans a large flat placement while wrapping the manager heartbeat hook. The wrapper forces a heartbeat write at each observed opportunity so the maximum gap is observable; it is an instrumented hook measurement, not an RSS or shared-filesystem result.

• join_evaluation measures one manager join-evaluation pass for the waiting parent and its terminal children.

• harvest_throughput measures a complete lazy harvest drain of terminal jobs.

• fsck_and_gc reports the combined wall time of a full fsck and gc pass. The JSON result also contains separate fsck_seconds and gc_seconds; gc uses journal_days=0 so retention is configured and the journal walk occurs.

Snapshot

Measured on 2026-07-27 in /home/rar/Documents/containers/devel/agents/httk2/httk-workflow:

Benchmark	Size	Wall time (s)	Rate
streamed submission	2,500	3.279033	762.42 jobs/s
streamed submission	10,000	13.480633	741.80 jobs/s
cold scheduling tick	2,500	0.852287	1.17 ticks/s
cold scheduling tick	10,000	3.419358	0.29 ticks/s
warm scheduling tick	2,500	0.852236	1.17 ticks/s
warm scheduling tick	10,000	3.360714	0.30 ticks/s
time to first claim	2,500	0.847354	1.18 claims/s
time to first claim	10,000	3.356350	0.30 claims/s
memory per active marker	2,500	4.311729	2,069.78 bytes/marker
memory per active marker	10,000	17.809655	2,117.59 bytes/marker
heartbeat under scan	2,500	1.143089	17.49 heartbeats/s
heartbeat under scan	10,000	4.439381	15.98 heartbeats/s
join evaluation (C=500)	500 children	0.669827 / 0.670408	746.46 / 745.81 children/s
harvest throughput	2,500	1.702431	1,468.49 records/s
harvest throughput	10,000	6.818689	1,466.56 records/s
fsck + gc	2,500	0.873996 (fsck 0.583754, gc 0.290242)	2,860.42 jobs/s
fsck + gc	10,000	3.509589 (fsck 2.326079, gc 1.183510)	2,849.34 jobs/s

The snapshot’s total wall time was 125.457 seconds. The machine was an Intel Xeon E5-2680 v4 system with 28 visible CPUs and 92 GiB RAM, using a local ZFS filesystem. CPU frequency, filesystem cache state, Python version, storage, and background load affect these values; they should be treated as a local snapshot rather than a portable promise.

Linearity check

The ratio below is t(10,000) / t(2,500). A ratio above approximately 5× is a review flag for super-linear behavior in this sample. None of these measured ratios crosses that flag; this is not a proof of asymptotic linearity.

Benchmark	Ratio
streamed submission	4.11×
cold scheduling tick	4.01×
warm scheduling tick	3.94×
time to first claim	3.96×
memory per active marker	4.13×
heartbeat under scan	3.88×
harvest throughput	4.01×
fsck + gc	4.02×

Join evaluation is not included in the ratio because its child count was held fixed at C=500 in both snapshot runs.

These numbers come from one local filesystem and differ on shared filesystems. Large campaigns should use project-partitioned workspaces, choosing each partition’s size from measurements appropriate to its storage and workload; see Campaigns.