httk.store.backend.sql.layout

Versioned physical layout for httk.store.backend.sql.store.SqlStore.

Attributes

STORAGE_PROTOCOL_VERSION	The persisted SqlStore layout protocol implemented by this package.
METADATA_TABLE_NAME	Reserved key/value table holding the store protocol and entry declaration.
WRITE_PROFILE_VOCABULARY

Exceptions

StorageLayoutUpgradeRequiredError	A database does not exactly implement the current persisted store layout.
StoreUnderConstructionError	A new open found an interrupted empty-store bulk ingest.

Classes

EntryFamilyLayout	One immutable configured entry family and its concrete records.
StorageLayout	The immutable normalized entry declaration of an initialized store.
BackendFacts	Dialect capabilities used by the SQL storage protocol.

Functions

declaration_json(layout)	Serialize a normalized declaration in its exact deterministic persisted form.
backend_facts_for_dialect(dialect_name)	Resolve the hardcoded protocol facts for one supported dialect.
normalize_entry_records(entry_records)	Normalize a declaration and apply SQL physical-name validation.
normalize_entry_families(entry_families)	Normalize application-owned declarations and apply SQL physical-name validation.
normalize_entry_declaration(entry_records, entry_families)	Merge registered and application-owned declarations and validate SQL names.
expected_metadata(layout, *[, store_timestamps])	Return SQLAlchemy metadata for all protocol-owned tables of layout.
metadata_table_for(metadata)	Return the reserved protocol key/value table in metadata.
actual_schema_objects(connection)	Return application schema-object names mapped to their stable object kinds.
actual_table_names(connection)	Return application base-table names without SQLAlchemy reflection.
actual_columns(connection, table_name)	Return the physical column names of table_name without SQLAlchemy reflection.
read_store_metadata(connection)	Read the marker values, or return None when no metadata table exists.

Module Contents

class httk.store.backend.sql.layout.EntryFamilyLayout

One immutable configured entry family and its concrete records.

name: str

family: type

definition_id: str | None

record_names: tuple[str, ...]

records: tuple[type, ...]

record_definition_ids: tuple[str | None, ...]

class httk.store.backend.sql.layout.StorageLayout

The immutable normalized entry declaration of an initialized store.

protocol_version: str

families: tuple[EntryFamilyLayout, ...]

property entry_records: collections.abc.Mapping[type, tuple[type, ...]]

Configured family classes mapped to their ordered concrete record classes.

property declaration: collections.abc.Mapping[str, tuple[str, ...]]

Configured stable family names mapped to their ordered stable record names.

exception httk.store.backend.sql.layout.StorageLayoutUpgradeRequiredError(diff, *, hint=None)

Bases: RuntimeError

A database does not exactly implement the current persisted store layout.

diff is immutable and JSON-shaped. Its top-level keys are stable categories (currently protocol, declaration and schema), so a caller can present a precise upgrade diagnostic without parsing the human-readable exception message. The declaration category maps named aspect keys (metadata_keys, store_timestamps, write_profile, entry_declaration) to their own diagnostics, so several independent declaration mismatches are reported together; the exception message names the mismatched aspects.

Parameters:

• diff (collections.abc.Mapping[str, object]) – The immutable JSON-shaped category-keyed difference.

• hint (str | None) – An optional remediation appended to the message and exposed as hint (e.g. that a purely additive schema change can be applied with upgrade=True).

diff: collections.abc.Mapping[str, object]

hint: str | None = None

httk.store.backend.sql.layout.declaration_json(layout)

Serialize a normalized declaration in its exact deterministic persisted form.

httk.store.backend.sql.layout.STORAGE_PROTOCOL_VERSION: Final = '2'

The persisted SqlStore layout protocol implemented by this package.

httk.store.backend.sql.layout.METADATA_TABLE_NAME: Final = '_httk_store_metadata'

Reserved key/value table holding the store protocol and entry declaration.

httk.store.backend.sql.layout.WRITE_PROFILE_VOCABULARY: Final

exception httk.store.backend.sql.layout.StoreUnderConstructionError

Bases: RuntimeError

A new open found an interrupted empty-store bulk ingest.

Crash window for new SQLite/DuckDB opens: before the marker commits the old clean state remains accepted; after the marker and through ingest, finalize, or before marker clear the store is rejected; after clear it is accepted again. The marker is intentionally not a resume protocol.

ClickHouse marker residue is fail-closed: the default recovery is to drop the database and re-ingest. Clearing the marker is valid only after an operator has restored and verified the declared empty-store invariant.

class httk.store.backend.sql.layout.BackendFacts

Dialect capabilities used by the SQL storage protocol.

transactional_ddl: bool

transactional_dml: bool

supports_sequences: bool

atomic_upsert: bool

serial_stage_format: Literal['sqlite', 'duckdb-attach', 'parquet']

parallel_shard_format: Literal['sqlite', 'parquet']

supports_deferred_finalize: bool

supports_degraded: bool

write_profiles: tuple[str, ...]

metadata_backend: Literal['table', 'keepermap']

supports_incremental_save: bool

system_catalog: Literal['sqlite', 'duckdb', 'clickhouse', 'postgresql']

stage_load: Literal['attach', 'duckdb-views', 'client-stream']

finalize_map_maintenance: Literal['update', 'swap']

supports_adhoc_indexes: bool

httk.store.backend.sql.layout.backend_facts_for_dialect(dialect_name)

Resolve the hardcoded protocol facts for one supported dialect.

httk.store.backend.sql.layout.normalize_entry_records(entry_records)

Normalize a declaration and apply SQL physical-name validation.

httk.store.backend.sql.layout.normalize_entry_families(entry_families)

Normalize application-owned declarations and apply SQL physical-name validation.

httk.store.backend.sql.layout.normalize_entry_declaration(entry_records, entry_families)

Merge registered and application-owned declarations and validate SQL names.

httk.store.backend.sql.layout.expected_metadata(layout, *, store_timestamps=True)

Return SQLAlchemy metadata for all protocol-owned tables of layout.

httk.store.backend.sql.layout.metadata_table_for(metadata)

Return the reserved protocol key/value table in metadata.

httk.store.backend.sql.layout.actual_schema_objects(connection)

Return application schema-object names mapped to their stable object kinds.

The DuckDB SQLAlchemy inspector presently routes column inspection through a PostgreSQL catalogue relation DuckDB does not expose, so the whole layout path intentionally uses the dialect catalogues directly.

httk.store.backend.sql.layout.actual_table_names(connection)

Return application base-table names without SQLAlchemy reflection.

httk.store.backend.sql.layout.actual_columns(connection, table_name)

Return the physical column names of table_name without SQLAlchemy reflection.

The DuckDB SQLAlchemy inspector routes column inspection through a PostgreSQL catalogue relation DuckDB does not expose (see actual_schema_objects()), so this reads the dialect catalogues directly.

Parameters:

• connection (sqlalchemy.Connection) – The open connection whose dialect selects the catalogue.

• table_name (str) – The table whose columns are read.

Returns:

The set of physical column names, empty when the table is absent.

Return type:

frozenset[str]

httk.store.backend.sql.layout.read_store_metadata(connection)

Read the marker values, or return None when no metadata table exists.