httk.store¶
Provide httk-store’s data-management capability layer for httk v2.
Built on the stdlib-only contracts and models in httk-core, httk-store supplies capabilities:
in-memory
EntryProviderimplementations for the standard OPTIMADE entry types (ReferenceEntryProvider,FileEntryProvider,CalculationEntryProvider), serving httk-core’s record models through the neutral provider contract; andproperty-definition validation (
validate_property(),validate_record(),PropertyValidationError) built onjsonschema(Draft 2020-12), checking record values against their OPTIMADE property definitions fully offline; andthe store/searcher query protocols (
httk.store.query) — the backend-agnostic query contract implemented by httk data stores and consumed by serving modules; andthe federated store (
httk.store.federated_store) — ordered, immutable source and target bindings plus lazy sequential union query execution; andthe generic OPTIMADE filter translation (
httk.store.query.optimade_filters) — turning filter syntax trees parsed byhttk.core.optimade.parse_optimade_filter()into search expressions over the query protocols (the machinery inhttk.store.query.optimade_filters, includingfilter_searcher()), with neutralFilterTranslationErrorcategories; andthe database storage layer (
httk.store.db, requiring thehttk-store[db]extra) — relational storage and querying of plain frozen dataclasses (SqlStoreover SQLite or DuckDB), served through the provider contract byStoreEntryProvider.
The providers self-register (under httk.registry.entries.store, as
store-references/store-files/store-calculations/store-db-store)
when httk.core discovers the module, so a serving module (such as
httk-serve) can find them through the registry.
- class httk.store.StandardEntryProvider¶
Submodules¶
Attributes¶
Why a filter could not be translated (see |
Exceptions¶
Report that a named source rejected or failed a federated operation. |
|
Report that a federation-level store operation failed. |
|
Report that a store cannot provide an exact query count. |
|
Report that a result-set |
|
Report that a result-set |
|
Report that a continuation cursor is malformed, expired, or belongs to another result plan. |
|
Report that a valid query operation is outside a store's supported profile. |
|
Report that a filter cannot be translated into a search expression. |
|
Report that a value did not conform to its OPTIMADE property definition. |
Classes¶
Serves OPTIMADE |
|
Serve core |
|
Serves OPTIMADE |
|
Serves OPTIMADE |
|
Serve core |
|
Represent a frozen, lazy, re-iterable federated result plan. |
|
Build and validate one portable, single-root federated query. |
|
Fan out read-only queries over an ordered collection of borrowed stores. |
|
Bind one logical target to exact concrete targets for named sources. |
|
Carry an opaque URL-safe continuation value. |
|
Expose optional continuation-page capability on a frozen result set. |
|
Order a continuation page by one named scalar result projection. |
|
Describe the query operations guaranteed by one property definition. |
|
Represent an immutable continuation-page result. |
|
Represent one named result row by position, name, or attribute. |
|
Require named access to one result row. |
|
Require the common operations of a materialized result set. |
|
Build one query and iterate its results. |
|
Require composable backend search expressions. |
|
Expose a queryable field of a search variable. |
|
Represent one match with declared output values and names. |
|
Bind a query variable to a target type whose attributes yield fields. |
|
Require a store that can create a query searcher. |
Functions¶
|
Build source-side relationships for a provider's |
|
Derive the portable operation subset for |
|
Return the ordered portable query fields described by |
|
Build a |
|
Validate a single |
|
Validate every property present in |
Package Contents¶
- class httk.store.CalculationEntryProvider(entries, *, relationships=None)[source]¶
Bases:
StandardEntryProviderServes OPTIMADE
calculationsfrom a mapping of id toCalculation.relationshipsoptionally maps a calculation id to its related entries (RelatedEntryvalues, served flat per id) — e.g. itsinput/outputfiles, expressed via therolemetadata.- Parameters:
entries (collections.abc.Mapping[str, httk.core.Calculation | collections.abc.Mapping[str, Any]]) – The calculations keyed by their served identifiers.
relationships (collections.abc.Mapping[str, collections.abc.Iterable[httk.core.RelatedEntry]] | None) – Optional related entries keyed by calculation identifier.
- class httk.store.DataRecordEntryProvider(entries, *, definitions=None, relationships=None)[source]¶
Bases:
httk.core.EntryProviderServe core
DataRecordvalues as provider properties.Definitions are resolved eagerly at construction. Every served property name must start with
_; absent record properties are emitted as JSON null.- Parameters:
entries (collections.abc.Mapping[str, httk.core.DataRecord | collections.abc.Mapping[str, Any]]) – The data records keyed by their served identifiers.
definitions (collections.abc.Mapping[str, httk.core.PropertyDefinition] | None) – Optional property definitions keyed by served property name.
relationships (collections.abc.Mapping[str, collections.abc.Iterable[httk.core.RelatedEntry]] | None) – Optional related entries keyed by record identifier.
- Raises:
ValueError – If a property name, definition, or non-nullable property is inconsistent with the supplied records.
- entry_types()[source]¶
Return the resolved
_httk_recordsentry definition.- Returns:
The served data-record entry-type definition.
- Return type:
- property_keys(entry_type)[source]¶
Return served property names mapped to data-record keys.
- relationships(entry_type)[source]¶
Return normalized data-record relationships by identifier.
- Parameters:
entry_type (str) – The entry type to inspect.
- Returns:
The relationship mapping supplied at construction.
- Raises:
KeyError – If
entry_typeis not_httk_records.- Return type:
collections.abc.Mapping[str, tuple[httk.core.RelatedEntry, Ellipsis]]
- class httk.store.FileEntryProvider(entries, *, relationships=None)[source]¶
Bases:
StandardEntryProviderServes OPTIMADE
filesfrom a mapping of id toFile.relationshipsoptionally maps a file id to its related entries (RelatedEntryvalues, served flat per id) — e.g. the calculations a file isinput/outputof.- Parameters:
entries (collections.abc.Mapping[str, httk.core.File | collections.abc.Mapping[str, Any]]) – The files keyed by their served identifiers.
relationships (collections.abc.Mapping[str, collections.abc.Iterable[httk.core.RelatedEntry]] | None) – Optional related entries keyed by file identifier.
- class httk.store.ReferenceEntryProvider(entries, *, relationships=None)[source]¶
Bases:
StandardEntryProviderServes OPTIMADE
referencesfrom a mapping of id toReference.relationshipsoptionally maps a reference id to its related entries (RelatedEntryvalues, served flat per id).- Parameters:
entries (collections.abc.Mapping[str, httk.core.Reference | collections.abc.Mapping[str, Any]]) – The references keyed by their served identifiers.
relationships (collections.abc.Mapping[str, collections.abc.Iterable[httk.core.RelatedEntry]] | None) – Optional related entries keyed by reference identifier.
- class httk.store.RunEntryProvider(entries)[source]¶
Bases:
httk.core.EntryProviderServe core
Runrecords and their provenance edges.- Parameters:
entries (collections.abc.Mapping[str, httk.core.Run | collections.abc.Mapping[str, Any]]) – The runs keyed by their served identifiers.
- entry_types()[source]¶
Return the vendored
_httk_runsentry definition.- Returns:
The served run entry-type definition.
- Return type:
- property_keys(entry_type)[source]¶
Return the served run-property to record-key mapping.
- relationships(entry_type)[source]¶
Return run provenance edges with role and edge-label metadata.
- Parameters:
entry_type (str) – The entry type to inspect.
- Returns:
Relationships grouped by run identifier.
- Raises:
KeyError – If
entry_typeis not_httk_runs.- Return type:
collections.abc.Mapping[str, tuple[httk.core.RelatedEntry, Ellipsis]]
- httk.store.product_relationships(links)[source]¶
Build source-side relationships for a provider’s
relationships=argument.Feed the inner mapping into the source-side provider’s
relationships=argument; per-edgeworkflow_declaration_uriis deliberately not served yet (relation-object serving is future work).- Parameters:
links (collections.abc.Iterable[httk.core.ProductLink]) – The product links to group by source type and identifier.
- Returns:
Source-type mappings of source identifiers to related product entries.
- Raises:
ValueError – If one source has duplicate product labels.
- Return type:
dict[str, dict[str, tuple[httk.core.RelatedEntry, Ellipsis]]]
- class httk.store.FederatedResultSet(store, plan)[source]¶
Represent a frozen, lazy, re-iterable federated result plan.
Results execute source-major in federation source order, preserve duplicate rows, and remain read-only views over the borrowed stores.
- Parameters:
store (FederatedStore) – The federation whose sources execute the plan.
plan (object) – The validated frozen federation plan.
- first()[source]¶
Return the first result row, or
Nonewhen no row matches.- Returns:
The first matching row, or
None.- Return type:
httk.store.query.ResultRow | None
- one()[source]¶
Return the only result row.
- Returns:
The sole matching result row.
- Raises:
httk.store.query.protocols.NoResultError – If no row matches.
httk.store.query.protocols.MultipleResultsError – If more than one row matches.
- Return type:
- scalars(name=None)[source]¶
Iterate over one named projection.
- Parameters:
name (str | None) – The projection name, required when more than one output exists.
- Returns:
An iterator over the selected projection values.
- Raises:
ValueError – If no name is supplied for multiple outputs.
KeyError – If
nameis not a declared output.
- Return type:
- column(name)[source]¶
Return a lazy scalar column by projection name.
- Parameters:
name (str) – The scalar projection name.
- Returns:
A lazy column view over the projection.
- Raises:
- Return type:
- abstractmethod cursor()[source]¶
Reject cursor access because federation cursors are unsupported.
- Returns:
Never returns.
- Raises:
NotImplementedError – Always, because federated cursors are not implemented.
- Return type:
- class httk.store.FederatedSearcher(store)[source]¶
Build and validate one portable, single-root federated query.
- Parameters:
store (FederatedStore) – The federation whose child stores provide the query surface.
- offset = 0¶
- origin¶
- variable(target)[source]¶
Bind one shared or explicit target against child searcher prototypes.
- Parameters:
target (object) – A shared child target or a source-specific target binding.
- Returns:
The federated root variable.
- Raises:
httk.store.query.protocols.UnsupportedQueryError – If a second root or foreign target is supplied.
FederatedSourceError – If a source rejects target binding.
- Return type:
- add(expression)[source]¶
Validate and retain a portable condition for the future frozen plan.
- Parameters:
expression (object) – An expression produced by this searcher.
- Returns:
None.
- Raises:
httk.store.query.protocols.UnsupportedQueryError – If the expression belongs to another searcher.
FederatedSourceError – If a source rejects the expression.
- Return type:
None
- output(value, name)[source]¶
Declare a record, scalar field, or origin output for a future plan.
- Parameters:
- Returns:
None.
- Raises:
ValueError – If
nameis empty or already declared.httk.store.query.protocols.UnsupportedQueryError – If
valueis not owned by this searcher.FederatedSourceError – If a source rejects the output.
- Return type:
None
- add_sort(field, descending=False)[source]¶
Reject global sorting until a portable sort-semantics contract exists.
- Parameters:
- Raises:
httk.store.query.protocols.UnsupportedQueryError – Always, because global federation sorting has no portable contract.
- count()[source]¶
Return the exact unpaged count of the current filtered union.
- Returns:
The exact sum of matching rows across participating sources.
- Raises:
httk.store.query.protocols.CountUnavailableError – If a source cannot provide an exact count.
FederatedSourceError – If a source fails while counting.
- Return type:
- add_offset(offset)[source]¶
Add a global source-union offset.
- Parameters:
offset (int) – The nonnegative number of union rows to skip.
- Returns:
None.
- Raises:
TypeError – If
offsetis not an integer.ValueError – If
offsetis negative.
- Return type:
None
- results(**outputs)[source]¶
Freeze a projection plan into a lazy, re-iterable result set.
- Parameters:
**outputs (object) – Optional output names mapped to root variables or fields.
- Returns:
The lazy frozen result set.
- Raises:
ValueError – If no outputs are declared or an output name is invalid.
httk.store.query.protocols.UnsupportedQueryError – If an output does not belong to this searcher.
FederatedSourceError – If a source rejects an output.
- Return type:
- exception httk.store.FederatedSourceError(source, operation)[source]¶
Bases:
FederatedStoreErrorReport that a named source rejected or failed a federated operation.
- Parameters:
- source¶
- operation¶
- class httk.store.FederatedStore(sources)[source]¶
Fan out read-only queries over an ordered collection of borrowed stores.
The union is source-major, lazy, and non-deduplicating. Queries require the strict common query surface accepted by every participating source, and counts are exact sums of the unpaged source counts. This live borrowed-store view is distinct from the persisted registry in
httk.store.db.stored_federation.- Parameters:
sources (collections.abc.Mapping[str, httk.store.query.Store]) – Child stores keyed by stable federation source name.
- Raises:
TypeError – If
sourcesis not a mapping.ValueError – If fewer than two sources or an invalid source name is supplied.
- property source_names: tuple[str, Ellipsis]¶
Return the immutable source names in constructor iteration order.
- target(name, targets)[source]¶
Create an immutable target mapping for an intentional source subset.
- Parameters:
name (str) – The logical target name.
targets (collections.abc.Mapping[str, object]) – Concrete targets keyed by federation source name.
- Returns:
The validated target binding.
- Raises:
TypeError – If
targetsis not a mapping.ValueError – If a target name or source name is invalid.
- Return type:
- exception httk.store.FederatedStoreError[source]¶
Bases:
RuntimeErrorReport that a federation-level store operation failed.
- class httk.store.FederatedTarget[source]¶
Bind one logical target to exact concrete targets for named sources.
- Parameters:
name – The nonempty logical target name.
targets – Concrete targets keyed by federation source name.
_owner – The federation that owns this target binding.
- Raises:
TypeError – If
targetsis not a mapping or_owneris not a federation.ValueError – If the name, source set, or source names are invalid.
- targets: collections.abc.Mapping[str, object]¶
- class httk.store.ContinuationToken[source]¶
Bases:
strCarry an opaque URL-safe continuation value.
It is a
strsubclass so normal JSON serializers preserve it as a scalar value. Applications should pass a token returned by a page back unchanged; data backends validate its version, structure, and result-plan fingerprint before using any decoded value as a bound parameter.- Parameters:
value – The opaque continuation value.
Bases:
RuntimeErrorReport that a store cannot provide an exact query count.
- exception httk.store.MultipleResultsError[source]¶
Bases:
LookupErrorReport that a result-set
one()operation found multiple results.
- exception httk.store.NoResultError[source]¶
Bases:
LookupErrorReport that a result-set
one()operation found no matching result.
- class httk.store.PageableResultSetLike[source]¶
Bases:
ProtocolExpose optional continuation-page capability on a frozen result set.
This deliberately extends neither
ResultSetLikenorSearcher: stores that do not support seek pagination remain fully conforming to the required portable contracts.
- class httk.store.PageOrder[source]¶
Order a continuation page by one named scalar result projection.
nameidentifies the name supplied toresults()(orSearcher.output()), never a backend column object. The result-set implementation validates that it is a root scalar projection before it generates SQL.- Parameters:
name – The declared scalar output name used for ordering.
descending – Whether to order this field in descending order.
nulls – Whether null values sort first or last.
- nulls: Literal['first', 'last'] = 'last'¶
- exception httk.store.PaginationCursorError[source]¶
Bases:
ValueErrorReport that a continuation cursor is malformed, expired, or belongs to another result plan.
- class httk.store.PortableQueryCapabilities[source]¶
Describe the query operations guaranteed by one property definition.
query-supportexpresses a cross-provider guarantee, not a particular server’s implementation detail.all optionalis deliberately fail-closed here: it gives a portable client no operation it can rely on. A server may offer more, but that is not represented by the definition.- Parameters:
query_support – The normalized declared query-support level.
operations – The portable operation families guaranteed by the definition.
- class httk.store.ResultPage[source]¶
Represent an immutable continuation-page result.
rowsis always a tuple. Returned rows are ordinary persistent result rows, not the expiring proxies produced bySqlResultSet.cursor().totalis populated only when the caller explicitly asks for it.- Parameters:
rows – The persistent rows returned by the page.
next – The token for the next page, if one exists.
previous – The token for the previous page, if one exists.
total – The exact result count when requested, otherwise
None.
- rows: tuple[ResultRowLike, Ellipsis]¶
- next: ContinuationToken | None¶
- previous: ContinuationToken | None¶
- class httk.store.ResultRow(values, names, resolver=None, guard=None)[source]¶
Represent one named result row by position, name, or attribute.
- Parameters:
- class httk.store.ResultSetLike[source]¶
Bases:
ProtocolRequire the common operations of a materialized result set.
- class httk.store.Searcher[source]¶
Bases:
ProtocolBuild one query and iterate its results.
Iteration yields one
SearchResultper match, soitem[0][0]is the first declared output of the match (typically the matched row object). The expressions received byaddare always ones produced by this same backend’s search variables, so implementations may type them as their own expression class; a backend that needs a second (post-filter) evaluation position decides that from the expression itself, not from the caller.
- class httk.store.SearchExpression[source]¶
Bases:
ProtocolRequire composable backend search expressions.
- class httk.store.SearchField[source]¶
Bases:
ProtocolExpose a queryable field of a search variable.
In addition to the methods below, fields support the rich comparison operators (
==,!=,<,<=,>,>=), returningSearchExpression. The handlers invoke those viagetattr(field, '__eq__')(value)since the comparison dunders cannot be typed as expression-returning.The three string-matching methods take literal text: no wildcard or pattern syntax whatsoever crosses this contract, so
%and_(and any other metacharacter) match themselves. A backend is therefore free to implement them with SQLLIKEover an escaped pattern, with a regular expression, or with a full-text index — the choice is invisible here.- is_in(*values)[source]¶
Match a root scalar field whose value is one of
values.Noneis an explicit member: it matches a null field value, and its negation excludes nulls rather than inheriting SQL’s three-valuedNOT IN (..., NULL)behavior.Backends define the corresponding semantics for child or set fields; for example, a backend may use the existing
has_only-style all-values reading for a child field.
- class httk.store.SearchResult[source]¶
Bases:
NamedTupleRepresent one match with declared output values and names.
valuesholds one entry perSearcher.output()call in declaration order; it is a tuple, sovalues, names = resultandresult[0][0]both work.
- class httk.store.SearchVariable[source]¶
Bases:
ProtocolBind a query variable to a target type whose attributes yield fields.
always_true/always_falseare reserved names: they are real methods of the variable, never stored fields resolved through__getattr__. They exist so a translation layer can express a constant truth value without inventing a probe field. Afield == fieldprobe is NULL-unsound, since it yields NULL (not true) for a NULL field.
- exception httk.store.UnsupportedQueryError[source]¶
Bases:
ValueErrorReport that a valid query operation is outside a store’s supported profile.
- httk.store.portable_query_capabilities(definition)[source]¶
Derive the portable operation subset for
definition.The operation names are
"equality","ordering","stringmatching", and"set". They intentionally describe the query-language operation families rather than storage implementation.IS [NOT] KNOWNis part of the equality family because it is the NULL spelling of equality/inequality in the OPTIMADE filter language.- Parameters:
definition (httk.core.PropertyDefinition) – The OPTIMADE property definition to inspect.
- Returns:
The guaranteed portable query capabilities.
- Return type:
- httk.store.portable_query_fields(entry_type, *, include=(), exclude=())[source]¶
Return the ordered portable query fields described by
entry_type.By default, this selects scalar fields and flat lists with at least one operation guaranteed by their definition.
includeis an explicit binding override for named existing properties; it is appended as a second ordered group after the derived fields, in entry-definition order among the included names, but does not manufacture query capabilities absent from that definition.excludealways wins. Both arguments reject unknown or duplicate names so binding mistakes cannot silently broaden a profile.- Parameters:
entry_type (httk.core.EntryTypeDefinition) – The entry definition whose properties are inspected.
include (collections.abc.Iterable[str]) – Existing property names to append to the derived selection.
exclude (collections.abc.Iterable[str]) – Existing property names to remove from the selection.
- Returns:
Derived property names followed by explicitly included names.
- Raises:
ValueError – If
includeorexcludecontains an unknown or duplicate property name.- Return type:
- type httk.store.FilterTranslationCategory = Literal['unrecognized-property', 'not-implemented', 'type-mismatch', 'internal'][source]¶
Why a filter could not be translated (see
FilterTranslationError).
- exception httk.store.FilterTranslationError(message, category, detail=None)[source]¶
Bases:
ExceptionReport that a filter cannot be translated into a search expression.
The exception message describes the failure;
categoryclassifies it neutrally (this module knows nothing about transports, so consumers map each category onto their own error codes):"unrecognized-property"— the filter names an unknown property carrying a recognized prefix (a caller error);"type-mismatch"— a filter constant does not match the property’s declared type (a caller error);"not-implemented"— the filter uses a construct this translation (or the supplied handler table) does not support;"internal"— an inconsistency in the translation itself.
detailoptionally carries extra machine-readable context.- Parameters:
message (str) – The human-readable translation failure.
category (FilterTranslationCategory) – The neutral failure category.
detail (str | None) – Optional machine-readable failure context.
- category: FilterTranslationCategory¶
- detail = None¶
- httk.store.filter_searcher(store, target, filter_string, *, entry_type, property_fulltypes, property_keys=None, handlers=None, recognized_prefixes=(), relationship_targets=(), related_property_resolver=None)[source]¶
Build a
Searcheroverstoreapplying an OPTIMADE filter.filter_stringis an OPTIMADE filter string (parsed withhttk.core.optimade.parse_optimade_filter()) or an already-parsedFilterAst. The searcher binds one search variable totarget(the store-specific query target, declared as the searcher output namedentry_type) and applies the translated filter. Whenhandlersis not supplied, a default table is built withsimple_property_handlers()fromproperty_keys(or, whenproperty_keysis also None, from an identity map overproperty_fulltypes). The remaining keyword arguments are passed through totranslate_filter_ast().- Parameters:
store (httk.store.query.Store) – The backend store on which to build the searcher.
target (Any) – The backend query target to bind.
filter_string (str | httk.core.optimade.FilterAst) – An OPTIMADE filter string or parsed filter AST.
entry_type (str) – The output name and served entry type.
property_fulltypes (collections.abc.Mapping[str, str]) – Fulltypes keyed by recognized property name.
property_keys (collections.abc.Mapping[str, str] | None) – Optional mapping from property names to backend field names.
handlers (HandlerTable | None) – Optional prebuilt property handler table.
recognized_prefixes (tuple[str, Ellipsis]) – Prefixes whose unknown properties are errors.
relationship_targets (tuple[str, Ellipsis]) – Related entry types that support dotted filters.
related_property_resolver (RelatedPropertyResolver | None) – Optional resolver for related-property filters.
- Returns:
A searcher with the translated filter already applied.
- Raises:
FilterTranslationError – If the filter cannot be translated.
httk.core.optimade.ParserSyntaxError – If a filter string does not parse.
- Return type:
- exception httk.store.PropertyValidationError(name, message)[source]¶
Bases:
ValueErrorReport that a value did not conform to its OPTIMADE property definition.
Carries the offending property
nameand a human-readablemessage. For single-value failures the message wraps the underlyingjsonschemaerror message, and thatjsonschema.exceptions.ValidationErroris preserved as the chained__cause__.- Parameters:
- name¶
- message¶
- httk.store.validate_property(definition, value)[source]¶
Validate a single
valueagainstdefinition’s JSON-Schema payload.Builds a
jsonschema.Draft202012Validatordirectly from the definition’s document (with the$schemameta-schema reference removed) and validatesvalueagainst it using the local format checker. ReturnsNoneon success; raisesPropertyValidationErroron failure, chaining the underlyingjsonschema.exceptions.ValidationErroras the cause. No network access or registry lookup ever happens.- Parameters:
definition (httk.core.PropertyDefinition) – The self-contained OPTIMADE property definition.
value (Any) – The value to validate.
- Returns:
None.
- Raises:
PropertyValidationError – If
valueviolatesdefinition.- Return type:
None
- httk.store.validate_record(entry_type, record)[source]¶
Validate every property present in
recordagainstentry_type.Each key in
recordmust be described byentry_type; unknown property names are rejected with aPropertyValidationErrornaming them and the entry type.idandtypemust both be present. Properties described by the definition but absent fromrecordare simply not checked (serving a subset of the described properties is normal). The value of every property that is present is validated viavalidate_property(). ReturnsNoneon success.- Parameters:
entry_type (httk.core.EntryTypeDefinition) – The entry definition describing allowed properties.
record (collections.abc.Mapping[str, Any]) – The record mapping to validate.
- Returns:
None.
- Raises:
PropertyValidationError – If a property is unknown,
idortypeis missing, or a value violates its property definition.- Return type:
None