# Reading a remote OPTIMADE service

`OptimadeStore` is the synchronous, read-only client entry point. It discovers
the remote schema eagerly, keeps the exact redacted `/info/<entry>` documents
that describe every result, and exposes the neutral `Store`/`Searcher` query
profile from *httk-store*.

```python
from httk.serve.optimade import OptimadeStore

with OptimadeStore("https://example.org/optimade") as store:
    for entry_type in store.entry_types:
        print(entry_type.name, entry_type.definition_id)
```

For an unversioned base URL, construction first requests `/versions`, reads its
strict preference-ordered restricted CSV, and selects the first major version
supported by httk (currently major `1`). It then requests `/v1/info`, followed
by `/v1/info/<name>` for every advertised entry endpoint, in advertised order.
`requested_base_url` is the normalized URL supplied by the caller and
`base_url` is the negotiated effective versioned URL used for all discovery and
query requests. Both public fields redact URL credentials; authenticated
transport requests retain the supplied credentials privately. An explicit final
path segment of `v1`, `v1.2`, or `v1.2.3` skips `/versions` and is used
unchanged; an explicit unsupported major, malformed version-like segment, or
version suffix fails before discovery. Malformed `/versions` CSV, duplicate
major versions, an empty list, or no compatible advertised major raise
`OptimadeVersionNegotiationError` rather than falling back to unversioned
`/info`.

The base `/info` response's `data.attributes.api_version` selects the
entry-info grammar. OPTIMADE 1.0 and 1.1 put `description`, `properties`,
`formats`, and `output_fields_by_format` directly in `/info/<entry>.data` and
do not require `type` or `id` there. OPTIMADE 1.2 and later use the newer info
resource form, for which httk requires `data.type == "info"`. This is protocol
version support, not a permissive fallback: a service that declares 1.2 or
later is still validated against the newer form. `api_version` exposes the
version declared by the service, or `None` for a legacy test/service that
omits it.

`entry_types` is an immutable tuple of
`RemoteEntryType` descriptors; `entry_type(name)` and the immutable
`entry_types_by_name` mapping provide transport-name lookup. `refresh()` makes
a complete new generation and replaces the descriptors only after every
request and validation succeeds, so objects from an earlier generation retain
their original schema snapshot.

## Semantic recognition

An endpoint's transport spelling is never its meaning. A `RemoteEntryType` is
typed only when its `links.describedby` is an exact registered entry-definition
IRI, or, when that link is absent, when the advertised property-definition
IRIs identify exactly one local binding. Missing, malformed, ambiguous, or
contradictory IRIs leave the endpoint generic. In particular, an unknown
`describedby` does not fall back to endpoint or property names.

Recognized standard endpoints resolve to `OptimadeReference`, `OptimadeFile`,
or `OptimadeCalculation`; a recognized structures endpoint resolves to
`httk.atomistic.OptimadeStructure` when *httk-atomistic* is installed. Generic
resources use `httk.core.optimade.OptimadeResource`. All of these retain their original
immutable source resource:

For ordinary use, obtain the backend from a result (shown below), then choose a
view only when needed: `ReferenceView(backend)`, `FileView(backend)`, and
`CalculationView(backend)` provide lazy canonical record views. For a typed
structure, `UnitcellStructureView(optimade_structure)` is likewise lazy: an
incomplete remote structure remains inspectable and storable until a component
that needs unavailable structural data is requested.

Every backend has `unwrap()`; `OptimadeResource.unwrap()` returns the immutable
JSON:API resource mapping. The enclosing `OptimadeDocument` keeps raw response
text (with credential-bearing URLs redacted) and the `OptimadeSchemaSnapshot`
that was current when the resource was read. This makes `unwrap()` the escape
hatch for unknown extension fields and exact source provenance, rather than a
lossy conversion API.

## One-root portable queries

The remote query implementation deliberately supports one root endpoint per
query. Start from either a unique typed backend class or, where there may be
more than one endpoint of a type, the exact `RemoteEntryType` descriptor.

```python
from httk.serve.optimade import OptimadeStore

store = OptimadeStore("https://example.org/optimade")
references = store.entry_type("references")

search = store.searcher()
reference = search.variable(references)
search.add(reference.id.startswith("cod/"))
rows = search.results(reference=reference, identifier=reference.id)

for row in rows:
    print(row.identifier, row.reference.unwrap().document.source_url)
```

`id`, `type`, `immutable_id`, and `last_modified` are the portable core fields
where they are advertised. Typed definitions can expose additional portable
query-supported scalar and flat-list properties; the exact available fields
come from the discovered definition rather than from their remote spelling.
List fields support singular `has(value)`, `has_any(...)`, and
`has_only(...)`. Use `add()`, `add_sort()`, `set_limit()`, and `add_offset()`
to finish the single-root plan. Depth-1 relationship traversal is available
through `variable.links`, described below; deeper joins, writes, and
asynchronous queries are outside this client.

`results()` produces a lazy, reusable `RemoteResultSet`. It supports iteration,
`first()` (`None` when empty), `one()` (raising `NoResultError` or
`MultipleResultsError`), `scalars()`, and scalar `column(name)` projections.
`search.count()` and `len(results)` use the server's filtered
`meta.data_returned` (the total for the current filter query, independent of
pagination); they raise `CountUnavailableError` when that exact count is absent
or invalid. `results[start:stop]` makes a derived lazy plan when
both bounds are nonnegative integers and the step is omitted or `1`; integer
indexing, negative bounds, and non-unit steps are unsupported. Cursor rows are
not implemented.

## The `links` relationship namespace

`variable.links` is a reserved relationship namespace, checked before the
endpoint's own field map, so a served property literally named `links`
(unusual, but a generic endpoint could advertise one) never shadows it.
Attribute access on it names one relationship of the queried resource --
either a served entry type (`material.links.structures`) or a wire-only
relationship key that is not itself an entry type, such as a StrongLink
provenance edge (`run.links._httk_has_input`). Unlike the field map, an
unknown name is never rejected up front: a resource's actual relationship set
is only known from a response, so it surfaces as `OptimadeClientError` at
resolution time (a predicate, an output, or `.links.<name>` on a returned
record), listing the resource's real relationships.

`variable.links.<name>` serves two roles:

- **A depth-1, filter-only predicate root.** `variable.links.<related>.<field>`
  renders as `<related>.<field>` in filter text, exactly as the server
  expects. This is exactly one level deep: the resulting field cannot be
  chained further, and cannot be used as an output or a sort key. Field
  chaining is only available when `<related>` is itself a served entry type;
  chaining on a wire-only relationship key raises `UnsupportedQueryError`. A
  dotted filter whose first segment is a served type but *not* a relationship
  of the queried endpoint is accepted by the server but returns zero rows,
  with an explanatory `meta.warnings` entry -- the client logs every
  `meta.warnings` entry (via `logging.getLogger(__name__).warning`) rather
  than silently dropping it.
- **A set-valued output**, in `output()` or `results(name=...)`. It yields a
  tuple of bound related records per row, in relationship order. Every
  served-entry-type relationship named this way is added to the request's
  `include=` parameter automatically, ordered and de-duplicated (a wire-only
  key is never a served entry type and always resolves by lazy fetch
  instead); `count()`'s own probe request never sends `include=`.

Every whole-record result -- an output, or a record reached through
`.links.<name>` -- is a *bound* object: alongside its own fields, it carries
a `.links` accessor with exactly this resolution behavior, so relationships
can be walked one hop at a time from any returned record, whether or not that
record was itself declared as an output. Resolution matches identifiers
against the response's `included` array by their own `(type, id)`, never by
the relationship block key: some relationships (StrongLink provenance edges)
use wire keys that differ from the identifier's own `type`. A related
resource already present in `included` -- because an output requested it, or
the service includes it by default -- is wrapped in place at no extra cost;
one absent from `included` costs one HTTP request per missing identifier.
Each named relationship resolves once per record and is memoized.

```python
from httk.serve.optimade import OptimadeStore

with OptimadeStore("https://example.org/optimade") as store:
    materials = store.entry_type("materials")
    search = store.searcher()
    material = search.variable(materials)
    search.add(material.links.structures.nelements > 2)
    row = search.results(item=material, structures=material.links.structures).one()

    (structure,) = row.structures
    print(structure.id, structure.chemical_formula_reduced)
    assert row.item.links.structures == row.structures

    # Further hops walk .links on any returned record, output or not.
    for run in row.item.links._httk_runs:
        print(run.links._httk_has_input)
```

Bound objects are a thin per-backend subclass added purely for the `.links`
accessor: equality, hashing, and `isinstance` against the plain backend class
all behave as if it were the plain object. `pickle`, `copy.copy`, and
`copy.deepcopy` produce the plain class; `dataclasses.replace` produces an
unbound instance of the same thin subclass whose `.links` is unavailable --
so a bound object is safe to store, cache, or serialize with ordinary tools,
but only a fresh result (an output, or one reached through `.links`) exposes
`.links` itself.

## Federating endpoints

`FederatedStore` (from *httk-store*) combines already-open stores into one
read-only, source-major union. Manage the remote connections yourself: the
federation borrows them and never closes either endpoint.

```python
from contextlib import ExitStack

from httk.atomistic import OptimadeStructure
from httk.store import FederatedStore
from httk.serve.optimade import OptimadeStore

with ExitStack() as stack:
    first = stack.enter_context(OptimadeStore("https://first.example/optimade"))
    second = stack.enter_context(OptimadeStore("https://second.example/optimade"))
    combined = FederatedStore({"first": first, "second": second})

    search = combined.searcher()
    structure = search.variable(OptimadeStructure)
    search.add(structure.elements.has("Li"))
    rows = search.results(record=structure, origin=search.origin)
    for row in rows:
        print(row.origin, row.record.id)
```

Only the strict intersection of source query support is portable: a source
failure or unsupported field raises rather than returning a partial result.
The federation is a union, not a deduplicating merge, so equal resource IDs
from different origins remain distinct. It pages sources sequentially in
constructor (source-major) order; offsets and limits apply globally after that
union. `count()` and `len(rows)` require exact child counts and can raise
`CountUnavailableError`; federation does not crawl pages to approximate them.
Global sorting is unsupported. When endpoint descriptors differ or a typed
backend is ambiguous at one endpoint, create an explicit per-source target with
`combined.target(...)` and each store's `entry_type(...)` descriptor.

### Response fields and exact literals

By default a whole-record query sends no `response_fields` parameter, so the
service chooses its normal response fields. Set
`response_fields` on `OptimadeStore` or on `store.searcher()` to request a
semantic field-name iterable for a typed endpoint. Use the identity sentinel
`ALL_ADVERTISED` to request every advertised field; generic endpoints accept
their exact advertised transport names for an explicit selection.

Adding a scalar projection to a whole-record result keeps that source record at
least as complete as the endpoint's advertised `response-default` fields and
makes the scalar explicit when necessary. If an endpoint does not advertise
that metadata, the conservative fallback requests every advertised field rather
than a thin scalar-only record. Record-only queries retain the literal
no-parameter default policy.

Numeric query literals are exact by policy. Use integers or `Decimal`; a
`Fraction` is accepted only when it has a finite decimal expansion. Binary
`float`, non-finite `Decimal`, and fractions such as `Fraction(1, 3)` are
rejected rather than silently approximated.

## Pagination, lifecycle, and errors

The client follows `links.next` synchronously while detecting cycles,
enforcing `max_pages`, and rejecting cross-origin continuations unless
`allow_cross_origin_pagination=True` was explicitly selected. A response that
claims `more_data_available` without a usable continuation raises
`OptimadePaginationError`.

When no client is supplied, `OptimadeStore` owns an `httpx.Client` and closes
it on `close()` or context-manager exit, including when `/versions` negotiation
or later discovery fails. An injected client is borrowed and is never closed.
`refresh()` reuses the already negotiated effective base URL and never repeats
`/versions`. Requests and `refresh()` after close raise `OptimadeClientError`.
Relevant failures are `OptimadeTransportError`, `OptimadeHTTPError`,
`OptimadeErrorDocumentError`, `OptimadeDiscoveryError`, `OptimadeResponseError`,
`OptimadeVersionNegotiationError`, `OptimadePaginationError`, and
`CountUnavailableError`; their diagnostics omit
credentials and sensitive query tokens.

Before a result page yields any item, the client verifies its JSON:API/OPTIMADE
envelope: object `meta`, an optional but (when present) nonnegative integer
`meta.data_returned` (the filtered total, independent of this page's length),
endpoint-matched resource types, and object-valued
`attributes`/`relationships` members when present. This
prevents a partially yielded typed page from being followed by a malformed
entry in the same response.

## Offline caching is explicit

Legacy services that predate property-definition IRIs remain generic: select
their exact `RemoteEntryType` descriptor and use its advertised transport field
names for queries. Results remain `OptimadeResource` objects rather than being
silently assigned a typed backend.

Remote reads never write local state. To retain a resource for offline work,
opt into the database capability and save the exact object yourself:

```python
from httk.store import Backend, SqlStore

cache = SqlStore(Backend.sqlite("optimade-cache.sqlite"), entry_records={})
backend = rows.one().reference  # a typed backend from a prior result
sid = cache.save(backend)
offline = cache.fetch(type(backend), sid)
raw_resource = offline.unwrap()
```

`OptimadeResource` carries a `member` field (`"data"` for a primary resource,
`"included"` for one resolved through `.links.<name>`) that participates in
its identity, so an offline cache created before that field existed uses an
incompatible storage layout and must be recreated; the store rejects the
old layout explicitly rather than silently reinterpreting it. Save the
resource directly -- a bound object saves and reads back as its plain class.

This reconstructs the same typed backend class and its exact raw resource.
`SqlStore.save()` deduplicates shared whole-page documents and schema snapshots
while retaining the resource index, so several resources from one response do
not duplicate their common source context. Saving `backend.unwrap()` directly
is also valid when only a generic `OptimadeResource` cache is wanted. These are
local cache operations, never OPTIMADE writeback.