httk.core.datastream

Submodules

Attributes

Classes

BytestreamBackend

Abstract base class for all backends of streaming byte data.

BytestreamBytes

Backend for streaming byte data backed by an actual bytes object.

BytestreamBytesView

A view presenting underlying streaming byte data as bytes.

BytestreamCommon

Common superclass for many of the implementations of backends for streaming byte data.

BytestreamFile

Backend for file-based (io.IOBase-conforming) streaming byte data.

BytestreamFileView

A view presenting an underlying data streaming backend via an io.IOBase-like API.

BytestreamFilename

Backend for streaming byte data via operations on a file specified by a filename.

BytestreamFilenameView

A view presenting an underlying data streaming backend via a filename.

BytestreamRequest

Backend for streaming byte data fetched via a urllib.request.Request.

BytestreamRequestView

A view presenting an underlying data streaming backend via a urllib.request.Request.

BytestreamURL

Backend for streaming byte data fetched from a URL string.

BytestreamURLView

A view presenting an underlying data streaming backend via a URL string.

BytestreamView

Abstract base class for all views of streaming byte data.

CompressionCodec

A decompression codec for a single container format.

DatastreamURL

An explicit network-consent value carrying a URL and optional timeout for lazy consumers, which resolve it through the existing

TextstreamBackend

Abstract base class for all backends of streaming text data.

TextstreamCommon

Common superclass for many of the implementations of backends for streaming text data.

TextstreamFile

Backend for file-based (io.TextIOBase-conforming) streaming text data

TextstreamFileView

A view presenting an underlying data streaming backend via the full io.TextIOBase API, which is a superset of TextstreamAPI.

TextstreamFilename

Backend for streaming text via operations on a file specfied by a filename

TextstreamFilenameView

A view presenting an underlying data streaming backend via a filename.

TextstreamRequest

Backend for streaming text fetched via a urllib.request.Request.

TextstreamRequestView

A view presenting an underlying data streaming backend via a urllib.request.Request.

TextstreamString

Backend for streaming text backed by an actual string

TextstreamStringView

A view presenting an underlying data streaming as a string.

TextstreamURL

Backend for streaming text fetched from a URL string.

TextstreamURLView

A view presenting an underlying data streaming backend via a URL string.

TextstreamView

Abstract base class for all views of streaming text data.

Functions

known_compressions()

Return the registered codec names, in registration order.

register_compression(codec)

Register (or replace) a codec under its name (case-insensitive).

Package Contents

class httk.core.datastream.BytestreamBackend(backend, **hints)[source]

Bases: httk.core.views.Backend[BytestreamBackend], httk.core.datastream.bytestream_api.BytestreamAPI

Abstract base class for all backends of streaming byte data.

Parameters:
  • backend (Any) – Initial backend value accepted by the backend family.

  • **hints (Any) – Backend-selection hints used by concrete backends.

backend_classes: ClassVar[list[type[httk.core.views.Backend[Any]]]]
class httk.core.datastream.BytestreamBytes(content, **hints)[source]

Bases: httk.core.datastream.bytestream_common.BytestreamCommon, httk.core.datastream.bytestream_backend.BytestreamBackend

Backend for streaming byte data backed by an actual bytes object. Compressed content is transparently decompressed according to the compression hint.

Parameters:
  • content (bytes | bytearray) – Bytes to expose as a stream.

  • **hints (Any) – Backend-selection and compression hints.

Raises:

ValueError – If the compression hint is unknown.

b: bytes
property name: str | None

Report that an in-memory stream has no source name.

property closed: bool

Report whether the in-memory stream is closed.

class httk.core.datastream.BytestreamBytesView(obj, **hints)[source]

Bases: httk.core.datastream.bytestream_view.BytestreamView, bytes

A view presenting underlying streaming byte data as bytes. This view can be used both to pass bytes in place of streaming data, and for reading streaming data into bytes. Note: this view is not lazy (this is impossible for views inheriting bytes, since bytes is immutable), hence all streaming data is read immediately upon creating this view.

Parameters:
unwrap()[source]

Return the raw representation of the wrapped backend.

Returns:

The backend’s most raw available representation.

Return type:

Any

unview()[source]

Return the presented data as plain bytes.

Returns:

A plain bytes value containing the presented data.

Return type:

bytes

class httk.core.datastream.BytestreamCommon[source]

Bases: abc.ABC

Common superclass for many of the implementations of backends for streaming byte data.

unwrap()[source]

Return the currently opened underlying stream.

Returns:

The stream used for reading and writing data.

Return type:

io.IOBase

read(size=-1)[source]

Read up to size bytes, or all remaining bytes when size is negative.

Parameters:

size (int) – Maximum number of bytes to read.

Returns:

The bytes read from the stream.

Return type:

bytes

close()[source]

Close the opened stream and any source stream owned by it.

seek(offset, whence=os.SEEK_SET)[source]

Move the stream position.

Parameters:
  • offset (int) – Position adjustment interpreted according to whence.

  • whence (int) – Reference point for offset.

Returns:

The resulting absolute stream position.

Return type:

int

tell()[source]

Return the current stream position.

Returns:

The absolute stream position.

Return type:

int

class httk.core.datastream.BytestreamFile(obj, **hints)[source]

Bases: httk.core.datastream.bytestream_common.BytestreamCommon, httk.core.datastream.bytestream_backend.BytestreamBackend

Backend for file-based (io.IOBase-conforming) streaming byte data. Compressed content is transparently decompressed according to the compression hint.

Parameters:
  • obj (io.IOBase) – Open binary stream to adopt and close with this backend.

  • **hints (Any) – Backend-selection and compression hints.

Raises:

ValueError – If the compression hint is unknown.

property name: str | None

Return the adopted stream’s name when it provides one.

property closed: bool

Report whether the adopted stream is closed.

class httk.core.datastream.BytestreamFileView(obj, **hints)[source]

Bases: httk.core.datastream.bytestream_view.BytestreamView, io.IOBase, httk.core.datastream.bytestream_api.BytestreamAPI

A view presenting an underlying data streaming backend via an io.IOBase-like API.

Parameters:
unwrap()[source]

Return the raw representation of the wrapped backend.

Returns:

The backend’s most raw available representation.

Return type:

Any

property name: str | None

Return the backend’s source name when one exists.

property closed: bool

Report whether the backend is closed.

close()[source]

Close the underlying backend.

readable()[source]

Report that the view supports reading.

Returns:

Always True.

Return type:

bool

writable()[source]

Report that the view does not support writing.

Returns:

Always False.

Return type:

bool

seekable()[source]

Report whether the backend supports seeking and telling.

Returns:

Whether both operations are available.

Return type:

bool

flush()[source]

Flush the backend when it provides flushing.

Raises:

ValueError – If the view is closed.

read(size=-1)[source]

Read up to size bytes, or all remaining bytes when size is negative.

Parameters:

size (int | None) – Maximum number of bytes to read; None also means all remaining bytes.

Returns:

The bytes read from the stream.

Raises:

ValueError – If the view is closed.

Return type:

bytes

readline(size=-1)[source]

Read one line, optionally limited to size bytes.

Parameters:

size (int | None) – Maximum number of bytes to read; None means no limit.

Returns:

The line read, including its newline when present.

Raises:

ValueError – If the view is closed.

Return type:

bytes

readlines(hint=-1)[source]

Read lines until EOF or until the accumulated size reaches hint.

Parameters:

hint (int) – Approximate minimum number of bytes to collect, or a negative value for no limit.

Returns:

The lines read from the stream.

Raises:

ValueError – If the view is closed.

Return type:

list[bytes]

seek(offset, whence=io.SEEK_SET)[source]

Move the stream position and discard buffered line data.

Parameters:
  • offset (int) – Position adjustment interpreted according to whence.

  • whence (int) – Reference point for offset.

Returns:

The resulting absolute stream position.

Raises:
Return type:

int

tell()[source]

Return the logical stream position before buffered line data.

Returns:

The logical absolute stream position.

Raises:
Return type:

int

detach()[source]

Reject detaching because the view owns its backend interface.

Raises:

io.UnsupportedOperation – Always, because detaching is unsupported.

class httk.core.datastream.BytestreamFilename(filename, **hints)[source]

Bases: httk.core.datastream.bytestream_common.BytestreamCommon, httk.core.datastream.bytestream_backend.BytestreamBackend

Backend for streaming byte data via operations on a file specified by a filename. Compressed content is transparently decompressed according to the compression hint.

Parameters:
  • filename (str | pathlib.Path) – File path to open lazily for binary reading.

  • **hints (Any) – Backend-selection and compression hints.

Raises:

ValueError – If the compression hint is unknown.

property name: str | None

Return the configured filename.

property closed: bool

Report whether the filename backend is closed.

class httk.core.datastream.BytestreamFilenameView(obj, **hints)[source]

Bases: httk.core.datastream.bytestream_view.BytestreamView, str

A view presenting an underlying data streaming backend via a filename. This view is mostly useful for providing filenames to functions that will open them. Note: this view is not lazy (this is impossible for views inheriting str, since str is immutable).

Raises TypeError if created with a streaming data source that does not come with a name.

Parameters:
Raises:

TypeError – If the source has no filename.

unwrap()[source]

Return the raw representation of the wrapped backend.

Returns:

The backend’s most raw available representation.

Return type:

Any

unview()[source]

Return the presented filename as a plain string.

Returns:

The filename represented by this view.

Return type:

str

type httk.core.datastream.BytestreamLike = bytestream_backend.BytestreamBackend | bytestream_view.BytestreamView | io.IOBase | io.BytesIO | bytes | bytearray | str | pathlib.Path | urllib.request.Request[source]
class httk.core.datastream.BytestreamRequest(request, **hints)[source]

Bases: httk.core.datastream.bytestream_common.BytestreamCommon, httk.core.datastream.bytestream_backend.BytestreamBackend

Backend for streaming byte data fetched via a urllib.request.Request. Response content is transparently decompressed according to the compression hint.

Parameters:
  • request (urllib.request.Request) – Request to execute lazily when data is first read.

  • **hints (Any) – Backend-selection, timeout, and compression hints.

Raises:

ValueError – If the compression hint is unknown.

property name: str | None

Report that a request backend has no filename.

property url: str

Return the request URL.

property request: urllib.request.Request

Return the request used to fetch the stream.

property closed: bool

Report whether the request backend is closed.

class httk.core.datastream.BytestreamRequestView(obj, **hints)[source]

Bases: httk.core.datastream.bytestream_view.BytestreamView, urllib.request.Request

A view presenting an underlying data streaming backend via a urllib.request.Request. This view is mostly useful for providing a Request to functions that will open it. Note: this view is not lazy (it does not fetch); it only mirrors the underlying request/URL.

Raises TypeError if created with a streaming data source that does not come with a URL.

Parameters:
Raises:

TypeError – If the source has no URL.

unwrap()[source]

Return the raw representation of the wrapped backend.

Returns:

The backend’s most raw available representation.

Return type:

Any

class httk.core.datastream.BytestreamURL(url, **hints)[source]

Bases: httk.core.datastream.bytestream_common.BytestreamCommon, httk.core.datastream.bytestream_backend.BytestreamBackend

Backend for streaming byte data fetched from a URL string. A bare string is interpreted as a URL when its scheme is one of http, https, ftp, or file, or when an explicit kind=”url” hint is given. Network access from an implicit bare network URL requires explicit consent before opening; URL views and kind="url" provide that consent. Content is transparently decompressed according to the compression hint.

Parameters:
  • url (str) – URL to fetch lazily when data is first read.

  • **hints (Any) – Backend-selection, consent, timeout, and compression hints.

Raises:

ValueError – If the compression hint is unknown.

property name: str | None

Report that a URL backend has no filename.

property url: str

Return the source URL.

property closed: bool

Report whether the URL backend is closed.

class httk.core.datastream.BytestreamURLView(obj, **hints)[source]

Bases: httk.core.datastream.bytestream_view.BytestreamView, str

A view presenting an underlying data streaming backend via a URL string. This view is mostly useful for providing a URL to functions that will open it. Note: this view is not lazy (this is impossible for views inheriting str, since str is immutable).

Raises TypeError if created with a streaming data source that does not come with a URL.

Parameters:
Raises:

TypeError – If the source has no URL.

unwrap()[source]

Return the raw representation of the wrapped backend.

Returns:

The backend’s most raw available representation.

Return type:

Any

class httk.core.datastream.BytestreamView[source]

Bases: httk.core.views.View[httk.core.datastream.bytestream_backend.BytestreamBackend]

Abstract base class for all views of streaming byte data.

Views retain the backend and expose it through a byte-stream interface.

class httk.core.datastream.CompressionCodec[source]

A decompression codec for a single container format.

A codec is an orthogonal layer below the datastream backends: it turns a compressed binary stream into an uncompressed binary stream, independently of where the compressed bytes come from (a filename, an open file, raw bytes, or a remote response).

Parameters:
  • name – Canonical name used to select the codec explicitly.

  • extensions – Filename suffixes that identify the codec.

  • magics – Leading byte signatures used to detect the codec.

  • open_stream – Function that wraps compressed bytes for reading.

name: str

Canonical, lower-case codec name (e.g. "gzip"); also how an explicit hint selects it.

extensions: tuple[str, Ellipsis]

Recognized filename suffixes including the leading dot (e.g. (".gz",)).

magics: tuple[bytes, Ellipsis]

Leading magic-byte signatures; an empty tuple means the format cannot be sniffed.

open_stream: collections.abc.Callable[[io.IOBase], io.IOBase]

Wrap a compressed binary stream and return a readable, decompressed binary stream.

httk.core.datastream.known_compressions()[source]

Return the registered codec names, in registration order.

Returns:

The registered codec names.

Return type:

list[str]

httk.core.datastream.register_compression(codec)[source]

Register (or replace) a codec under its name (case-insensitive).

Parameters:

codec (CompressionCodec) – Codec to add to the registry.

class httk.core.datastream.DatastreamURL(url, *, timeout=None)[source]

An explicit network-consent value carrying a URL and optional timeout for lazy consumers, which resolve it through the existing fetch/loader machinery; constructing it performs no network I/O.

Parameters:
  • url (str) – URL whose explicit use permits network-backed lazy access.

  • timeout (float | None) – Optional timeout to apply when the URL is opened.

Raises:

ValueError – If the URL uses an unsupported scheme.

property url: str

Return the explicit URL.

property timeout: float | None

Return the timeout to use when opening the URL.

class httk.core.datastream.TextstreamBackend(backend, **hints)[source]

Bases: httk.core.views.Backend[TextstreamBackend], httk.core.datastream.textstream_api.TextstreamAPI

Abstract base class for all backends of streaming text data.

Parameters:
  • backend (Any) – Initial backend value accepted by the backend family.

  • **hints (Any) – Backend-selection hints used by concrete backends.

backend_classes: ClassVar[list[type[httk.core.views.Backend[Any]]]]
class httk.core.datastream.TextstreamCommon[source]

Bases: abc.ABC

Common superclass for many of the implementations of backends for streaming text data.

unwrap()[source]

Return the currently opened underlying text stream.

Returns:

The stream used for reading text.

Return type:

io.TextIOBase

read(size=-1)[source]

Read up to size characters, or all remaining characters when size is negative.

Parameters:

size (int) – Maximum number of characters to read.

Returns:

The text read from the stream.

Return type:

str

close()[source]

Close the opened stream and any source stream owned by it.

seek(offset, whence=os.SEEK_SET)[source]

Move the stream position.

Parameters:
  • offset (int) – Position adjustment interpreted according to whence.

  • whence (int) – Reference point for offset.

Returns:

The resulting absolute stream position.

Return type:

int

tell()[source]

Return the current stream position.

Returns:

The absolute stream position.

Return type:

int

class httk.core.datastream.TextstreamFile(obj, **hints)[source]

Bases: httk.core.datastream.textstream_common.TextstreamCommon, httk.core.datastream.textstream_backend.TextstreamBackend

Backend for file-based (io.TextIOBase-conforming) streaming text data

Parameters:
  • obj (io.TextIOBase) – Open text stream to adopt and close with this backend.

  • **hints (Any) – Backend-selection and compression hints.

Raises:

ValueError – If the compression hint is not a no-op mode for text-native content.

property name: str | None

Return the adopted stream’s name when it provides one.

property closed: bool

Report whether the adopted stream is closed.

class httk.core.datastream.TextstreamFileView(obj, **hints)[source]

Bases: httk.core.datastream.textstream_view.TextstreamView, io.TextIOBase, httk.core.datastream.textstream_api.TextstreamAPI

A view presenting an underlying data streaming backend via the full io.TextIOBase API, which is a superset of TextstreamAPI.

Parameters:
unwrap()[source]

Return the raw representation of the wrapped backend.

Returns:

The backend’s most raw available representation.

Return type:

Any

property name: str | None

Return the backend’s source name when one exists.

property closed: bool

Report whether the backend is closed.

close()[source]

Close the underlying backend.

readable()[source]

Report that the view supports reading.

Returns:

Always True.

Return type:

bool

writable()[source]

Report that the view does not support writing.

Returns:

Always False.

Return type:

bool

seekable()[source]

Report whether the backend supports seeking and telling.

Returns:

Whether both operations are available.

Return type:

bool

flush()[source]

Flush the backend when it provides flushing.

Raises:

ValueError – If the view is closed.

read(size=-1)[source]

Read up to size characters, or all remaining characters when size is negative.

Parameters:

size (int | None) – Maximum number of characters to read; None also means all remaining characters.

Returns:

The text read from the stream.

Raises:

ValueError – If the view is closed.

Return type:

str

readline(size=-1)[source]

Read one line, optionally limited to size characters.

Parameters:

size (int | None) – Maximum number of characters to read; None means no limit.

Returns:

The line read, including its newline when present.

Raises:

ValueError – If the view is closed.

Return type:

str

readlines(hint=-1)[source]

Read lines until EOF or until the accumulated size reaches hint.

Parameters:

hint (int) – Approximate minimum number of characters to collect, or a negative value for no limit.

Returns:

The lines read from the stream.

Raises:

ValueError – If the view is closed.

Return type:

list[str]

seek(offset, whence=io.SEEK_SET)[source]

Move the stream position and discard buffered line data.

Parameters:
  • offset (int) – Position adjustment interpreted according to whence.

  • whence (int) – Reference point for offset.

Returns:

The resulting absolute stream position.

Raises:
Return type:

int

tell()[source]

Return the logical stream position before buffered line data.

Returns:

The logical absolute stream position.

Raises:
Return type:

int

detach()[source]

Reject detaching because the view owns its backend interface.

Raises:

io.UnsupportedOperation – Always, because detaching is unsupported.

property encoding: str | None

Return the backend’s text encoding when available.

property errors: str | None

Return the backend’s error handling mode when available.

property newlines: str | tuple[str, Ellipsis] | None

Return the newline conventions observed by the backend when available.

class httk.core.datastream.TextstreamFilename(filename, **hints)[source]

Bases: httk.core.datastream.textstream_common.TextstreamCommon, httk.core.datastream.textstream_backend.TextstreamBackend

Backend for streaming text via operations on a file specfied by a filename Compressed content is transparently decompressed before text decoding.

Parameters:
  • filename (str | pathlib.Path) – File path to open lazily for text reading.

  • **hints (Any) – Backend-selection, encoding, and compression hints.

Raises:

ValueError – If the compression hint is unknown.

property name: str | None

Return the configured filename.

property closed: bool

Report whether the filename backend is closed.

class httk.core.datastream.TextstreamFilenameView(obj, **hints)[source]

Bases: httk.core.datastream.textstream_view.TextstreamView, str

A view presenting an underlying data streaming backend via a filename. This view is probably mostly useful for providing filenames to functions that will open them. Note: this view is not lazy (this is impossible for views inherting str, since str is immutable)

Raises TypeError if created with a streaming data source that does not have a name.

Parameters:
Raises:

TypeError – If the source has no filename.

unwrap()[source]

Return the raw representation of the wrapped backend.

Returns:

The backend’s most raw available representation.

Return type:

Any

unview()[source]

Return the presented filename as a plain string.

Returns:

The filename represented by this view.

Return type:

str

type httk.core.datastream.TextstreamLike = textstream_backend.TextstreamBackend | textstream_view.TextstreamView | io.TextIOBase | io.StringIO | str | pathlib.Path | urllib.request.Request[source]
class httk.core.datastream.TextstreamRequest(request, **hints)[source]

Bases: httk.core.datastream.textstream_common.TextstreamCommon, httk.core.datastream.textstream_backend.TextstreamBackend

Backend for streaming text fetched via a urllib.request.Request. Response content is transparently decompressed before text decoding.

Parameters:
  • request (urllib.request.Request) – Request to execute lazily when text is first read.

  • **hints (Any) – Backend-selection, encoding, timeout, and compression hints.

Raises:

ValueError – If the compression hint is unknown.

property name: str | None

Report that a request backend has no filename.

property url: str

Return the request URL.

property request: urllib.request.Request

Return the request used to fetch the stream.

property closed: bool

Report whether the request backend is closed.

class httk.core.datastream.TextstreamRequestView(obj, **hints)[source]

Bases: httk.core.datastream.textstream_view.TextstreamView, urllib.request.Request

A view presenting an underlying data streaming backend via a urllib.request.Request. This view is mostly useful for providing a Request to functions that will open it. Note: this view is not lazy (it does not fetch); it only mirrors the underlying request/URL.

Raises TypeError if created with a streaming data source that does not come with a URL.

Parameters:
Raises:

TypeError – If the source has no URL.

unwrap()[source]

Return the raw representation of the wrapped backend.

Returns:

The backend’s most raw available representation.

Return type:

Any

class httk.core.datastream.TextstreamString(content, **hints)[source]

Bases: httk.core.datastream.textstream_common.TextstreamCommon, httk.core.datastream.textstream_backend.TextstreamBackend

Backend for streaming text backed by an actual string

Parameters:
  • content (str) – Text to expose as a stream.

  • **hints (Any) – Backend-selection and compression hints.

Raises:

ValueError – If the compression hint is not a no-op mode for text-native content.

s: str
property name: str | None

Report that an in-memory stream has no source name.

property closed: bool

Report whether the in-memory stream is closed.

class httk.core.datastream.TextstreamStringView(obj, **hints)[source]

Bases: httk.core.datastream.textstream_view.TextstreamView, str

A view presenting an underlying data streaming as a string. This view can be used both to pass a string in place of streaming data, and for reading streaming data into a string. Note: this view is not lazy (this is impossible for views inherting str, since str is immutable), hence all the streaming data is read immedately upon creating this view.

Parameters:
unwrap()[source]

Return the raw representation of the wrapped backend.

Returns:

The backend’s most raw available representation.

Return type:

Any

unview()[source]

Return the presented text as a plain string.

Returns:

A plain string containing the presented text.

Return type:

str

class httk.core.datastream.TextstreamURL(url, **hints)[source]

Bases: httk.core.datastream.textstream_common.TextstreamCommon, httk.core.datastream.textstream_backend.TextstreamBackend

Backend for streaming text fetched from a URL string. A bare string is interpreted as a URL when its scheme is one of http, https, ftp, or file, or when an explicit kind=”url” hint is given. Network access from an implicit bare network URL requires explicit consent before opening; URL views and kind="url" provide that consent. Content is transparently decompressed before text decoding.

Parameters:
  • url (str) – URL to fetch lazily when text is first read.

  • **hints (Any) – Backend-selection, consent, encoding, timeout, and compression hints.

Raises:

ValueError – If the compression hint is unknown.

property name: str | None

Report that a URL backend has no filename.

property url: str

Return the source URL.

property closed: bool

Report whether the URL backend is closed.

class httk.core.datastream.TextstreamURLView(obj, **hints)[source]

Bases: httk.core.datastream.textstream_view.TextstreamView, str

A view presenting an underlying data streaming backend via a URL string. This view is mostly useful for providing a URL to functions that will open it. Note: this view is not lazy (this is impossible for views inheriting str, since str is immutable).

Raises TypeError if created with a streaming data source that does not come with a URL.

Parameters:
Raises:

TypeError – If the source has no URL.

unwrap()[source]

Return the raw representation of the wrapped backend.

Returns:

The backend’s most raw available representation.

Return type:

Any

class httk.core.datastream.TextstreamView[source]

Bases: httk.core.views.View[httk.core.datastream.textstream_backend.TextstreamBackend]

Abstract base class for all views of streaming text data.

Views retain the backend and expose it through a text-stream interface.

type httk.core.datastream.DatastreamLike = BytestreamLike | TextstreamLike | DatastreamURL[source]