httk.core.datastream¶
Submodules¶
- httk.core.datastream.bytestream_api
- httk.core.datastream.bytestream_backend
- httk.core.datastream.bytestream_bytes
- httk.core.datastream.bytestream_bytes_view
- httk.core.datastream.bytestream_common
- httk.core.datastream.bytestream_file
- httk.core.datastream.bytestream_file_view
- httk.core.datastream.bytestream_filename
- httk.core.datastream.bytestream_filename_view
- httk.core.datastream.bytestream_like
- httk.core.datastream.bytestream_request
- httk.core.datastream.bytestream_request_view
- httk.core.datastream.bytestream_url
- httk.core.datastream.bytestream_url_view
- httk.core.datastream.bytestream_view
- httk.core.datastream.compression
- httk.core.datastream.datastream_url
- httk.core.datastream.network_policy
- httk.core.datastream.textstream_api
- httk.core.datastream.textstream_backend
- httk.core.datastream.textstream_common
- httk.core.datastream.textstream_file
- httk.core.datastream.textstream_file_view
- httk.core.datastream.textstream_filename
- httk.core.datastream.textstream_filename_view
- httk.core.datastream.textstream_like
- httk.core.datastream.textstream_request
- httk.core.datastream.textstream_request_view
- httk.core.datastream.textstream_string
- httk.core.datastream.textstream_string_view
- httk.core.datastream.textstream_url
- httk.core.datastream.textstream_url_view
- httk.core.datastream.textstream_view
Attributes¶
Classes¶
Abstract base class for all backends of streaming byte data. |
|
Backend for streaming byte data backed by an actual bytes object. |
|
A view presenting underlying streaming byte data as bytes. |
|
Common superclass for many of the implementations of backends for streaming byte data. |
|
Backend for file-based (io.IOBase-conforming) streaming byte data. |
|
A view presenting an underlying data streaming backend via an io.IOBase-like API. |
|
Backend for streaming byte data via operations on a file specified by a filename. |
|
A view presenting an underlying data streaming backend via a filename. |
|
Backend for streaming byte data fetched via a urllib.request.Request. |
|
A view presenting an underlying data streaming backend via a urllib.request.Request. |
|
Backend for streaming byte data fetched from a URL string. |
|
A view presenting an underlying data streaming backend via a URL string. |
|
Abstract base class for all views of streaming byte data. |
|
A decompression codec for a single container format. |
|
An explicit network-consent value carrying a URL and optional timeout for lazy consumers, which resolve it through the existing |
|
Abstract base class for all backends of streaming text data. |
|
Common superclass for many of the implementations of backends for streaming text data. |
|
Backend for file-based (io.TextIOBase-conforming) streaming text data |
|
A view presenting an underlying data streaming backend via the full io.TextIOBase API, which is a superset of TextstreamAPI. |
|
Backend for streaming text via operations on a file specfied by a filename |
|
A view presenting an underlying data streaming backend via a filename. |
|
Backend for streaming text fetched via a urllib.request.Request. |
|
A view presenting an underlying data streaming backend via a urllib.request.Request. |
|
Backend for streaming text backed by an actual string |
|
A view presenting an underlying data streaming as a string. |
|
Backend for streaming text fetched from a URL string. |
|
A view presenting an underlying data streaming backend via a URL string. |
|
Abstract base class for all views of streaming text data. |
Functions¶
Return the registered codec names, in registration order. |
|
|
Register (or replace) a codec under its |
Package Contents¶
- class httk.core.datastream.BytestreamBackend(backend, **hints)[source]¶
Bases:
httk.core.views.Backend[BytestreamBackend],httk.core.datastream.bytestream_api.BytestreamAPIAbstract base class for all backends of streaming byte data.
- Parameters:
backend (Any) – Initial backend value accepted by the backend family.
**hints (Any) – Backend-selection hints used by concrete backends.
- backend_classes: ClassVar[list[type[httk.core.views.Backend[Any]]]]¶
- class httk.core.datastream.BytestreamBytes(content, **hints)[source]¶
Bases:
httk.core.datastream.bytestream_common.BytestreamCommon,httk.core.datastream.bytestream_backend.BytestreamBackendBackend for streaming byte data backed by an actual bytes object. Compressed content is transparently decompressed according to the compression hint.
- Parameters:
- Raises:
ValueError – If the compression hint is unknown.
- class httk.core.datastream.BytestreamBytesView(obj, **hints)[source]¶
Bases:
httk.core.datastream.bytestream_view.BytestreamView,bytesA view presenting underlying streaming byte data as bytes. This view can be used both to pass bytes in place of streaming data, and for reading streaming data into bytes. Note: this view is not lazy (this is impossible for views inheriting bytes, since bytes is immutable), hence all streaming data is read immediately upon creating this view.
- Parameters:
obj (httk.core.datastream.bytestream_like.BytestreamLike) – Byte-stream source to present or consume.
**hints (Any) – Backend-selection and compression hints.
- class httk.core.datastream.BytestreamCommon[source]¶
Bases:
abc.ABCCommon superclass for many of the implementations of backends for streaming byte data.
- unwrap()[source]¶
Return the currently opened underlying stream.
- Returns:
The stream used for reading and writing data.
- Return type:
- class httk.core.datastream.BytestreamFile(obj, **hints)[source]¶
Bases:
httk.core.datastream.bytestream_common.BytestreamCommon,httk.core.datastream.bytestream_backend.BytestreamBackendBackend for file-based (io.IOBase-conforming) streaming byte data. Compressed content is transparently decompressed according to the compression hint.
- Parameters:
obj (io.IOBase) – Open binary stream to adopt and close with this backend.
**hints (Any) – Backend-selection and compression hints.
- Raises:
ValueError – If the compression hint is unknown.
- class httk.core.datastream.BytestreamFileView(obj, **hints)[source]¶
Bases:
httk.core.datastream.bytestream_view.BytestreamView,io.IOBase,httk.core.datastream.bytestream_api.BytestreamAPIA view presenting an underlying data streaming backend via an io.IOBase-like API.
- Parameters:
obj (httk.core.datastream.bytestream_like.BytestreamLike) – Byte-stream source to present through the file API.
**hints (Any) – Backend-selection and compression hints.
- unwrap()[source]¶
Return the raw representation of the wrapped backend.
- Returns:
The backend’s most raw available representation.
- Return type:
Any
- writable()[source]¶
Report that the view does not support writing.
- Returns:
Always
False.- Return type:
- seekable()[source]¶
Report whether the backend supports seeking and telling.
- Returns:
Whether both operations are available.
- Return type:
- flush()[source]¶
Flush the backend when it provides flushing.
- Raises:
ValueError – If the view is closed.
- read(size=-1)[source]¶
Read up to
sizebytes, or all remaining bytes whensizeis negative.- Parameters:
size (int | None) – Maximum number of bytes to read;
Nonealso means all remaining bytes.- Returns:
The bytes read from the stream.
- Raises:
ValueError – If the view is closed.
- Return type:
- readline(size=-1)[source]¶
Read one line, optionally limited to
sizebytes.- Parameters:
size (int | None) – Maximum number of bytes to read;
Nonemeans no limit.- Returns:
The line read, including its newline when present.
- Raises:
ValueError – If the view is closed.
- Return type:
- readlines(hint=-1)[source]¶
Read lines until EOF or until the accumulated size reaches
hint.- Parameters:
hint (int) – Approximate minimum number of bytes to collect, or a negative value for no limit.
- Returns:
The lines read from the stream.
- Raises:
ValueError – If the view is closed.
- Return type:
- seek(offset, whence=io.SEEK_SET)[source]¶
Move the stream position and discard buffered line data.
- Parameters:
- Returns:
The resulting absolute stream position.
- Raises:
ValueError – If the view is closed.
io.UnsupportedOperation – If the backend is not seekable.
- Return type:
- tell()[source]¶
Return the logical stream position before buffered line data.
- Returns:
The logical absolute stream position.
- Raises:
ValueError – If the view is closed.
io.UnsupportedOperation – If the backend does not support telling.
- Return type:
- detach()[source]¶
Reject detaching because the view owns its backend interface.
- Raises:
io.UnsupportedOperation – Always, because detaching is unsupported.
- class httk.core.datastream.BytestreamFilename(filename, **hints)[source]¶
Bases:
httk.core.datastream.bytestream_common.BytestreamCommon,httk.core.datastream.bytestream_backend.BytestreamBackendBackend for streaming byte data via operations on a file specified by a filename. Compressed content is transparently decompressed according to the compression hint.
- Parameters:
filename (str | pathlib.Path) – File path to open lazily for binary reading.
**hints (Any) – Backend-selection and compression hints.
- Raises:
ValueError – If the compression hint is unknown.
- class httk.core.datastream.BytestreamFilenameView(obj, **hints)[source]¶
Bases:
httk.core.datastream.bytestream_view.BytestreamView,strA view presenting an underlying data streaming backend via a filename. This view is mostly useful for providing filenames to functions that will open them. Note: this view is not lazy (this is impossible for views inheriting str, since str is immutable).
Raises TypeError if created with a streaming data source that does not come with a name.
- Parameters:
obj (httk.core.datastream.bytestream_like.BytestreamLike) – Byte-stream source whose filename should be presented.
**hints (Any) – Backend-selection and compression hints.
- Raises:
TypeError – If the source has no filename.
- type httk.core.datastream.BytestreamLike = bytestream_backend.BytestreamBackend | bytestream_view.BytestreamView | io.IOBase | io.BytesIO | bytes | bytearray | str | pathlib.Path | urllib.request.Request[source]¶
- class httk.core.datastream.BytestreamRequest(request, **hints)[source]¶
Bases:
httk.core.datastream.bytestream_common.BytestreamCommon,httk.core.datastream.bytestream_backend.BytestreamBackendBackend for streaming byte data fetched via a urllib.request.Request. Response content is transparently decompressed according to the compression hint.
- Parameters:
request (urllib.request.Request) – Request to execute lazily when data is first read.
**hints (Any) – Backend-selection, timeout, and compression hints.
- Raises:
ValueError – If the compression hint is unknown.
- property request: urllib.request.Request¶
Return the request used to fetch the stream.
- class httk.core.datastream.BytestreamRequestView(obj, **hints)[source]¶
Bases:
httk.core.datastream.bytestream_view.BytestreamView,urllib.request.RequestA view presenting an underlying data streaming backend via a urllib.request.Request. This view is mostly useful for providing a Request to functions that will open it. Note: this view is not lazy (it does not fetch); it only mirrors the underlying request/URL.
Raises TypeError if created with a streaming data source that does not come with a URL.
- Parameters:
obj (httk.core.datastream.bytestream_like.BytestreamLike) – Byte-stream source to present as a request.
**hints (Any) – Backend-selection and compression hints.
- Raises:
TypeError – If the source has no URL.
- class httk.core.datastream.BytestreamURL(url, **hints)[source]¶
Bases:
httk.core.datastream.bytestream_common.BytestreamCommon,httk.core.datastream.bytestream_backend.BytestreamBackendBackend for streaming byte data fetched from a URL string. A bare string is interpreted as a URL when its scheme is one of http, https, ftp, or file, or when an explicit kind=”url” hint is given. Network access from an implicit bare network URL requires explicit consent before opening; URL views and
kind="url"provide that consent. Content is transparently decompressed according to the compression hint.- Parameters:
url (str) – URL to fetch lazily when data is first read.
**hints (Any) – Backend-selection, consent, timeout, and compression hints.
- Raises:
ValueError – If the compression hint is unknown.
- class httk.core.datastream.BytestreamURLView(obj, **hints)[source]¶
Bases:
httk.core.datastream.bytestream_view.BytestreamView,strA view presenting an underlying data streaming backend via a URL string. This view is mostly useful for providing a URL to functions that will open it. Note: this view is not lazy (this is impossible for views inheriting str, since str is immutable).
Raises TypeError if created with a streaming data source that does not come with a URL.
- Parameters:
obj (httk.core.datastream.bytestream_like.BytestreamLike) – Byte-stream source whose URL should be presented.
**hints (Any) – Backend-selection, consent, timeout, and compression hints.
- Raises:
TypeError – If the source has no URL.
- class httk.core.datastream.BytestreamView[source]¶
Bases:
httk.core.views.View[httk.core.datastream.bytestream_backend.BytestreamBackend]Abstract base class for all views of streaming byte data.
Views retain the backend and expose it through a byte-stream interface.
- class httk.core.datastream.CompressionCodec[source]¶
A decompression codec for a single container format.
A codec is an orthogonal layer below the datastream backends: it turns a compressed binary stream into an uncompressed binary stream, independently of where the compressed bytes come from (a filename, an open file, raw bytes, or a remote response).
- Parameters:
name – Canonical name used to select the codec explicitly.
extensions – Filename suffixes that identify the codec.
magics – Leading byte signatures used to detect the codec.
open_stream – Function that wraps compressed bytes for reading.
- extensions: tuple[str, Ellipsis]¶
Recognized filename suffixes including the leading dot (e.g.
(".gz",)).
- magics: tuple[bytes, Ellipsis]¶
Leading magic-byte signatures; an empty tuple means the format cannot be sniffed.
- open_stream: collections.abc.Callable[[io.IOBase], io.IOBase]¶
Wrap a compressed binary stream and return a readable, decompressed binary stream.
- httk.core.datastream.known_compressions()[source]¶
Return the registered codec names, in registration order.
- httk.core.datastream.register_compression(codec)[source]¶
Register (or replace) a codec under its
name(case-insensitive).- Parameters:
codec (CompressionCodec) – Codec to add to the registry.
- class httk.core.datastream.DatastreamURL(url, *, timeout=None)[source]¶
An explicit network-consent value carrying a URL and optional timeout for lazy consumers, which resolve it through the existing fetch/loader machinery; constructing it performs no network I/O.
- Parameters:
- Raises:
ValueError – If the URL uses an unsupported scheme.
- class httk.core.datastream.TextstreamBackend(backend, **hints)[source]¶
Bases:
httk.core.views.Backend[TextstreamBackend],httk.core.datastream.textstream_api.TextstreamAPIAbstract base class for all backends of streaming text data.
- Parameters:
backend (Any) – Initial backend value accepted by the backend family.
**hints (Any) – Backend-selection hints used by concrete backends.
- backend_classes: ClassVar[list[type[httk.core.views.Backend[Any]]]]¶
- class httk.core.datastream.TextstreamCommon[source]¶
Bases:
abc.ABCCommon superclass for many of the implementations of backends for streaming text data.
- unwrap()[source]¶
Return the currently opened underlying text stream.
- Returns:
The stream used for reading text.
- Return type:
- read(size=-1)[source]¶
Read up to
sizecharacters, or all remaining characters whensizeis negative.
- class httk.core.datastream.TextstreamFile(obj, **hints)[source]¶
Bases:
httk.core.datastream.textstream_common.TextstreamCommon,httk.core.datastream.textstream_backend.TextstreamBackendBackend for file-based (io.TextIOBase-conforming) streaming text data
- Parameters:
obj (io.TextIOBase) – Open text stream to adopt and close with this backend.
**hints (Any) – Backend-selection and compression hints.
- Raises:
ValueError – If the compression hint is not a no-op mode for text-native content.
- class httk.core.datastream.TextstreamFileView(obj, **hints)[source]¶
Bases:
httk.core.datastream.textstream_view.TextstreamView,io.TextIOBase,httk.core.datastream.textstream_api.TextstreamAPIA view presenting an underlying data streaming backend via the full io.TextIOBase API, which is a superset of TextstreamAPI.
- Parameters:
obj (httk.core.datastream.textstream_like.TextstreamLike) – Text-stream source to present through the text file API.
**hints (Any) – Backend-selection, encoding, and compression hints.
- unwrap()[source]¶
Return the raw representation of the wrapped backend.
- Returns:
The backend’s most raw available representation.
- Return type:
Any
- writable()[source]¶
Report that the view does not support writing.
- Returns:
Always
False.- Return type:
- seekable()[source]¶
Report whether the backend supports seeking and telling.
- Returns:
Whether both operations are available.
- Return type:
- flush()[source]¶
Flush the backend when it provides flushing.
- Raises:
ValueError – If the view is closed.
- read(size=-1)[source]¶
Read up to
sizecharacters, or all remaining characters whensizeis negative.- Parameters:
size (int | None) – Maximum number of characters to read;
Nonealso means all remaining characters.- Returns:
The text read from the stream.
- Raises:
ValueError – If the view is closed.
- Return type:
- readline(size=-1)[source]¶
Read one line, optionally limited to
sizecharacters.- Parameters:
size (int | None) – Maximum number of characters to read;
Nonemeans no limit.- Returns:
The line read, including its newline when present.
- Raises:
ValueError – If the view is closed.
- Return type:
- readlines(hint=-1)[source]¶
Read lines until EOF or until the accumulated size reaches
hint.- Parameters:
hint (int) – Approximate minimum number of characters to collect, or a negative value for no limit.
- Returns:
The lines read from the stream.
- Raises:
ValueError – If the view is closed.
- Return type:
- seek(offset, whence=io.SEEK_SET)[source]¶
Move the stream position and discard buffered line data.
- Parameters:
- Returns:
The resulting absolute stream position.
- Raises:
ValueError – If the view is closed.
io.UnsupportedOperation – If the backend is not seekable.
- Return type:
- tell()[source]¶
Return the logical stream position before buffered line data.
- Returns:
The logical absolute stream position.
- Raises:
ValueError – If the view is closed.
io.UnsupportedOperation – If the backend does not support telling.
- Return type:
- detach()[source]¶
Reject detaching because the view owns its backend interface.
- Raises:
io.UnsupportedOperation – Always, because detaching is unsupported.
- class httk.core.datastream.TextstreamFilename(filename, **hints)[source]¶
Bases:
httk.core.datastream.textstream_common.TextstreamCommon,httk.core.datastream.textstream_backend.TextstreamBackendBackend for streaming text via operations on a file specfied by a filename Compressed content is transparently decompressed before text decoding.
- Parameters:
filename (str | pathlib.Path) – File path to open lazily for text reading.
**hints (Any) – Backend-selection, encoding, and compression hints.
- Raises:
ValueError – If the compression hint is unknown.
- class httk.core.datastream.TextstreamFilenameView(obj, **hints)[source]¶
Bases:
httk.core.datastream.textstream_view.TextstreamView,strA view presenting an underlying data streaming backend via a filename. This view is probably mostly useful for providing filenames to functions that will open them. Note: this view is not lazy (this is impossible for views inherting str, since str is immutable)
Raises TypeError if created with a streaming data source that does not have a name.
- Parameters:
obj (httk.core.datastream.textstream_like.TextstreamLike) – Text-stream source whose filename should be presented.
**hints (Any) – Backend-selection, encoding, and compression hints.
- Raises:
TypeError – If the source has no filename.
- type httk.core.datastream.TextstreamLike = textstream_backend.TextstreamBackend | textstream_view.TextstreamView | io.TextIOBase | io.StringIO | str | pathlib.Path | urllib.request.Request[source]¶
- class httk.core.datastream.TextstreamRequest(request, **hints)[source]¶
Bases:
httk.core.datastream.textstream_common.TextstreamCommon,httk.core.datastream.textstream_backend.TextstreamBackendBackend for streaming text fetched via a urllib.request.Request. Response content is transparently decompressed before text decoding.
- Parameters:
request (urllib.request.Request) – Request to execute lazily when text is first read.
**hints (Any) – Backend-selection, encoding, timeout, and compression hints.
- Raises:
ValueError – If the compression hint is unknown.
- property request: urllib.request.Request¶
Return the request used to fetch the stream.
- class httk.core.datastream.TextstreamRequestView(obj, **hints)[source]¶
Bases:
httk.core.datastream.textstream_view.TextstreamView,urllib.request.RequestA view presenting an underlying data streaming backend via a urllib.request.Request. This view is mostly useful for providing a Request to functions that will open it. Note: this view is not lazy (it does not fetch); it only mirrors the underlying request/URL.
Raises TypeError if created with a streaming data source that does not come with a URL.
- Parameters:
obj (httk.core.datastream.textstream_like.TextstreamLike) – Text-stream source to present as a request.
**hints (Any) – Backend-selection, encoding, timeout, and compression hints.
- Raises:
TypeError – If the source has no URL.
- class httk.core.datastream.TextstreamString(content, **hints)[source]¶
Bases:
httk.core.datastream.textstream_common.TextstreamCommon,httk.core.datastream.textstream_backend.TextstreamBackendBackend for streaming text backed by an actual string
- Parameters:
content (str) – Text to expose as a stream.
**hints (Any) – Backend-selection and compression hints.
- Raises:
ValueError – If the compression hint is not a no-op mode for text-native content.
- class httk.core.datastream.TextstreamStringView(obj, **hints)[source]¶
Bases:
httk.core.datastream.textstream_view.TextstreamView,strA view presenting an underlying data streaming as a string. This view can be used both to pass a string in place of streaming data, and for reading streaming data into a string. Note: this view is not lazy (this is impossible for views inherting str, since str is immutable), hence all the streaming data is read immedately upon creating this view.
- Parameters:
obj (httk.core.datastream.textstream_like.TextstreamLike) – Text-stream source to present or consume.
**hints (Any) – Backend-selection and compression hints.
- class httk.core.datastream.TextstreamURL(url, **hints)[source]¶
Bases:
httk.core.datastream.textstream_common.TextstreamCommon,httk.core.datastream.textstream_backend.TextstreamBackendBackend for streaming text fetched from a URL string. A bare string is interpreted as a URL when its scheme is one of http, https, ftp, or file, or when an explicit kind=”url” hint is given. Network access from an implicit bare network URL requires explicit consent before opening; URL views and
kind="url"provide that consent. Content is transparently decompressed before text decoding.- Parameters:
url (str) – URL to fetch lazily when text is first read.
**hints (Any) – Backend-selection, consent, encoding, timeout, and compression hints.
- Raises:
ValueError – If the compression hint is unknown.
- class httk.core.datastream.TextstreamURLView(obj, **hints)[source]¶
Bases:
httk.core.datastream.textstream_view.TextstreamView,strA view presenting an underlying data streaming backend via a URL string. This view is mostly useful for providing a URL to functions that will open it. Note: this view is not lazy (this is impossible for views inheriting str, since str is immutable).
Raises TypeError if created with a streaming data source that does not come with a URL.
- Parameters:
obj (httk.core.datastream.textstream_like.TextstreamLike) – Text-stream source whose URL should be presented.
**hints (Any) – Backend-selection, consent, encoding, timeout, and compression hints.
- Raises:
TypeError – If the source has no URL.
- class httk.core.datastream.TextstreamView[source]¶
Bases:
httk.core.views.View[httk.core.datastream.textstream_backend.TextstreamBackend]Abstract base class for all views of streaming text data.
Views retain the backend and expose it through a text-stream interface.
- type httk.core.datastream.DatastreamLike = BytestreamLike | TextstreamLike | DatastreamURL[source]¶