API and capability reference
Functions import from rivretrieve. The reference renders their source docstrings automatically.
Public functions
as_frame
rivretrieve.as_frame
Return the source-series table for a selection, the same table as series.
Selecting rows from the table does not change the selection. Pass chosen
identifiers back to pick instead. Use to_bundle to save a selection
without losing information.
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
audit_cache
rivretrieve.audit_cache
Check every file and observation in one committed local store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
provider
|
str
|
Registered live or bulk provider identifier. |
required |
Returns:
| Type | Description |
|---|---|
CacheAuditResult
|
Provider, path, generation identity, UTC check time and counts of partitions, rows and bytes checked. Return means all local integrity and consistency checks passed for that generation. |
Raises:
| Type | Description |
|---|---|
ObservationStoreRefusedError
|
If data is absent, incompatible, malformed or changed since publication. |
StoreLifecycleError
|
If active ownership or interrupted publication prevents an unambiguous
audit. Inspect |
Notes
This read-only operation makes no source request and does not repeat
certification against original publisher artifacts. It can read the whole
national dataset. cache_status does not perform this complete check.
recover_cache
rivretrieve.recover_cache
Finish interrupted local work without downloading observations.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
provider
|
str
|
Registered live or bulk provider identifier. |
required |
Returns:
| Type | Description |
|---|---|
CacheRecoveryResult
|
Provider, canonical path, resulting status and actions taken. Recovery validates committed data before preserving or restoring it. Uncommitted first-install staging is discarded; it is never promoted to a store. |
Raises:
| Type | Description |
|---|---|
StoreLifecycleError
|
If ownership is active or ambiguous, or surviving paths do not establish which generation was committed. No guess-based restoration is performed. |
ObservationStoreRefusedError
|
If the committed generation fails its complete local audit. |
OSError
|
If local restoration or cleanup fails. Inspect status before retrying. |
Notes
Recovery coordinates with writers and clear. It can read every observation
byte. It preserves source-selection facts and never merges old and new bulk
snapshots. Use clear_cache only when removal of all owned data is intended.
cache_status
rivretrieve.cache_status
Inspect one provider's local observation store without source access.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
provider
|
str
|
Registered provider identifier, for a live or bulk store. |
required |
Returns:
| Type | Description |
|---|---|
StoreStatus
|
Resolved path, committed generation and metadata-checked manifest,
interrupted paths, cleanup residue and local ownership. Observation bytes
are not audited. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If provider is not a nonempty string. |
UnknownProviderError
|
If provider is not registered. |
ObservationStoreRefusedError
|
If committed metadata is malformed or its publication identity changed. |
StoreLifecycleError
|
If the journal is malformed or managed paths are unsafe. |
OSError
|
If local file operations fail. |
ValueError
|
If |
clear_cache
rivretrieve.clear_cache
Delete one provider's compiled observation store or accumulated live store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
provider
|
str
|
Registered provider identifier. This call is explicit destructive consent. |
required |
Returns:
| Type | Description |
|---|---|
CacheClearResult
|
Provider identifier, store path, whether anything existed, total bytes removed and every removed path. Includes recognized pending publisher downloads, staging, previous generations and transaction workspaces. The permanent coordination file remains. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If provider is not a nonempty string. |
UnknownProviderError
|
If provider is not registered. |
BulkArtifactCleanupRefusedError
|
If the pending-download namespace is symlinked or contains an unsafe entry. |
StoreLifecycleError
|
If ownership is active or ambiguous, or a managed provider path is unsafe. |
OSError
|
If deletion fails. This operation is not transactional. |
ValueError
|
If |
Notes
This destructive action also removes preserved pending publisher downloads and recognized staging, previous generations and transaction workspaces, allowing a retry after interrupted retrieval or failed compilation. It does not download replacement data. It removes symlinks themselves rather than following them. Unrelated sibling paths are not removed.
describe
rivretrieve.describe
Read one provider's packaged Croissant JSON-LD descriptor offline.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
provider
|
str
|
Built-in provider identifier. |
required |
Returns:
| Type | Description |
|---|---|
dict[str, object]
|
Parsed descriptor with catalogue file identities, extraction rules, evidence relations and recorded absences. This is not an observation descriptor or a live inventory. |
Raises:
| Type | Description |
|---|---|
UnknownProviderError
|
If provider is not in the built-in manifest. |
FatalContractError
|
If the JSON document is not an object. |
OSError
|
If the packaged descriptor cannot be read. |
JSONDecodeError
|
If its JSON is malformed. |
download
rivretrieve.download
Download a bulk provider's national dataset and compile it into the local cache.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
provider
|
str
|
Registered bulk provider identifier, currently |
required |
Returns:
| Type | Description |
|---|---|
ValidatedStore
|
Validated compiled store with its root, manifest and partition paths. Successful certification replaces the previous store and deletes the downloaded publisher artifacts. Their identities remain in the manifest. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If provider is not a nonempty string. |
UnknownProviderError
|
If provider is not registered. |
BulkOperationsUnavailableError
|
If provider is not a bulk provider. |
InsufficientDiskSpaceError
|
If free space is below the declared requirement before any transfer. |
InsufficientPreparationSpaceError
|
If estimated additional growth exceeds free space after transfer. This check covers each workspace filesystem and does not reserve space. Retained inputs or cleanup residue can wrap this in StoreTransactionError. |
FatalContractError
|
If a source response or store validation violates a contract. |
StoreCertificationError
|
If compilation certification fails before publication. |
StoreTransactionError
|
If preparation failed and retained inputs or cleanup residue require
recovery. |
StorePostCommitCleanupError
|
If publication succeeded but cleanup failed. The new generation remains authoritative; the exception identifies it and any remaining paths. |
OSError
|
If local file operations fail. |
ValueError
|
If the provider finds the publisher's listing or artifacts
inconsistent, or finds that the newly published history would end
before the source vintage of the existing compiled store. The existing
store is not replaced. Also raised if |
Notes
This call can transfer a national dataset. It is not needed for live providers. Before publication, a failed compilation preserves the previous store and publisher inputs. Use recover_cache for interrupted work; use clear_cache only when deleting all owned state is intended. Transport failures can also propagate rather than becoming result issues.
Canada's disk estimate requires SQLITE_TMPDIR and TMPDIR to be set
before Python starts and unchanged for the process lifetime. SQLite can cache
those settings. If they changed after startup, restart Python with the intended
environment before downloading Canada; admission cannot recover the old state.
An existing compiled store that passes validation supplies its source
vintage, so the provider can refuse a download whose published history
would regress. An existing store that fails validation does not block the
call, because download is how such a store is rebuilt.
The store is written under RIVRETRIEVE_CACHE_DIR when that variable is
set in the environment or the working directory .env file, otherwise
under the platform's user cache directory. cache_status and
clear_cache resolve the same location.
metadata
The columns of both views are listed in the station metadata guide.
rivretrieve.metadata
Read selected gauges' packaged station metadata offline.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
selection
|
selection
|
Selection from find, pick or from_bundle. Multiple providers and series are accepted without repeating gauges or their attributes. |
required |
view
|
('summary', 'source')
|
Summary returns one row per gauge with canonical geometry, a scalar station name, and aligned lists of water-body, area and elevation fields. Source returns separate source attributes with exact values and support. |
"summary"
|
Returns:
| Type | Description |
|---|---|
DataFrame
|
Both views retain provider_id and station_id as strings. Summary station_name is the verbatim name when exactly one distinct nonblank supported name exists. It is null when none or several exist; the source view distinguishes these cases. Equal names from separate fields count as one name. Latitude, longitude and crs preserve the canonical catalogue's values and unknowns, independently of locations retained in the selection. Summary list columns are water_body_name_field, water_body_name_value; drainage_area_field, drainage_area_value, drainage_area_unit; and elevation_field, elevation_value, elevation_unit, elevation_datum. All have List(String) dtype. Within each role, positions align and sort by exact native field name, with source_scope breaking ties without preference. Identical field names from different source scopes remain separate entries. Water-body names are decoded strings. Area and elevation values remain JSON scalar text: json.loads distinguishes numbers from numeric-looking source strings. In the supported Japanese, Bosnian and Swiss quantity fields, complete numeric strings with explicit units keep their numeric text separately from the unit. For example, '"142.00km2"' becomes '"142.00"' with unit 'km2'. Numeric spelling is preserved and still needs parsing for arithmetic. In USGS alt_va, leading spaces, tabs and nonbreaking spaces are removed only from complete numeric strings: '" 0.00"' becomes '"0.00"', still a JSON string with unchanged numeric spelling, unit and datum. Qualified or unrecognised text stays unchanged. The source view always retains the exact original scalar. Units and datum labels or codes are plain strings. Unknowns remain null. No exposed fields gives null list cells; exposed null fields give named entries with null values. Blanks, whitespace, equal values in different fields, placeholders and zero values are retained. Source columns are provider_id, station_id, source_field, source_scope, source_value, source_dtype, source_unit, state, attribute_role, support_fact, source_datum, source_datum_field, source_datum_dtype and datum_support_fact. All are String except state and attribute_role, which are Enums. Decode non-null source_value with json.loads to recover the exact scalar. State is value, source_null (an exposed field with null), or no_metadata (no exposed field for this role). The latter has null source and support columns. Roles are station_name, water_body_name, drainage_area and elevation. Source scope identifies the source collection or entity when native field names overlap; it is null for an unscoped native projection. Datum fields carry the published label or code and its association support. Native datum fields retain their name and dtype, including when null; documented datum declarations have no native field or dtype. No inferred unit, preferred field or conversion is applied. Empty selections retain the chosen view's schema. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If selection is not a RivRetrieve selection. |
ValueError
|
If view is not summary or source. |
FatalContractError
|
If packaged metadata violates its schema, value states or gauge scope. |
OSError
|
If packaged metadata cannot be read. |
Notes
No observations, credentials, cache or network access are needed. Gauge scope is independent of observation admission and map eligibility. Metadata does not establish observation availability. Neither absence state means zero or that an agency publishes no metadata elsewhere. See docs/station-metadata.md for interpretation and examples.
fetch
rivretrieve.fetch
fetch(
selection: _Selection,
*,
start: object = None,
end: object = None,
receipts: bool = False,
cache: CacheMode = "bypass",
on_issue: OnIssue = "warn",
) -> ObservationResult
Download observations for a selection whose series come from one provider.
fetch requests each station and access route in the selection and
returns every series whose physical facts and identity match the
selection's filters. An unrestricted selection includes matching series
that are first identified in the source response. An explicit variant
or series_id restriction returns only those series, even when the
catalogue could not settle whether they exist.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
selection
|
selection
|
Selection from |
required |
start
|
str or datetime
|
Required first time label to include, on the source's own wall clock.
Accepts an ISO date ( |
None
|
end
|
(str, datetime or None)
|
Last time label to include, in the same forms as |
None
|
receipts
|
bool
|
If True, keep the source bytes behind the returned rows in
|
False
|
cache
|
('bypass', 'reuse', 'refresh')
|
Local observation cache behaviour for live providers. |
"bypass"
|
on_issue
|
('warn', 'raise', 'ignore')
|
Handling of |
"warn"
|
Returns:
| Type | Description |
|---|---|
ObservationResult
|
Observation rows in A source failure for one station or access route becomes an issue and a
Failed HTTP requests produce A part requested only as padding, wholly outside the requested dates,
produces no outcome unless its source request fails. A failed padding
request is reported with an issue and a When every request fails, the issues and outcomes explain why. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
ValueError
|
If |
EmptySelectionError
|
If the selection routes to no provider, station and access route. |
MultiProviderSelectionError
|
If the selection routes to more than one provider. |
InvalidObservationRequestError
|
If |
MissingCredentialError
|
If a required credential is not set. No source request is made. |
ObservationsUnavailableError
|
If the provider only publishes a catalogue, such as |
FatalContractError
|
If |
ObservationStoreRefusedError
|
If an existing local store is malformed or incompatible. |
IssuePolicyError
|
If |
Notes
Live providers are contacted over the network unless reuse finds every
requested station and access route covered in the cache. Requests that are
safe to repeat are retried for transient failures. The usage guide
describes the retry limits.
Credentials are read from the process environment, then from a .env
file in the working directory. They are required even when the answer
comes from the cache. The cache location is RIVRETRIEVE_CACHE_DIR,
read the same way, or the platform's user cache directory.
A bulk provider without a compiled store returns an empty result with a
bulk.store_missing warning and an unresolved outcome. fetch
never starts a national download. Call download to create the store.
Examples:
This example contacts the USGS service. The output shown was checked against a recorded USGS response.
>>> import rivretrieve as rr
>>> gauge = rr.find(
... provider="usgs_nwis", station="07374000",
... quantity="discharge", frequency="daily", statistic="mean",
... )
>>> result = rr.fetch(gauge, start="2023-01-01", end="2023-01-01")
>>> result.data.select("time", "time_zone", "source_unit", "unit", "value").rows()
[(datetime.datetime(2023, 1, 1, 0, 0), 'unknown', 'ft^3/s', 'm3/s', 10562.183778816001)]
>>> [outcome.status.value for outcome in result.outcomes]
['success']
fetch_by_provider
rivretrieve.fetch_by_provider
fetch_by_provider(
selection: _Selection,
*,
start: object = None,
end: object = None,
receipts: bool = False,
cache: CacheMode = "bypass",
on_issue: OnIssue = "warn",
) -> dict[str, ObservationResult]
Download observations for a multi-provider selection, one result per provider.
Parameters are the same as for fetch. The selection can route to any
number of providers.
Returns:
| Type | Description |
|---|---|
dict[str, ObservationResult]
|
One result per routed provider, keyed by provider identifier. Each
result has the same contents as a |
Raises:
| Type | Description |
|---|---|
MissingCredentialError
|
If any selected provider lacks a required credential. Credentials for every provider are checked before any provider is contacted. |
FatalContractError
|
If |
IssuePolicyError
|
If |
Notes
Providers are retrieved one after another. Source failures stay inside the
affected provider's result as issues and outcomes. An exception raised
while retrieving one provider stops the call, and results already
retrieved for other providers are not returned. The other exceptions
listed for fetch, apart from EmptySelectionError and
MultiProviderSelectionError, can also be raised.
find
rivretrieve.find
find(
*,
provider: str | None = None,
station: str | None = None,
quantity: str | None = None,
frequency: str | None = None,
statistic: str | None = None,
temporal_support: str | None = None,
day_definition: str | None = None,
timestamp_anchor: str | None = None,
time_zone: str | None = None,
vertical_reference: str | None = None,
vertical_datum: str | None = None,
variant: str | None = None,
series_id: str | None = None,
on_issue: OnIssue = "warn",
) -> _Selection
Search the packaged catalogues for source series that match the given filters.
find reads the packaged catalogues only. It does not contact observation
services, read credentials or use the observation cache. The returned
selection records the filters as request intent together with the catalogue
evidence known today. fetch uses that intent, so an unrestricted
selection also includes matching series that a source response reveals later.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
provider
|
str or None
|
Provider identifier, such as |
None
|
station
|
str or None
|
Station identifier as the provider publishes it. Keep identifiers as
strings so leading zeros survive. Without |
None
|
quantity
|
str or None
|
Exact filters on established physical facts. Quantities are
|
None
|
frequency
|
str or None
|
Exact filters on established physical facts. Quantities are
|
None
|
statistic
|
str or None
|
Exact filters on established physical facts. Quantities are
|
None
|
temporal_support
|
str or None
|
Exact filters on established physical facts. Quantities are
|
None
|
day_definition
|
str or None
|
Exact filters on established physical facts. Quantities are
|
None
|
timestamp_anchor
|
str or None
|
Exact filters on established physical facts. Quantities are
|
None
|
time_zone
|
str or None
|
Exact filters on established physical facts. Quantities are
|
None
|
vertical_reference
|
str or None
|
Exact filters on established physical facts. Quantities are
|
None
|
vertical_datum
|
str or None
|
Exact filters on established physical facts. Quantities are
|
None
|
variant
|
str or None
|
Explicit source restriction. |
None
|
series_id
|
str or None
|
Explicit source restriction. |
None
|
on_issue
|
('warn', 'raise', 'ignore')
|
Handling of warning and error issues found during selection. See
|
"warn"
|
Returns:
| Type | Description |
|---|---|
selection
|
A selection to pass to |
Raises:
| Type | Description |
|---|---|
UnknownProviderError
|
If |
UnknownStationError
|
If |
ValueError
|
If |
IssuePolicyError
|
If |
Notes
Two issue codes can be recorded on the selection's issues for an
explicit variant or series_id that matches no known series.
selection.no_match means the packaged evidence establishes that nothing
matches. selection.unresolved_inventory means the catalogue cannot tell
whether the source publishes that series. RivRetrieve still requests an
unresolved restriction during fetch. Physical filters that match nothing
produce an empty selection without an issue. fetch refuses such a
selection with EmptySelectionError.
The packaged catalogue is a snapshot. It does not establish that every listed series has observations for a particular period.
from_bundle
rivretrieve.from_bundle
Restore a selection or result saved by to_bundle.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
content
|
bytes
|
Bytes produced by |
required |
Returns:
| Type | Description |
|---|---|
selection or ObservationResult
|
The kind of value that was exported. Identities, physical facts, inventories, issues and, for results, observations, outcomes, provenance and receipts come from the bundle. The current packaged catalogue is not consulted, so a restored selection keeps the evidence it had when exported. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
ValueError
|
If the bytes are not a bundle, the bundle version is not 3, or its contents fail validation. Bundles from other format versions must be exported again or re-fetched. |
ObservationDataSchemaError
|
If a result bundle's observation rows contradict its source-series definitions or the observation frame schema, its outcomes contradict those definitions, or its receipt provider differs from its provenance provider. |
from_frame
rivretrieve.from_frame
Refuse to rebuild a selection from a table; use from_bundle instead.
Every call raises ValueError. An inspection frame omits request intent,
inventory state and evidence, so it cannot be turned back into a selection.
Use to_bundle and from_bundle to save and restore a selection.
Raises:
| Type | Description |
|---|---|
ValueError
|
Always. |
map
rivretrieve.map
Render selected stations without narrowing the selection.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
selection
|
selection
|
Selection from find, pick or from_bundle. Each station is shown once. |
required |
Returns:
| Type | Description |
|---|---|
Map
|
Station markers with identity, coordinates and CRS in their popups. Unknown CRS markers are orange and drawn as if EPSG:4326. Established CRS markers are blue. The rendering does not rewrite catalogue facts. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If selection is not a RivRetrieve selection. |
MissingOptionalDependencyError
|
If folium is not installed. Install the rivretrieve[map] extra. |
pick
rivretrieve.pick
pick(
selection: _Selection | ObservationResult,
*,
provider: str | Sequence[str] | None = None,
station: str | Sequence[str] | None = None,
quantity: str | None = None,
frequency: str | None = None,
statistic: str | None = None,
temporal_support: str | None = None,
day_definition: str | None = None,
timestamp_anchor: str | None = None,
time_zone: str | None = None,
vertical_reference: str | None = None,
vertical_datum: str | None = None,
variant: str | Sequence[str] | None = None,
series_id: str | Sequence[str] | None = None,
on_issue: OnIssue = "warn",
) -> _Selection | ObservationResult
Filter a selection or a fetched result without contacting a source.
Filters combine with the filters already held by selection. A filter
that conflicts with an existing one, such as a different quantity, leaves an
empty selection or an empty result view.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
selection
|
selection or ObservationResult
|
Selection from |
required |
provider
|
str, sequence of str, or None
|
One identifier or a list of identifiers. A list keeps series matching
any of its values. |
None
|
station
|
str, sequence of str, or None
|
One identifier or a list of identifiers. A list keeps series matching
any of its values. |
None
|
variant
|
str, sequence of str, or None
|
One identifier or a list of identifiers. A list keeps series matching
any of its values. |
None
|
series_id
|
str, sequence of str, or None
|
One identifier or a list of identifiers. A list keeps series matching
any of its values. |
None
|
quantity
|
str or None
|
Exact filters on established physical facts, as in |
None
|
frequency
|
str or None
|
Exact filters on established physical facts, as in |
None
|
statistic
|
str or None
|
Exact filters on established physical facts, as in |
None
|
temporal_support
|
str or None
|
Exact filters on established physical facts, as in |
None
|
day_definition
|
str or None
|
Exact filters on established physical facts, as in |
None
|
timestamp_anchor
|
str or None
|
Exact filters on established physical facts, as in |
None
|
time_zone
|
str or None
|
Exact filters on established physical facts, as in |
None
|
vertical_reference
|
str or None
|
Exact filters on established physical facts, as in |
None
|
vertical_datum
|
str or None
|
Exact filters on established physical facts, as in |
None
|
on_issue
|
('warn', 'raise', 'ignore')
|
Handling of warning and error issues relevant to the narrowed selection
or view. See |
"warn"
|
Returns:
| Type | Description |
|---|---|
selection or ObservationResult
|
The same kind of value as For a selection, the result is a new selection with the combined filters.
Findings about explicit For a result, the returned result keeps only observation rows whose
series and fact segment match the combined filters. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
UnknownProviderError
|
If |
UnknownStationError
|
If |
ValueError
|
If |
IssuePolicyError
|
If |
products
rivretrieve.products
Inspect source access-coordinate names recorded in packaged catalogues.
These internal routing names do not establish physical meaning. They are not accepted by find or pick; use quantity and established physical facts there.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
provider
|
str or None
|
Registered provider identifier. None includes every provider. |
None
|
Returns:
| Type | Description |
|---|---|
list[str]
|
Sorted, unique access-coordinate names. Neither a name nor its wording establishes a series statistic, temporal support or current availability. |
Raises:
| Type | Description |
|---|---|
UnknownProviderError
|
If provider is not registered. |
FatalContractError
|
If a shipped declaration or catalogue is invalid. |
providers
rivretrieve.providers
List registered providers and local credential readiness.
Returns:
| Type | Description |
|---|---|
DataFrame
|
Columns are provider_id (String), credentials (List(String)) and access
(String). Rows are sorted by provider_id. Access is "open", "ready",
or "missing |
Raises:
| Type | Description |
|---|---|
FatalContractError
|
If a shipped declaration or catalogue cannot be loaded. |
Notes
Reads credential names from declarations and values from the process environment or the working directory .env file. A present environment variable takes precedence, including a blank value. Values are not returned. No source request is made.
series
rivretrieve.series
Describe each source series in a selection or result as a table.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
selection or ObservationResult
|
A selection from |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
One row per source series and physical-fact segment that matches the
selection or view. The schema is fixed and shown under "Series
inspection frame" in the API reference. Rows sort by Columns form these groups:
For a result, a series with a |
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
Notes
A row with admission equal to unsupported is still listed. Retrieval
returns no numeric observation rows for such a series.
to_bundle
rivretrieve.to_bundle
Save a selection or result as bytes that from_bundle can restore.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
selection or ObservationResult
|
Selection from |
required |
Returns:
| Type | Description |
|---|---|
bytes
|
ZIP archive in bundle format version 3. Write the bytes to a file to keep them. A selection bundle holds its request intent, source-series definitions, inventories, issues, station locations, catalogue evidence and empty-selection reason. A result bundle also holds the observation rows as Parquet, the outcomes, the view scope, provenance and any retained receipt bytes. Nothing is read from or written to disk. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
to_utc
rivretrieve.to_utc
Convert observation labels to UTC using each row's published zone.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
result
|
ObservationResult
|
Result whose time_zone values are established IANA identifiers or fixed offsets. Unknown zones are refused before converting any row. |
required |
Returns:
| Type | Description |
|---|---|
ObservationResult
|
New result with naive UTC time values and time_zone="+00:00" on every row. Data keeps its ten identity-bearing columns and timestamp dtype. Conversion uses Python datetime values with microsecond precision, so submicrosecond precision is not preserved. Provenance, issues and receipts are unchanged. The input is not modified. |
Raises:
| Type | Description |
|---|---|
FatalContractError
|
If any row has time_zone="unknown". The error gives the provider and count. |
ValueError
|
If a zone value violates the zone vocabulary or an IANA zone cannot be resolved by the local timezone database. |
Notes
No station catalogue or coordinate lookup is used. IANA conversions use Python timezone rules and the default fold for ambiguous labels. This does not establish a daily product's day definition or make series comparable.
Returned domain types
These types are not re-exported from rivretrieve. Their headings identify their implementation locations. Create selections through the public functions rather than constructing _Selection.
_Selection
rivretrieve._internal.selection._Selection
dataclass
_Selection(
scope: SeriesScope,
known_series: tuple[SourceSeries, ...] = (),
inventories: tuple[InventorySnapshot, ...] = (),
issues: tuple[Issue, ...] = (),
locations: tuple[StationLocation, ...] = (),
acquisition_provenance: tuple[
CatalogueEvidence, ...
] = (),
empty_reason: _EmptyReason | None = None,
)
A search to retrieve later: requested filters plus the catalogue evidence found.
A selection is returned by find, pick and from_bundle and cannot
be changed in place. Pass it to the public functions rather than building
one directly. The known series are evidence, and the filters remain the
request, so retrieval can include series found later. series(selection)
shows their contents as a frame.
Attributes:
| Name | Type | Description |
|---|---|---|
scope |
SeriesScope
|
Requested filters and source restrictions. |
known_series |
tuple[SourceSeries, ...]
|
Source-series definitions retained from packaged evidence for the selected stations and access routes. Some may not match the physical filters. |
inventories |
tuple[InventorySnapshot, ...]
|
Packaged inventory snapshots for the selected coordinates. |
issues |
tuple[Issue, ...]
|
Findings recorded during selection, such as |
locations |
tuple[StationLocation, ...]
|
Catalogue coordinates and CRS statements used by |
acquisition_provenance |
tuple[CatalogueEvidence, ...]
|
Evidence describing how the packaged catalogues were acquired. |
empty_reason |
empty-selection reason or None
|
None when at least one known admitted series matches. Otherwise
|
series |
tuple[SourceSeries, ...]
|
Property returning the known series that match |
acquisition_provenance
class-attribute
instance-attribute
ObservationResult
rivretrieve._internal.observations.ObservationResult
Bases: BaseModel
Harmonised observations with source identities, physical facts and retrieval outcomes.
data preserves source wall-clock labels, zones, station/product routing,
series and fact-segment identities, source-unit vocabulary, physical quantity,
harmonised unit and nullable values. Definitions, scoped inventory snapshots
and outcomes remain inspectable even when a series has no observation rows.
scope is the original request; view_scope records explicit post-fetch
narrowing without pretending another source request occurred.
Results cannot be changed in place. They are returned by fetch,
fetch_by_provider, pick, to_utc and from_bundle. Every
result belongs to one provider.
Attributes:
| Name | Type | Description |
|---|---|---|
data |
DataFrame
|
Observation rows with exactly the ten columns of the observation frame
schema, even when empty. |
provenance |
ObservationProvenance
|
Request, source-call, terms and cache context for the retrieval. |
issues |
tuple[Issue, ...]
|
Every finding retained for the retrieval, including source failures with their identity and reason. |
receipts |
Receipts
|
Source bytes kept when retrieval used |
source_series |
tuple[SourceSeries, ...]
|
Definitions and physical facts for the requested series, including series first identified in the response. |
inventories |
tuple[InventorySnapshot, ...]
|
Inventory snapshots used or acquired during retrieval. |
outcomes |
tuple[RetrievalOutcome, ...]
|
Retrieval statuses, each for one source series, or for a request that
has no concrete series identity, over the interval in its |
supporting_outcomes |
tuple[RetrievalOutcome, ...]
|
Original acquisition outcomes needed to interpret retained inventories.
These records can include historical failures that later retrievals
resolved. They are separate from active |
scope |
SeriesScope
|
The filters and restrictions of the original request. |
view_scope |
SeriesScope or None
|
Filters applied afterwards with |
supporting_outcomes
class-attribute
instance-attribute
ObservationResult.to_polars
rivretrieve._internal.observations.ObservationResult.to_polars
Return observation rows with identity and physical-unit context.
Use to_bundle to retain inventory, outcomes, provenance and receipts
through a durable export. These cannot be encoded by absent observation rows.
ObservationResult.to_pandas
rivretrieve._internal.observations.ObservationResult.to_pandas
Return the observation rows as a pandas DataFrame.
Returns:
| Type | Description |
|---|---|
DataFrame
|
The ten observation columns of |
ObservationProvenance
rivretrieve._internal.observations.ObservationProvenance
Bases: BaseModel
Source and request facts that accompany observations.
Attributes:
| Name | Type | Description |
|---|---|---|
source |
str
|
Observation source identifier, including "local" for compiled stores. |
provider_id |
ProviderId
|
Provider identity for every row in the observation frame. |
rivretrieve_version, catalogue_version |
str or None
|
Software and packaged catalogue versions when recorded. |
license, citation |
str or None
|
Established source terms and credit. None means not established here. |
requested_at, retrieved_at |
datetime or None
|
Local UTC invocation instant and latest known original acquisition instant among returned calls. Reuse does not create an acquisition time; no known returned acquisition time leaves retrieved_at as None. |
request |
dict[str, object] or None
|
Selected series and resolved start and end wall-clock endpoints. |
calls_made |
tuple[dict[str, object], ...]
|
Ordered source-call origins and sanitized prerequisite exchange events.
A failed request for one series and interval also appears, with its
|
time_windows |
tuple[dict[str, object], ...]
|
Additional window metadata. The current engine leaves this tuple empty. |
decomposition |
tuple[str, ...]
|
Additional decomposition metadata. The current engine leaves this tuple empty. Unit conversion currently emits no informational result issue. |
endpoints |
tuple[str, ...]
|
Distinct source URLs used by retained call metadata. |
query |
dict[str, object] or None
|
Executed local-query description when applicable. |
response_version, metadata |
str or None
|
Optional source response version and metadata text. |
served_intervals |
tuple[CoverageInterval, ...]
|
Held intervals served from an accumulated store, with retrieval instants. |
source_vintage |
date or None
|
Source vintage of the compiled store that served the rows, as
described for |
publisher_artifact_checksum |
str or None
|
Checksum of the first publisher artifact for compiled-store provenance. |
publisher_artifact_checksums, publisher_artifact_urls |
tuple[str, ...]
|
Ordered identities of all compiled publisher artifacts. |
acquisition_provenance |
CatalogueEvidence or None
|
Normalized evidence for the packaged catalogue, not observation quality. |
served_intervals
class-attribute
instance-attribute
publisher_artifact_checksum
class-attribute
instance-attribute
acquisition_provenance
class-attribute
instance-attribute
publisher_artifact_checksums
class-attribute
instance-attribute
Receipts
rivretrieve._internal.observations.Receipts
dataclass
Optional byte receipts for one provider.
Receipts let you inspect source material, for example values before unit
conversion. Their content can include rows outside the requested window or
a pick view, and values in source units. They are not a full
reproducibility archive.
Attributes:
| Name | Type | Description |
|---|---|---|
provider_id |
ProviderId
|
Provider identity shared by the entries. |
entries |
tuple[ReceiptEntry, ...]
|
Retained entries. Empty by default and when retrieval omits receipts. |
ReceiptEntry
rivretrieve._internal.observations.ReceiptEntry
dataclass
Retained bytes with an origin and explicit authorship.
Attributes:
| Name | Type | Description |
|---|---|---|
content |
bytes
|
Exact provider parse input, or a store excerpt for StoreExcerptReceipt. |
origin |
SourceCallOrigin
|
Source-call facts with explicit unknown states and no request headers. |
authorship |
ReceiptAuthorship
|
publisher_payload identifies source-authored bytes. store_excerpt identifies rows encoded by RivRetrieve, not a publisher response. |
StoreExcerptReceipt
rivretrieve._internal.observations.StoreExcerptReceipt
dataclass
StoreExcerptReceipt(
content: bytes,
origin: SourceCallOrigin,
authorship: ReceiptAuthorship,
store_path: StoreRoot,
executed_query: ExecutedStoreQuery,
format_version: int,
source_vintage: date | None,
)
Bases: ReceiptEntry
A Parquet re-encoding of the exact rows returned by a store query.
Attributes:
| Name | Type | Description |
|---|---|---|
content, origin, authorship |
ReceiptEntry fields. Authorship is always store_excerpt. |
|
store_path |
StoreRoot
|
Local store queried. |
executed_query |
ExecutedStoreQuery
|
Product, year, station and closed wall-clock predicates used by the scan. |
format_version |
int
|
Store layout revision, 5 for compiled or 8 for accumulated stores. |
source_vintage |
date or None
|
Source vintage of the compiled store, as described for
|
EvidenceState
rivretrieve._internal.source_series.EvidenceState
Bases: StrEnum
Whether a physical fact is known, absent from the source, or not established here.
EvidenceFact
rivretrieve._internal.source_series.EvidenceFact
Bases: BaseModel
One independently established physical fact and its evidence.
known requires a nonempty value and evidence. source_silent means
the source does not state the fact; not_established means it has not
been established here. Both unknown states retain a null value.
ClippingAxis
rivretrieve._internal.source_series.ClippingAxis
SourceUnitCodeDefinition
rivretrieve._internal.source_series.SourceUnitCodeDefinition
PhysicalFacts
rivretrieve._internal.source_series.PhysicalFacts
Bases: BaseModel
Physical meaning for one fact segment within a source series.
Each EvidenceFact retains its own knowledge state. Frequency and statistic
describe observations, not how often the service updates. Temporal support
describes the interval represented by a value. Day definition, timestamp
anchor and time zone remain independent facts; none is inferred from another.
normalized_unit preserves the source unit scale and zero. Conversion
to the output unit is described by Admission, not by unit normalization.
facts_id joins this segment to observation rows and retrieval outcomes.
source_unit_definition
class-attribute
instance-attribute
vertical_reference
class-attribute
instance-attribute
clipping_axis
class-attribute
instance-attribute
Admission
rivretrieve._internal.source_series.Admission
Bases: BaseModel
Whether established quantity and unit facts permit harmonised numeric rows.
supported carries the target unit and multiplicative conversion factor.
unsupported carries a reason. Admission does not rank source quality or
require every temporal fact to be known.
SourceIdentity
rivretrieve._internal.source_series.SourceIdentity
Bases: BaseModel
Agency-published identity within a source namespace, with origin and evidence.
published_id and description can be absent. Origin records whether the
identity was acquired from a catalogue, response or established mapping.
An identity is not a harmonised quality score.
SourceSeries
rivretrieve._internal.source_series.SourceSeries
Bases: BaseModel
One concrete source identity at a provider, station and access route.
series_id joins the definition to rows, inventories and outcomes.
product_id is an internal access route, not a physical classification.
variant preserves a selectable source alternative when established.
facts contains independently identified physical-fact segments.
Matching physical facts do not establish scientific interchangeability.
PhysicalPredicate
rivretrieve._internal.source_series.PhysicalPredicate
Bases: BaseModel
An exact filter on a known physical fact; unknown facts do not match.
RestrictionKind
rivretrieve._internal.source_series.RestrictionKind
ScopeState
rivretrieve._internal.source_series.ScopeState
SeriesScope
rivretrieve._internal.source_series.SeriesScope
Bases: BaseModel
Physical filters and source restrictions retained by a selection or result.
Empty coordinate tuples impose no coordinate restriction. all includes
matching identities discovered during retrieval, not only packaged members.
explicit retains requested variants or series IDs without fallback to
other identities. Variant matching accepts a variant or published source ID.
All physical predicates must match known facts within a fact segment.
An empty scope matches nothing.
SeriesWindow
rivretrieve._internal.source_series.SeriesWindow
InventoryCompleteness
rivretrieve._internal.source_series.InventoryCompleteness
CatalogueSeriesClaim
rivretrieve._internal.source_series.CatalogueSeriesClaim
Bases: BaseModel
A scoped publisher catalogue claim, not an alias for a response series.
InventorySnapshot
rivretrieve._internal.source_series.InventorySnapshot
Bases: BaseModel
What is known about source-series membership for a scope and access route.
members holds concrete series IDs; member_facts can identify their
fact segments. Catalogue claims retain separately published catalogue
identities rather than pretending they are response series.
Completeness applies only to this scope, access and optional window, not
countrywide coverage or a full historical census. Incomplete and unresolved
snapshots carry a reason. Acquisition time and catalogue check date record
evidence timing, not continuous observation coverage.
member_facts
class-attribute
instance-attribute
catalogue_claims
class-attribute
instance-attribute
OutcomeStatus
rivretrieve._internal.source_series.OutcomeStatus
Bases: StrEnum
Result of one retrieval: success, empty, failed, unsupported, unresolved or no_match.
success and empty retain a concrete series and fact IDs. empty
means no observation rows, not a row with a null value. failed records
a source failure; unsupported records an unsupported series.
unresolved means availability cannot be established. no_match means
the available evidence establishes that nothing matches the request.
RequestedSelector
rivretrieve._internal.source_series.RequestedSelector
RetrievalOutcome
rivretrieve._internal.source_series.RetrievalOutcome
Bases: BaseModel
Retrieval status for one source series over the interval in window.
An outcome can instead describe a requested station, access route or selector that has no concrete series identity. A series can have several outcomes for different intervals or fact segments.
series_id can be absent when no concrete identity is established.
requested_selector preserves the caller restriction without inventing
a source identity. Unsuccessful outcomes retain a reason. calls links
source-call evidence; retrieved_at records retrieval timing when known.
Outcomes remain present even when there are no observation rows.
coverage="observations" limits a successful result to its published row
identities; it does not establish completeness over window.
observation_keys records admitted snapshot rows as tuples of physical-fact
ID, native wall-clock timestamp and time zone. A key can identify several
published rows with the same label; their values and multiplicity remain
unchanged. Each key retains this outcome's calls and retrieval instant. Stored keys describe accepted row updates;
returned held keys describe only the observations actually served.
requested_selector
class-attribute
instance-attribute
coverage
class-attribute
instance-attribute
Issue
rivretrieve._internal.issues.Issue
Bases: BaseModel
A problem or note recorded during selection or retrieval, kept with the results.
Issues are kept on selections and results whatever on_issue policy is
chosen. They let independent series return data while a failure elsewhere
stays visible with its identity and reason.
Attributes:
| Name | Type | Description |
|---|---|---|
severity |
{info, warning, error}
|
Finding severity. Only warning and error activate the caller issue policy. |
code |
str
|
Machine-readable classification, usually written as
|
message |
str
|
Human-readable finding. |
details |
dict[str, object] or None
|
Structured context, such as station, product and source failure reason.
Source request failures record |
provider_id |
ProviderId or None
|
Provider responsible for the affected series when known. |
CatalogueEvidence
rivretrieve._internal.catalogues.evidence.CatalogueEvidence
Bases: _EvidenceModel
Validated catalogue evidence in normalized Polars relations.
Attributes:
| Name | Type | Description |
|---|---|---|
header |
EvidenceHeader
|
Versioned source declarations, exact terms, acquisition descriptions, transformations, withheld facts and identities of the five relations. |
facts |
DataFrame
|
Fact identities and station/product locators. |
acquisitions |
DataFrame
|
Source-local acquisition identities, methods, times and recordings. |
bindings |
DataFrame
|
Links between output groups, sources, acquisitions and transformations. |
binding_facts |
DataFrame
|
Ordered fact membership for each binding. |
external_inputs |
DataFrame
|
Exact ordered input facts for transformations. JSON serialization uses one array per physical column and is explicit, not part of discovery. |
RequestedInterval
rivretrieve._internal.coverage.RequestedInterval
dataclass
A closed interval on an explicit time axis.
start and end are naive timestamps, and both are included. The
native axis uses source wall-clock labels. The UTC axis uses UTC labels.
CoverageInterval
rivretrieve._internal.coverage.CoverageInterval
dataclass
CoverageInterval(
series_id: str,
interval: RequestedInterval,
retrieved_at: datetime | None,
outcome_id: str,
facts_ids: tuple[str, ...] = (),
)
Successful coverage for exactly one concrete source series, not inventory.
Coverage records that a retrieval succeeded for an interval. It does not mean that observations exist at every time step in that interval.
Attributes:
| Name | Type | Description |
|---|---|---|
series_id |
str
|
Source series that was retrieved. |
interval |
RequestedInterval
|
Covered interval on its declared native or UTC time axis. |
retrieved_at |
datetime or None
|
UTC instant of the source retrieval, or None when it is not known. |
outcome_id |
str
|
Retrieval outcome that established the coverage. A successful answer with no rows can also establish coverage. |
facts_ids |
tuple[str, ...]
|
Physical-fact segments covered, when recorded. |
SourceCallOrigin
rivretrieve._internal.engine.SourceCallOrigin
dataclass
SourceCallOrigin(
url: str | UnknownOriginFact,
request_parameters: Mapping[str, SourceCallParameter]
| UnknownOriginFact,
status_code: int | UnknownOriginFact,
retrieved_at: datetime | UnknownOriginFact,
content_type: str | UnknownOriginFact,
source_path: str | UnknownOriginFact,
query: SourceQuery | UnknownOriginFact,
*,
attempts: int | None = None,
)
Source-call facts carried with retained receipt bytes.
Attributes:
| Name | Type | Description |
|---|---|---|
url |
str or UnknownOriginFact
|
Source URL when applicable. |
request_parameters |
Mapping[str, SourceCallParameter] or UnknownOriginFact
|
Source query parameters. Request headers are never stored here. |
status_code |
int or UnknownOriginFact
|
Source response status when applicable. |
retrieved_at |
datetime or UnknownOriginFact
|
UTC source retrieval instant when established. |
content_type |
str or UnknownOriginFact
|
Published content type when established. |
source_path |
str or UnknownOriginFact
|
Local input path when applicable. |
query |
SourceQuery or UnknownOriginFact
|
Executed local statement and parameters when applicable. UnknownOriginFact carries a reason rather than filling an inapplicable fact by assumption. |
attempts |
int or None
|
Transport attempt count when retained. Payload attempt traces carry the individual outcomes when the transport exposes them. |
request_parameters
instance-attribute
StoreStatus
rivretrieve._internal.store.reader.StoreStatus
dataclass
StoreStatus(
store: StoreRoot,
provider_id: ProviderId,
presence: StorePresence,
manifest: StoreManifest
| AccumulatedStoreManifest
| None = None,
bytes_on_disk: int = 0,
generation_id: str | None = None,
committed_path: Path | None = None,
interrupted_paths: tuple[Path, ...] = (),
cleanup_paths: tuple[Path, ...] = (),
ownership: Ownership = Ownership.IDLE,
)
Metadata-checked local state without reading observation bytes.
Attributes:
| Name | Type | Description |
|---|---|---|
store |
StoreRoot
|
Resolved local store path, also exposed as root. |
provider_id |
ProviderId
|
Provider whose store was inspected. |
presence |
StorePresence
|
|
manifest |
(StoreManifest, AccumulatedStoreManifest or None)
|
Manifest checked against its publication identity and metadata rules. Presence does not claim all observation bytes have been checked. |
bytes_on_disk |
int
|
Logical bytes in committed metadata, partitions and seal, zero without committed data. Cleanup residue is listed separately. This is not allocated disk space or the bytes a clear operation will reclaim. |
coverage |
tuple[CoverageInterval, ...]
|
Current successful accumulated-store coverage, otherwise empty. |
partition_row_counts |
Mapping[PartitionIdentifier, int]
|
Property exposing physical row counts, empty when absent. |
format_version, compiler_version, built_at, source_vintage |
Properties exposing applicable manifest fields, otherwise None. |
|
publisher_artifact_url, publisher_artifact_checksum, source_schema_fingerprint |
Compiled-store identity properties, otherwise None. |
|
publisher_artifact_urls, publisher_artifact_checksums |
tuple
|
Ordered identities for all compiled artifacts, otherwise empty. |
generation_id |
str or None
|
Identity of the committed publication whose metadata was checked. |
committed_path |
Path or None
|
Selected committed data, including recoverable prior data during an interrupted replacement. It can differ from the canonical store path. |
interrupted_paths |
tuple[Path, ...]
|
Recognized work requiring recovery before reads or another acquisition. |
cleanup_paths |
tuple[Path, ...]
|
Residue remaining after commit. Its presence does not invalidate the selected new generation. |
ownership |
Ownership
|
Local ownership state: idle, active, abandoned or ambiguous. Active or ambiguous ownership prevents recovery and clear. |
manifest
class-attribute
instance-attribute
ValidatedStore
rivretrieve._internal.store.validation.ValidatedStore
dataclass
ValidatedStore(
root: StoreRoot,
manifest: StoreManifest | AccumulatedStoreManifest,
partition_files: Mapping[PartitionIdentifier, Path],
)
A store whose manifest and physical partitions passed validation.
Attributes:
| Name | Type | Description |
|---|---|---|
root |
StoreRoot
|
Resolved local store path. |
manifest |
StoreManifest or AccumulatedStoreManifest
|
Validated manifest. download returns a compiled StoreManifest. |
partition_files |
Mapping[PartitionIdentifier, Path]
|
Read-only product/year partition paths. |
StoreManifest
rivretrieve._internal.store.validation.StoreManifest
dataclass
StoreManifest(
format_version: Literal[5],
provider_id: ProviderId,
compiler_version: str,
built_at: datetime,
source_vintage: date,
publisher_artifact: PublisherArtifact,
publisher_artifacts: tuple[PublisherArtifact, ...],
source_schema: SourceSchema,
source_column_dispositions: tuple[
SourceColumnDisposition, ...
],
partition_row_counts: Mapping[PartitionIdentifier, int],
series: tuple[SourceSeries, ...] = (),
inventories: tuple[InventorySnapshot, ...] = (),
outcomes: tuple[RetrievalOutcome, ...] = (),
issues: tuple[Issue, ...] = (),
source_calls: tuple[dict[str, object], ...] = (),
)
Manifest of a compiled bulk store.
Attributes:
| Name | Type | Description |
|---|---|---|
format_version |
int
|
Compiled layout revision, 5. |
provider_id |
ProviderId
|
Provider whose native observations are stored. |
compiler_version |
str
|
Software version used to compile the store. |
built_at |
datetime
|
UTC build instant. |
source_vintage |
date
|
Identifies the source publication state the store was compiled from.
Its derivation depends on the provider: for example, the date of a
dated publisher release, or the last date covered by the latest
published period. It is not a freshness verdict. |
publisher_artifact |
PublisherArtifact
|
First publisher artifact identity, retained for single-artifact access. |
publisher_artifacts |
tuple[PublisherArtifact, ...]
|
Ordered artifact URLs and SHA-256 checksums. These identify deleted inputs. |
source_schema |
SourceSchema
|
Native column names, types and schema fingerprint. |
source_column_dispositions |
tuple[SourceColumnDisposition, ...]
|
Retained, reconstructible or deliberately discarded fields and rationale. |
partition_row_counts |
Mapping[PartitionIdentifier, int]
|
Exact physical row counts keyed by product/year. |
source_column_dispositions
instance-attribute
AccumulatedStoreManifest
rivretrieve._internal.store.validation.AccumulatedStoreManifest
dataclass
AccumulatedStoreManifest(
format_version: Literal[8],
provider_id: ProviderId,
built_at: datetime,
coverage: tuple[CoverageInterval, ...],
partition_row_counts: Mapping[PartitionIdentifier, int],
series: tuple[SourceSeries, ...] = (),
inventories: tuple[InventorySnapshot, ...] = (),
outcomes: tuple[RetrievalOutcome, ...] = (),
issues: tuple[Issue, ...] = (),
source_calls: tuple[dict[str, object], ...] = (),
)
Manifest of accumulated live parse output.
Attributes:
| Name | Type | Description |
|---|---|---|
format_version |
int
|
Accumulated layout revision, 8. |
provider_id |
ProviderId
|
Provider whose native observations are stored. |
built_at |
datetime
|
UTC store write instant. |
coverage |
tuple[CoverageInterval, ...]
|
Successfully retrieved series intervals and source retrieval instants. |
partition_row_counts |
Mapping[PartitionIdentifier, int]
|
Exact physical row counts keyed by product/year. |
CacheAuditResult
rivretrieve._internal.store.integrity.CacheAuditResult
dataclass
CacheAuditResult(
provider_id: ProviderId,
path: Path,
generation_id: str,
checked_at: datetime,
partitions_checked: int,
rows_checked: int,
bytes_checked: int,
)
A complete local integrity and semantic check of one generation.
A returned result means every managed file and observation passed its local
checks. It does not repeat certification against publisher artifacts.
bytes_checked counts the sealed metadata and partition bytes hashed.
checked_at is the UTC completion instant.
CacheRecoveryResult
rivretrieve._internal.store.reader.CacheRecoveryResult
dataclass
CacheRecoveryResult(
provider_id: ProviderId,
path: Path,
status: StoreStatus,
actions: tuple[str, ...],
)
The resulting local state and actions taken by explicit recovery.
Attributes:
| Name | Type | Description |
|---|---|---|
provider_id |
ProviderId
|
Provider whose local storage was inspected and recovered. |
path |
Path
|
Resolved canonical observation-store path. |
status |
StoreStatus
|
Resulting metadata-checked state. Recovery validates any committed generation it preserves or restores before discarding recovery inputs. |
actions |
tuple[str, ...]
|
Stable readable descriptions of preserved, restored or removed paths. |
CacheClearResult
rivretrieve._internal.bulk.CacheClearResult
dataclass
CacheClearResult(
provider_id: ProviderId,
path: Path,
existed: bool,
bytes_freed: int,
removed_paths: tuple[Path, ...],
)
The paths and bytes removed by explicit cache deletion.
Attributes:
| Name | Type | Description |
|---|---|---|
provider_id |
ProviderId
|
Provider whose cache was cleared. |
path |
Path
|
Resolved observation-store path. |
existed |
bool
|
True if any recognized store or recovery path existed. |
bytes_freed |
int
|
Sum of removed file and symlink sizes, in bytes. |
removed_paths |
tuple[Path, ...]
|
Store and recognized recovery inputs removed by the call. |
Exception imports
Function Raises sections state the conditions. FatalContractError subclasses bypass the caller issue policy. Local I/O and bulk transfer failures can also propagate.
EmptySelectionError
rivretrieve._internal.discovery.EmptySelectionError
Bases: FatalContractError
Raised by fetch when a selection routes to no station and access route.
reason.code is no_match when the packaged evidence establishes that
nothing matches, or unresolved_inventory when it cannot settle the
request. reason also names the requested providers, stations and
access routes. No source request is made.
MultiProviderSelectionError
rivretrieve._internal.discovery.MultiProviderSelectionError
Bases: FatalContractError
Raised by fetch when a selection routes to several providers.
provider_ids names them. Use fetch_by_provider or narrow the
selection with pick(selection, provider=...). No source request is made.
UnknownProviderError
rivretrieve._internal.registry.UnknownProviderError
Bases: FatalContractError
UnknownStationError
rivretrieve._internal.selection.UnknownStationError
Bases: FatalContractError
UnknownProductError
rivretrieve._internal.selection.UnknownProductError
Bases: FatalContractError
RivRetrieveError
rivretrieve._internal.issues.RivRetrieveError
Bases: Exception
Base exception for RivRetrieve harness errors.
FatalContractError
rivretrieve._internal.issues.FatalContractError
Bases: RivRetrieveError
Raised when a request or internal contract cannot be satisfied.
These errors are raised whatever on_issue policy is chosen. Its
subclasses name specific causes, such as invalid requests or missing
credentials.
IssuePolicyError
rivretrieve._internal.issues.IssuePolicyError
Bases: RivRetrieveError
Raised when on_issue="raise" and warning or error issues exist.
issues holds those warning and error issues. The result they belong to
is not returned.
InvalidObservationRequestError
rivretrieve._internal.issues.InvalidObservationRequestError
Bases: FatalContractError
MissingCredentialError
rivretrieve._internal.issues.MissingCredentialError
Bases: FatalContractError
Raised before retrieval when a required credential is not set.
missing_by_provider maps each provider identifier to the missing
variable names. No source request is made.
MissingOptionalDependencyError
rivretrieve._internal.issues.MissingOptionalDependencyError
Bases: FatalContractError
ObservationsUnavailableError
rivretrieve._internal.issues.ObservationsUnavailableError
Bases: FatalContractError
Raised by fetch for a catalogue-only provider, such as za_dws.
The provider ships a station catalogue but no observations.
ObservationDataSchemaError
rivretrieve._internal.issues.ObservationDataSchemaError
Bases: FatalContractError
Raised when observation rows break the frame schema or contradict their facts.
The facts are the source-series definitions that the rows reference.
BulkOperationsUnavailableError
rivretrieve._internal.bulk.BulkOperationsUnavailableError
BulkArtifactCleanupRefusedError
rivretrieve._internal.bulk.BulkArtifactCleanupRefusedError
InsufficientDiskSpaceError
rivretrieve._internal.bulk.InsufficientDiskSpaceError
ObservationStoreRefusedError
rivretrieve._internal.store.validation.ObservationStoreRefusedError
Bases: FatalContractError
Raised when an existing local observation store is malformed or incompatible.
refusal records the kind (malformed or incompatible), the store
path, the provider and the defect. The store is left in place. Rebuild a
compiled store with download or remove an accumulated store with
clear_cache.
StoreCertificationError
rivretrieve._internal.store.certification.StoreCertificationError
Bases: RuntimeError
The publisher artifact or staged store failed certification.
StoreLifecycleError
rivretrieve._internal.store.lifecycle.StoreLifecycleError
StoreTransactionError
rivretrieve._internal.store.lifecycle.StoreTransactionError
StoreTransactionError(
original: BaseException,
*,
cleanup_errors: tuple[BaseException, ...] = (),
committed_path: Path | None = None,
transaction_id: str | None = None,
generation_id: str | None = None,
residue_paths: tuple[Path, ...] = (),
)
Bases: StoreLifecycleError
A failed operation, its cleanup failures and surviving paths.
committed_path identifies authoritative data, if any. original and
cleanup_errors retain exception objects rather than only their messages.
StorePostCommitCleanupError
rivretrieve._internal.store.certification.StorePostCommitCleanupError
StorePostCommitCleanupError(
original: BaseException,
*,
cleanup_errors: tuple[BaseException, ...] = (),
committed_path: Path | None = None,
transaction_id: str | None = None,
generation_id: str | None = None,
residue_paths: tuple[Path, ...] = (),
)
Bases: StoreTransactionError
Publication succeeded; cleanup failed and must not be treated as acquisition failure.
Frame schemas
Series inspection frame
| Column | Polars dtype |
|---|---|
provider_id |
String |
station_id |
String |
product_id |
String |
series_id |
String |
facts_id |
String |
identity_namespace |
String |
published_id |
String |
description |
String |
identity_origin |
String |
variant |
String |
requested_variants |
List(String) |
requested_series_ids |
List(String) |
requested_selector_kind |
String |
requested_selector_value |
String |
physical_match |
String |
admission |
String |
admission_reason |
String |
source_unit |
String |
source_unit_state |
String |
source_unit_evidence |
List(String) |
normalized_unit |
String |
identity_evidence |
List(String) |
unit |
String |
quantity |
String |
frequency |
String |
statistic |
String |
temporal_support |
String |
day_definition |
String |
timestamp_anchor |
String |
time_zone |
String |
vertical_reference |
String |
vertical_datum |
String |
quantity_state |
String |
frequency_state |
String |
statistic_state |
String |
temporal_support_state |
String |
day_definition_state |
String |
timestamp_anchor_state |
String |
time_zone_state |
String |
vertical_reference_state |
String |
vertical_datum_state |
String |
quantity_evidence |
List(String) |
frequency_evidence |
List(String) |
statistic_evidence |
List(String) |
temporal_support_evidence |
List(String) |
day_definition_evidence |
List(String) |
timestamp_anchor_evidence |
List(String) |
time_zone_evidence |
List(String) |
vertical_reference_evidence |
List(String) |
vertical_datum_evidence |
List(String) |
inventory_ids |
List(String) |
inventory_scope |
List(String) |
inventory_windows |
List(String) |
outcome_windows |
List(String) |
inventory_vintage |
List(String) |
inventory_status |
List(String) |
outcomes |
List(String) |
outcome_reasons |
List(String) |
Identity and facts are separate. Nullable facts carry explicit evidence states; admission and inventory are not completeness scores. Use to_bundle for lossless exports.
Station metadata summary
| Column | Polars dtype |
|---|---|
provider_id |
String |
station_id |
String |
station_name |
String |
latitude |
Float64 |
longitude |
Float64 |
crs |
String |
water_body_name_field |
List(String) |
water_body_name_value |
List(String) |
drainage_area_field |
List(String) |
drainage_area_value |
List(String) |
drainage_area_unit |
List(String) |
elevation_field |
List(String) |
elevation_value |
List(String) |
elevation_unit |
List(String) |
elevation_datum |
List(String) |
Source metadata frame
| Column | Polars dtype |
|---|---|
provider_id |
String |
station_id |
String |
source_field |
String |
source_scope |
String |
source_value |
String |
source_dtype |
String |
source_unit |
String |
state |
Enum(categories=['value', 'source_null', 'no_metadata']) |
attribute_role |
Enum(categories=['station_name', 'water_body_name', 'drainage_area', 'elevation']) |
support_fact |
String |
source_datum |
String |
source_datum_field |
String |
source_datum_dtype |
String |
datum_support_fact |
String |
See metadata above and station metadata for name alternatives, JSON decoding and absence states.
Observation frame
| Column | Polars dtype |
|---|---|
time |
Datetime(time_unit='us', time_zone=None) |
time_zone |
String |
station_id |
String |
product_id |
String |
series_id |
String |
facts_id |
String |
quantity |
String |
source_unit |
String |
unit |
String |
value |
Float64 |
Only value is nullable. Time precision can vary while remaining Datetime. The zone belongs to each row, not the timestamp dtype. See ObservationResult above for units and meanings.
Shipped software capabilities
The table uses the built-in manifest and declarations, not source probes. Credential names are requirements, not a readiness or successful-access test. Bulk retrieval needs explicit download() consent before a store exists.
Counts describe packaged inventory accounting only. They do not establish countrywide completeness, continuous history or present-day source access. Pair counts describe access routes, not concrete source-series counts or admission. See the provider handoff for ownership and coverage qualifications.
| Provider | Observation kind | Required credential variables | Stations | Available pairs | Unknown pairs | Unavailable pairs |
|---|---|---|---|---|---|---|
ba_fhmzbih |
live | none | 60 | 132 | 48 | 0 |
br_ana |
live | ANA_IDENTIFICADOR, ANA_SENHA |
17914 | 6 | 107478 | 0 |
ca_eccc |
bulk store | none | 8057 | 0 | 16114 | 0 |
ch_foen |
live | none | 246 | 0 | 738 | 0 |
cz_chmi |
live | none | 831 | 0 | 4155 | 0 |
fr_hubeau |
live | none | 7347 | 15283 | 5014 | 0 |
fr_hydroportail |
live | none | 6409 | 59 | 12759 | 0 |
jp_mlit |
live | none | 1023 | 0 | 4092 | 0 |
lt_lhmt |
live | none | 97 | 0 | 194 | 0 |
no_nve |
live | NVE_API_KEY |
4902 | 14847 | 0 | 29271 |
pl_imgw |
bulk store | none | 1301 | 0 | 3903 | 0 |
th_thaiwater |
live | none | 825 | 1096 | 554 | 0 |
usgs_nwis |
live | none | 26258 | 58421 | 0 | 99127 |
za_dws |
catalogue-only | none | 2905 | 0 | 8715 | 0 |
Packaged access coordinates
Access coordinates and declared units come from products.parquet. These are internal routes, not public physical filters or scientific authority. A provider listing a route does not imply that every station offers it. Use physical filters in find and inspect series for admission, units and facts. A product route does not select a preferred source variant.
| Provider | Product | Canonical unit |
|---|---|---|
ba_fhmzbih |
discharge_reported |
m3/s |
ba_fhmzbih |
stage_reported |
m |
ba_fhmzbih |
water_temperature_reported |
degC |
br_ana |
discharge_daily_mean_bruto |
m3/s |
br_ana |
discharge_daily_mean_consistido |
m3/s |
br_ana |
discharge_instantaneous |
m3/s |
br_ana |
stage_daily_mean_bruto |
m |
br_ana |
stage_daily_mean_consistido |
m |
br_ana |
stage_instantaneous |
m |
ca_eccc |
discharge_daily_mean |
m3/s |
ca_eccc |
stage_daily_mean |
m |
ch_foen |
discharge_reported |
m3/s |
ch_foen |
stage_reported |
m |
ch_foen |
water_temperature_reported |
degC |
cz_chmi |
discharge_daily_mean |
m3/s |
cz_chmi |
discharge_hourly_mean |
m3/s |
cz_chmi |
stage_daily_mean |
m |
cz_chmi |
stage_hourly_mean |
m |
cz_chmi |
water_temperature_daily_mean |
degC |
fr_hubeau |
discharge_daily_max |
m3/s |
fr_hubeau |
discharge_daily_mean |
m3/s |
fr_hubeau |
stage_daily_max |
m |
fr_hubeau |
water_temperature_reported |
degC |
fr_hydroportail |
discharge_instantaneous |
m3/s |
fr_hydroportail |
stage_instantaneous |
m |
jp_mlit |
discharge_daily |
m3/s |
jp_mlit |
discharge_hourly |
m3/s |
jp_mlit |
stage_daily |
m |
jp_mlit |
stage_hourly |
m |
lt_lhmt |
discharge_daily_mean |
m3/s |
lt_lhmt |
stage_daily_mean |
m |
no_nve |
discharge_daily_mean |
m3/s |
no_nve |
discharge_hourly_mean |
m3/s |
no_nve |
discharge_instantaneous |
m3/s |
no_nve |
stage_daily_mean |
m |
no_nve |
stage_hourly_mean |
m |
no_nve |
stage_instantaneous |
m |
no_nve |
water_temperature_daily_mean |
degC |
no_nve |
water_temperature_hourly_mean |
degC |
no_nve |
water_temperature_instantaneous |
degC |
pl_imgw |
discharge_daily |
m3/s |
pl_imgw |
stage_daily |
m |
pl_imgw |
water_temperature_daily |
degC |
th_thaiwater |
discharge_reported |
m3/s |
th_thaiwater |
stage_reported |
m |
usgs_nwis |
discharge_daily_mean |
m3/s |
usgs_nwis |
discharge_instantaneous |
m3/s |
usgs_nwis |
stage_daily_max |
m |
usgs_nwis |
stage_daily_mean |
m |
usgs_nwis |
stage_daily_min |
m |
usgs_nwis |
stage_instantaneous |
m |
za_dws |
discharge_daily_mean |
m3/s |
za_dws |
discharge_instantaneous |
m3/s |
za_dws |
stage_instantaneous |
m |