recon_gen.common.provenance

Audit-grade provenance fingerprint primitives.

Binds a generated artifact (e.g. an audit PDF) to its source data via SHA256 over base table rows + L2 yaml file bytes + code identity. Designed to be reusable: cli/audit.py is the first consumer (audit apply to compute + embed; audit verify to recompute + diff), but any future tool that wants a reproducibility binding for its output can pull this module in.

Locked design (Phase U.7):

• Hash the base tables + external inputs, NOT matviews. Matviews are derived data; their hash drifting from a recompute is a technical signal (matviews need refresh, schema drift) but isn’t authoritative for “what was this report bound to”.

• Per-table column set is discovered at runtime via cur.description (DB-API 2.0) and sorted alphabetically by lowercased name. Hardcoded column lists were a footgun: a new column added to a base table would silently be excluded from the hash, producing a fingerprint that claimed “binds to all source data” while missing whatever the new column carried.

• Composite fingerprint = SHA256 over the per-source values concatenated in a fixed order (each on its own labeled line). short form (footer) = first 8 hex chars; composite_sha = full 64.

Functions

canonical_value(v)	Stable bytes repr for one cell value when hashing rows.
compute_provenance(cfg, instance, *, ...)	Compute the report's full provenance fingerprint.
hash_matview_rows(cur, *, matview)	SHA256 + row count over every row in a matview.
hash_table_rows(cur, *, table, hwm)	SHA256 over canonical row bytes for WHERE entry <= hwm.
l2_fingerprint_placeholder()	Long-form fingerprint placeholder for the no-DB code path.
l2_yaml_sha256(l2_instance_path)	SHA256 of the L2 YAML file bytes (verbatim, no normalization).
recon_gen_code_identity(version)	Code identity string baked into the fingerprint.
short_fingerprint_placeholder()	Short-form fingerprint placeholder for the per-page footer.

Classes

ProvenanceFingerprint(transactions_hwm, ...)

The four base inputs that fully determine a generated artifact.

class recon_gen.common.provenance.ProvenanceFingerprint(transactions_hwm, transactions_sha, balances_hwm, balances_sha, l2_yaml_sha, code_identity)

Bases: object

The four base inputs that fully determine a generated artifact.

Locked per U.7: hash the base tables (transactions + daily_balances) bounded by their high-water-mark entry ids, plus the L2 instance YAML and the code identity. Matviews are deliberately excluded — they’re derived data; a fingerprint over them would conflate “the source data changed” with “we recomputed the matview SQL differently”, and the auditor needs to bind the report to the AUTHORITATIVE source.

composite_sha is the SHA256 of the per-source values concatenated in a fixed order; short is the first 8 hex chars (footer). The dict-form serializes to JSON for embedding in PDF metadata so audit verify can recompute and compare.

Parameters:

• transactions_hwm (int)

• transactions_sha (str)

• balances_hwm (int)

• balances_sha (str)

• l2_yaml_sha (str)

• code_identity (str)

balances_hwm: int

balances_sha: str

code_identity: str

property composite_sha: str

classmethod from_dict(d)

Return type:

ProvenanceFingerprint

Parameters:

d (dict[str, Any])

l2_yaml_sha: str

property short: str

to_dict()

Return type:

dict[str, str | int]

transactions_hwm: int

transactions_sha: str

recon_gen.common.provenance.canonical_value(v)

Stable bytes repr for one cell value when hashing rows.

Cross-dialect goal: PG and Oracle return the same logical row as the same bytes here. Decimal via str() keeps trailing zeros + sign; date/datetime via isoformat() is timezone-naive (matches our schema convention); bool is coerced to "1"/"0" since Oracle returns ints for booleans where PG returns Python bools; None is empty string (distinct from the field separator).

Return type:

bytes

Parameters:

v (object)

recon_gen.common.provenance.compute_provenance(cfg, instance, *, l2_instance_path, version)

Compute the report’s full provenance fingerprint.

Returns None when demo_database_url is not configured — the artifact then renders with the long-form <pending> placeholder (skeleton mode). Reads MAX(entry) for both base tables, hashes the rows up to those high-water marks, hashes the L2 YAML file bytes, captures the code identity, and bundles everything into a ProvenanceFingerprint whose composite_sha binds the artifact to its inputs.

Return type:

ProvenanceFingerprint | None

Parameters:

• cfg (Config)

• instance (L2Instance)

• l2_instance_path (str | None)

• version (str)

recon_gen.common.provenance.hash_matview_rows(cur, *, matview)

SHA256 + row count over every row in a matview.

Distinct from hash_table_rows (which is bounded by an entry high-water-mark, since base tables are append-only and we want a stable snapshot point). Matviews don’t have entry; they’re recomputable from base tables and we want the SHA256 to represent “what the matview contained at audit time”. So this helper just hashes ALL rows in a deterministic order.

Determinism: discover columns alphabetically by lowercased name (same convention as hash_table_rows — works portably on PG + Oracle), pull all rows into memory as canonicalized tuples, sort by tuple-lex (works for any matview without needing to know its natural key), then stream into SHA256. Matviews are bounded (~tens to hundreds of rows in practice) so memory is fine. Returns (row_count, sha256_hex) so the appendix can show both side by side.

Return type:

tuple[int, str]

Parameters:

• cur (Any)

• matview (str)

recon_gen.common.provenance.hash_table_rows(cur, *, table, hwm)

SHA256 over canonical row bytes for WHERE entry <= hwm.

Column set is discovered at runtime from cur.description (DB-API 2.0 standard, works for both psycopg2 and oracledb) and sorted alphabetically by lowercased name. This avoids the footgun where a hardcoded column list would silently exclude new columns added to the base table from the fingerprint — producing a hash that claims “this binds to all source data” while missing whatever the new column carries.

Lowercasing before sorting makes the order portable across Postgres (returns lowercase identifiers) and Oracle (returns UPPERCASE for unquoted identifiers).

Streams results so memory stays flat regardless of row count. Field separator: \x1f (unit separator). Row separator: \x1e (record separator). Both are control codes that can’t appear in our schema’s data types, so we don’t need to escape them.

Return type:

str

Parameters:

• cur (Any)

• table (str)

• hwm (int)

recon_gen.common.provenance.l2_fingerprint_placeholder()

Long-form fingerprint placeholder for the no-DB code path.

Used on the cover-page provenance block + sign-off page when an artifact ran without a DB connection configured (skeleton mode — no DB queries, no real fingerprint to compute). When the DB is wired the renderers receive a ProvenanceFingerprint and substitute its real composite_sha instead.

Return type:

str

recon_gen.common.provenance.l2_yaml_sha256(l2_instance_path)

SHA256 of the L2 YAML file bytes (verbatim, no normalization).

When the user passed --l2 path, hash that file. When they didn’t (audit ran against the bundled default), hash the packaged spec_example.yaml bytes via the shared accessor so the fingerprint is still deterministic for the no-flag case.

Return type:

str

Parameters:

l2_instance_path (str | None)

recon_gen.common.provenance.recon_gen_code_identity(version)

Code identity string baked into the fingerprint.

Prefer v{version}+g{git_short} when running from a git checkout (carries both the released version AND the precise commit). Fall back to just v{version} when git isn’t available (pip-installed package, no .git dir nearby) so the fingerprint stays deterministic for distributed installs.

Return type:

str

Parameters:

version (str)

recon_gen.common.provenance.short_fingerprint_placeholder()

Short-form fingerprint placeholder for the per-page footer.

Distinct compact stand-in (vs the long-form <pending>) so a sweep that resolves one when fingerprints land doesn’t accidentally rewrite the other.

Return type:

str