recon_gen.common.l2.v_overlay

BV.4.0/4.1 — v-overlay lifecycle orchestration.

The /training/ page operates on a <base>_v_* schema that’s a clone of the production <base>_* schema + the operator’s enabled plant set. Per docs/audits/bv_5_dual_prefix_spike.md:

• Session Start (create_or_refresh_v_overlay): ensures the v schema exists + clones data from base + refreshes v matviews. Does NOT call etl_hook directly — the route handler chains this with /etl/run when the operator clicks Session Start.

• Apply plants (apply_plants_to_v_overlay): emits + executes the enabled plant set into the v overlay + refreshes its matviews. Naive shape (clone-and-replay) for the vertical slice; DL.9 diff-only Apply lands in BV.4.4.

• Cleanup (drop_v_overlay): drops the v schema entirely.

The v-overlay’s prefix is always <cfg.db_table_prefix>_v — derived at runtime, not configured.

Functions

applied_state_read_sql(base_prefix)	SELECT the JSON-encoded applied-plant set from <v>_config_kv.
applied_state_write_sql(base_prefix, ...)	UPSERT the JSON-encoded applied-plant set into <v>_config_kv.
apply_plants(cfg, instance, enabled_plants, *)	BV.4.9 (DL.9) — diff-only Apply.
cleanup(cfg, instance)	Drop the v overlay schema.
clone_base_to_v_sql(base_prefix)	Pure data copy from base prefix tables → v overlay tables.
compute_apply_diff(current, new)	Pure diff between two {kind: form_values} maps.
create_v_overlay_sql(instance, *, ...)	Schema DDL for the v overlay.
drop_v_overlay_sql(instance, *, base_prefix, ...)	Drop every per-prefix object the v overlay created.
read_applied_state(cfg)	Read the persisted {kind: form_values} map from <v>_config_kv.
read_failed_kinds(cfg)	{kind: error_message} for plants whose plant_function or plant SQL raised in the last Apply.
read_last_apply(cfg)	CF.1 — last-Apply outcome for the kv-sourced banner.
read_session_metadata(cfg)	Session-start timestamp + L2 yaml mtime captured at Session Start (DL.14 staleness banner).
refresh_v_overlay_matviews_sql(instance, *, ...)	refresh_matviews_sql against the v overlay prefix.
session_metadata_l2_mtime_key()
session_metadata_session_start_key()	Constant exposing the session-start key name so render code can pull it off the dict without re-hardcoding.
session_start(cfg, instance, *[, ...])	Orchestrates Session Start (DL.10):
v_overlay_prefix(base_prefix)	The conventional <base>_v suffix per DL.3.

Classes

ApplyDiff(unchanged, to_add, to_remove)

DL.9 — what changed between currently-applied plant state and the newly-requested checkbox/form selection.

class recon_gen.common.l2.v_overlay.ApplyDiff(unchanged, to_add, to_remove)

Bases: object

DL.9 — what changed between currently-applied plant state and the newly-requested checkbox/form selection.

unchanged is in both with identical form-value fingerprints (no re-run needed). to_add is in the new selection but not currently applied (or with a changed fingerprint — treat as remove+add so the new fingerprint becomes truth without trying to surgically update one plant). to_remove is in current but not in the new selection (or with a changed fingerprint).

Empty to_remove is the fast-path signal — skip the clone, just run new plants against existing v data. Non-empty to_remove triggers the slow-path reclone+replay since INSERT-style plants can be surgically DELETE’d but DELETE-style plants (uncovered_*, dead_*) can’t be trivially undone; full reclone is the safe default.

Parameters:

• unchanged (frozenset[str])

• to_add (frozenset[str])

• to_remove (frozenset[str])

to_add: frozenset[str]

to_remove: frozenset[str]

unchanged: frozenset[str]

recon_gen.common.l2.v_overlay.applied_state_read_sql(base_prefix)

SELECT the JSON-encoded applied-plant set from <v>_config_kv.

Return type:

str

Parameters:

base_prefix (str)

recon_gen.common.l2.v_overlay.applied_state_write_sql(base_prefix, json_payload)

UPSERT the JSON-encoded applied-plant set into <v>_config_kv.

Return type:

str

Parameters:

• base_prefix (str)

• json_payload (str)

async recon_gen.common.l2.v_overlay.apply_plants(cfg, instance, enabled_plants, *, anchor=None, dev_log=None)

BV.4.9 (DL.9) — diff-only Apply. Reads the currently-applied state from <v>_config_kv and decides between two paths:

• Fast path (no kinds being removed / no fingerprint changes): keep the existing v overlay data, just run the newly-enabled plants. Skips the clone — Apply latency drops from ~clone+matview-refresh+N-plants to ~N-plants+matview-refresh.

• Slow path (something has to come out — either an unchecked kind or a kind whose form values changed): reclone base → v and replay every enabled plant. Safe default because DELETE- style plants (uncovered_*, dead_*) can’t be trivially undone.

Each entry in enabled_plants is the registry entry + the operator’s form values (the kwargs the plant_function expects).

BV.4.10.d — dev_log is a DevLogWriter | None callback that accumulates per-step events for the live-tail UI (mirrors session_start’s plumbing).

Return type:

None

Parameters:

• cfg (Config)

• instance (L2Instance)

• enabled_plants (Iterable[tuple[PlantKindEntry, Mapping[str, object]]])

• anchor (datetime | None)

• dev_log (object)

async recon_gen.common.l2.v_overlay.cleanup(cfg, instance)

Drop the v overlay schema. Base prefix untouched.

Return type:

None

Parameters:

• cfg (Config)

• instance (L2Instance)

recon_gen.common.l2.v_overlay.clone_base_to_v_sql(base_prefix)

Pure data copy from base prefix tables → v overlay tables.

Three base data tables (per step_2_wipe survey): - <base>_transactions → <base>_v_transactions - <base>_daily_balances → <base>_v_daily_balances - <base>_config_kv → <base>_v_config_kv

Matviews are NOT cloned — they get rebuilt by refresh_v_overlay_matviews_sql() against the cloned base data. Cheaper than copying every matview row + guarantees v matview rows are derivable from the v base tables (the v overlay is internally consistent, not a snapshot artifact).

Return type:

str

Parameters:

base_prefix (str)

recon_gen.common.l2.v_overlay.compute_apply_diff(current, new)

Pure diff between two {kind: form_values} maps. Tested directly without spinning up a DB.

Return type:

ApplyDiff

Parameters:

• current (Mapping[str, Mapping[str, str]])

• new (Mapping[str, Mapping[str, str]])

recon_gen.common.l2.v_overlay.create_v_overlay_sql(instance, *, base_prefix, dialect)

Schema DDL for the v overlay. Mirrors the base prefix’s schema one-for-one but under the _v suffix. Idempotent: callers should drop + recreate via drop_v_overlay_sql() first if the schema may already exist (Studio’s Session Start does).

Return type:

str

Parameters:

• instance (L2Instance)

• base_prefix (str)

• dialect (object)

recon_gen.common.l2.v_overlay.drop_v_overlay_sql(instance, *, base_prefix, dialect)

Drop every per-prefix object the v overlay created.

Return type:

str

Parameters:

• instance (L2Instance)

• base_prefix (str)

• dialect (object)

async recon_gen.common.l2.v_overlay.read_applied_state(cfg)

Read the persisted {kind: form_values} map from <v>_config_kv.

Returns empty dict when the v overlay doesn’t exist OR the state row is absent (no Apply has ever fired).

Return type:

dict[str, dict[str, str]]

Parameters:

cfg (Config)

async recon_gen.common.l2.v_overlay.read_failed_kinds(cfg)

{kind: error_message} for plants whose plant_function or plant SQL raised in the last Apply. Empty when no Apply has fired or all succeeded.

Return type:

dict[str, str]

Parameters:

cfg (Config)

async recon_gen.common.l2.v_overlay.read_last_apply(cfg)

CF.1 — last-Apply outcome for the kv-sourced banner. Three return states:

• None — kv unreachable (connect error / cursor error / JSON parse error / non-dict shape). Banner should render nothing under this state, NOT fall back to a stale optimistic claim.

• {} — empty dict, row exists with value ‘{}’ (Session Start has fired but no Apply yet). Banner renders nothing.

• populated dict with finished_at — render green/amber/red based on succeeded + failed.

Payload shape (last-Apply-wins, not cumulative): ``{“attempted”: [<kind>, …],

“succeeded”: [<kind>, …], “failed”: {<kind>: <ExcName: msg>, …}, “finished_at”: <ISO seconds, local TZ>, “path”: “fast” | “slow”}``

Return type:

dict[str, object] | None

Parameters:

cfg (Config)

async recon_gen.common.l2.v_overlay.read_session_metadata(cfg)

Session-start timestamp + L2 yaml mtime captured at Session Start (DL.14 staleness banner). Empty when no Session Start has fired.

Return type:

dict[str, str]

Parameters:

cfg (Config)

recon_gen.common.l2.v_overlay.refresh_v_overlay_matviews_sql(instance, *, base_prefix, dialect)

refresh_matviews_sql against the v overlay prefix.

Return type:

str

Parameters:

• instance (L2Instance)

• base_prefix (str)

• dialect (object)

recon_gen.common.l2.v_overlay.session_metadata_l2_mtime_key()

Return type:

str

recon_gen.common.l2.v_overlay.session_metadata_session_start_key()

Constant exposing the session-start key name so render code can pull it off the dict without re-hardcoding.

Return type:

str

async recon_gen.common.l2.v_overlay.session_start(cfg, instance, *, refresh_base=True, l2_yaml_path=None, dev_log=None)

Orchestrates Session Start (DL.10):

1. Optionally invoke run_deploy_pipeline (the /etl/run flow) against the base prefix — fresh upstream data + matview refresh on <base>_*. Skipped when refresh_base=False (the Re-clone button uses this).

2. Drop v overlay schema (idempotent — silently no-ops if absent).

3. Create v overlay schema.

4. Clone base → v overlay (data tables only).

5. Refresh v overlay matviews.

The /etl/run leg uses the TRAINER_CLEAN overlay (baseline only, no plants) since the operator’s plant choices live on the v overlay — not the base.

BV.4.10.d — dev_log is a DevLogWriter | None callback that accumulates per-step events for the live-tail UI. None silences progress events (CLI / test callers); supplying it makes the Studio’s /training/session-start/stream endpoint useful.

Return type:

None

Parameters:

• cfg (Config)

• instance (L2Instance)

• refresh_base (bool)

• l2_yaml_path (object)

• dev_log (object)

recon_gen.common.l2.v_overlay.v_overlay_prefix(base_prefix)

The conventional <base>_v suffix per DL.3.

Single function so callers don’t string-concatenate ad hoc — keeps a future rename atomic.

Return type:

str

Parameters:

base_prefix (str)