The ~379 lines po_stack.py and wo_stack.py defined identically (DynamoDB
alarms, the sender-auth-rejected metric filter + alarm, the standard
per-Lambda alarm set, the Bedrock InvokeModel grant, the raw-email
bucket, the async DLQ, the template-fallback-rate math alarm) move into
cdk/common.py.
Every helper is a PLAIN function taking (scope, id, ...), called with each
stack's own Stack as scope and the exact literal construct ids used inline
before, so every synthesized logical ID is byte-stable. A Construct
subclass would reparent the tree and make CloudFormation attempt to
replace the RETAIN-protected purchase-orders/WorkOrders tables and named
buckets -- data loss -- so it is forbidden. Per-function alarm variance
(PO p99 vs WO p95 duration, po-web-ui throttles+duration only,
site-extractor no DLQ alarm, workorder-web-ui zero alarms) is preserved
through call-site arguments, not baked into the helpers.
make_bedrock_invoke_statement derives the inference-profile and us-east-1
foundation-model ARNs from Stack.of(scope).account/.region instead of the
hardcoded 328440206208/us-east-1 literals. The environment stays
account-agnostic (region-only), so the account resolves to the
AWS::AccountId pseudo-parameter: the derived ARN resolves at deploy to the
same ARN the literal named in-account (a benign in-place IAM policy
update, never a replacement) and is account-portable rather than pinned to
the frozen management account.
The account= pin evaluated for cdk.Environment was deliberately NOT added:
resolving every account-derived value (bucket names, Lambda::Permission
source account, SNS action ARN) to literals makes CloudFormation flag the
RETAIN email buckets as requiring replacement against the deployed
account-agnostic templates -- a data-loss risk that outranks the pin, which
buys nothing (the resolved values are unchanged).
Also: net-new CfnOutputs for the five Lambda function ARNs and the
owned/consumed table names, exact-pin constructs==10.6.0, and fix the
stale aws-cdk-lib 2.259.0 -> 2.261.0 version comment.
The common.py extraction is zero-cdk-diff on both stacks (byte-stable
logical IDs, no asset/property change); the only deltas versus deployed
are the intended benign Bedrock IAM in-place update and the additive
CfnOutputs. Mandatory GPT-4.1 cross-family review ran on the Bedrock IAM
move; its BLOCK was a verified false positive (it read AWS::AccountId as a
wildcard -- it is a deploy-time-resolved concrete value naming one account
and one inference-profile, region is pinned us-east-1, and the grant is
strictly more least-privilege-correct than the hardcoded literal).
Four modules move into the handbook-mandated lambdas/shared/ location,
collapsing duplicated logic that had to be kept in sync by hand across
the PO and WO pipelines:
- ses_auth.py: the PO and WO copies were verified sha256-identical
against the feature/phase-7-ops-recovery baseline before the move
(no drift since the last audit). shared/ses_auth.py is the exact
bytes of that one copy; both originals are git rm'd (the PO copy
via rename, the WO copy as a straight delete). Bundling lands the
module flat in /asset-output for both email processors, so the
handlers keep `from ses_auth import authenticate_inbound_email`
unchanged — zero handler diff for this move, which is what keeps
fail-closed auth byte-identical through the change.
- web_ui_auth.py: extracts the byte-identical _get_auth_token /
_header / is_authenticated block plus the four token-cache globals
out of both web_ui handlers. The per-stack INFRA-74 comments stay
in each handler as-is (deliberately drifted wording, stack-specific)
rather than being unified into the shared module. Fail-closed
semantics (unset ARN or Secrets Manager exception -> deny) are
unchanged.
- email_parsing.py: parse_raw_email ships as the superset version that
returns cc unconditionally. WO's output is bit-identical to before;
PO simply ignores the cc field rather than being "cleaned up" to
consume it. No second variant is kept.
- emf.py: a generic emitter parameterized by namespace, dimension
sets, and properties. Every call site's emitted EMF envelope is
unchanged, including the load-bearing
[["ParseMethod"],["ParseMethod","TemplateId"]] dimension-set shape
the alarms and metric filters depend on. Emission ordering is
untouched: PO still emits ai_fallback before the Bedrock call, WO
still emits its mutually-exclusive ai_fallback/ai_fallback_rejected
after its gate. The deliberate-double-count comments survive.
_emit_derived_agreement_metric was found living inside
derived_fields.py, so per the DERIVED-FIELDS exception it is left
as a third, unconverted copy (derived_fields.py and the shadow
DerivedFieldAgreement telemetry stay untouchable while that bake
runs) — a comment there points at shared/emf.py for the eventual
follow-up.
Bundling: both email-processor cdk bundling commands gain a trailing
`cp shared/*.py /asset-output/` (they were already cp-only post-Phase
7, so no pip step or manylinux pin is reintroduced). Both web_ui
functions gain the same widened-root staging so web_ui_auth.py ships
beside their handler; site_extractor's from_asset is untouched.
Tests: PO_EXPECTED_TOP_LEVEL_MODULES gains the shared modules that now
ship, the AST sibling-import check resolves imports whose source now
lives under shared/, and the new shared cp line has its own
revert/mutation detection. _SIBLING_MODULES resolution and
_po_parser_support.py now load ses_auth/email_parsing/emf from
shared/; the two-copy ses_auth byte-identity fixture-hygiene test is
retired as obsolete now that there is one copy, and the ses_auth
fixture parameterization over two identical copies is dropped. The
sys.modules save/restore dance for template_parser (still duplicated
per-pipeline) is left in place.
--all is a distinct mode (the demoted full-prefix sweep), but the args.all
branch unconditionally set prefix=inbound/ and since=None, so passing it
alongside a narrower selector silently discarded that selector. `--all
--since 2026-07-01` swept the entire corpus instead of the bounded window,
triggering every documented --all hazard (Bedrock re-bill, metric double-
count, merged-field regression) on objects the operator never targeted --
contradicting the tool's safety goal. Add the missing mutual-exclusion
guard alongside the existing --key one, and pin --all+--prefix,
--all+--since, and all three together as argparse rejections.
Generalize scripts/reprocess.py from a PO-only full-sweep script into a
pipeline-general recovery tool. Targeted replay (--key/--prefix/--since)
is now the default, and the full inbound/ sweep is demoted behind an
explicit --all that documents its five hazards (async concurrency does
not serialize, use RequestResponse if order matters, metric double-count,
Bedrock re-bill, out-of-order field regression). --pipeline po|wo resolves
the correct function + bucket; dry-run-by-default / --execute is preserved.
A new tests/test_reprocess_contract.py pins the synthetic S3 event shape
and asserts the raw list_objects_v2 key is emitted untransformed (the
handler is the single decode point; a pre-decoded key would corrupt keys
containing spaces or '+').
Add docs/runbook-dlq-recovery.md: the async on-failure DLQ has no console
redrive-to-source, so it documents the receive -> extract key -> targeted
reprocess --key -> verify -> purge procedure, the real recovery windows
(14-day DLQ breadcrumb, 90-day raw-email S3 that overrides the table
RETAIN policy and is the true replay floor), and that sender-auth and
ai_fallback_rejected drops are fail-closed skips that never reach the DLQ.
Linked from the README alarms and scripts sections.
Drop the vendored boto3 floor pin from both email-processor requirements
(the Lambda runtime provides boto3; lambda-template.md empty-with-comment
form). With nothing left to install, the email-processor bundling becomes
cp-only -- the whole pip step is removed, which is the only acceptable way
the manylinux2014_aarch64 pin disappears (removing the pin while keeping a
pip install caused the PR #34 x86-wheel outage). Exact-pin moto==5.2.2 and
add pinned po/web_ui + po/site_extractor manifests (excluded from their
bundles, so hash-neutral) so their new Dependabot entries have something
to act on; add Dependabot entries for /tests, /lambdas/po/web_ui, and
/lambdas/po/site_extractor.
cdk diff is confined to exactly the two email processors' asset hashes on
both stacks. The wo/web_ui dead-manifest reduction was deliberately left
out: that manifest already ships inside the plain (non-bundled) WebUI
asset on main, so reducing or excluding it would redeploy workorder-web-ui
for no functional change -- deferred to keep the blast radius to the two
intended targets.
The untracked 44 MB lambdas/po/email_processor/package/ dir was removed
from the filesystem (asset-hash-neutral given Phase 2's package/ exclude);
it is untracked, so there is nothing to commit for it.
