Procurement Ingest

Unified email ingestion pipelines for Amazon procurement data. Two independent pipelines — purchase orders (Coupa) and work orders (APM/Hexagon EAM) — share a single repo and CDK app but deploy as separate CloudFormation stacks.

Pipelines

Purchase Orders (po-ingest stack)

Coupa PO emails are received at amazon_po@int.seahaven.com, parsed deterministic-template-first with a Claude-Haiku-4.5-on-Bedrock fallback, and written to the purchase-orders DynamoDB table.

Flow:

1. Coupa sends a PO email (new, revision, or cancellation).
2. SES (INBOUND_MAIL rule set) drops the raw MIME into s3://po-ingest-emails-{AccountId}/inbound/.
3. S3 ObjectCreated triggers the po-email-processor Lambda.
4. Fail-closed sender authentication (INFRA-107): the SES-stamped Authentication-Results header must show dkim=pass for amazon.coupahost.com (see Sender authentication); otherwise the email is logged and dropped.
5. Parse: a pure, offline template parser (template_parser.py) tries the two known Coupa templates first — coupa_new_po (95.5% of traffic) and coupa_cancellation (2.9%) — behind a strict fail-closed validation gate. Only on a miss/invalid result does the Lambda fall back to the Claude-on-Bedrock AI extractor (InvokeModel), which extracts the identical structured-JSON contract (PO number, status, supplier, nested ship-to, line items). Numeric amounts are Decimal on both paths (the AI decode uses parse_float=Decimal; DynamoDB rejects floats). The derived classification fields (site_code, trade, fiscal_year) are computed by a deterministic Python classifier in the shared enrich_parsed() post-stage (zip padding, ship_to_raw/state promotion, metadata) which runs identically on both paths — Python-authoritative on the template path and an LLM-authoritative-with-Python-shadow bake on the AI-fallback path (see the Derived fields note below).
6. Merge write to DynamoDB:
   • new_po — merge insert. Creates the PO, or backfills data into a pre-existing Cancelled skeleton left by an out-of-order cancellation (preserving the Cancelled status). No longer silently dropped when a record already exists.
   • revision — field-level merge (update_item SETs only the fields present in the revision). A revision that omits line_items/supplier no longer deletes them. Will not un-cancel a Cancelled PO.
   • cancellation — marks the row Cancelled (creating a minimal skeleton if the cancellation arrives before the new_po).
7. DynamoDB Streams feeds the site extractor (po-ingest-site-extractor) — real-time site address extraction into the verified-sites table. (LedgerFlow / seahaven-slack-bot/po-sync, the former second stream consumer, was decommissioned 2026-07-23.)

Deterministic template parser. template_parser.try_deterministic_parse() classifies by exact subject regex, extracts the nested contract (supplier{}, ship_to{} — 8 keys, line_items[] — 10 keys per item), and returns a parsed result only if it passes a fail-closed validation gate: recursive exact key-set at every nesting level; email_type emitted only on the exact new-PO subject and a confirmed-safe Status (never defaulted — a cancellation misrouted as new_po would defeat the sticky-Cancelled guard); po_number shape + byte-equality with the subject, the body PO ID, the Amazon Purchase Order # heading, and the orders/<id> URL; duplicate-label anchor integrity (Supplier/Shipping/Total each appear twice, the first Shipping must be the literal None placeholder); money fidelity (every Decimal re-serializes byte-identically to its source token with a digit/comma border check — the thousands-separator-truncation kill switch — plus sum(line amounts) == total, proven against both Total blocks); bullet-metadata label discipline (closed label set, assigned by leading label, never ordinal — immune to the optional Part Number segment); USPS address shape on the raw pre-enrichment value; and rejection of any unparseable sentinel or residual \r/\xa0 artifact. Multi-line-item (0.18%) and non-USD (0 observed) new-POs, comment emails, and anything else falls back to the AI extractor. The AI path is gated too: the untrusted email reaches Bedrock inside a neutralized <email> data block (forged tag lookalikes in the body are defanged) with temperature=0, and the raw model output must pass the fail-closed validate_ai_fallback() gate — the PO-specific nested contract (exact key-set at every level, with missing keys normalized rather than rejected), a po_number shape check hardened against fullwidth-digit and trailing-artifact injection (the same regex family protecting the DynamoDB partition key the handler builds from it), an email_type allow-list enforced before dispatch so a miss can never fall into the new_po default branch, and Decimal/int/None money typing (PO decodes with parse_float=Decimal) — before any DynamoDB write. The template path is gate-enforced end-to-end; the AI-fallback path is validated and fail-closed — output that fails the gate is skipped, never written (see ai_fallback_rejected below), so a malformed or injected email raises the fallback rate rather than corrupting a record. Every record emits one CloudWatch EMF metric (see below).

Derived fields (site_code, trade, fiscal_year). These three are classified, not verbatim-extracted, so neither parse path emits them directly — instead a deterministic Python classifier (derived_fields.derive_all()) computes them inside the shared enrich_parsed() post-stage, which runs identically on both paths. derived_fields.py is byte-frozen this phase (shadow-bake freeze, per the constraint-9 module-diff gate); note for accuracy that its in-file docstring's "FAITHFUL v1 port" self-description is aspirational, not descriptive — the module is a deliberate spec-superset of the original EXTRACTION_PROMPT rule text (its own inline backtest-tuning comments diverge from those three prompt sections), with the LLM staying runtime-authoritative on the AI-fallback path during the bake regardless. The docstring reword is deferred to the first post-bake PR, since even a text-only edit is barred by this phase's machine-checked empty-diff gate on the file. trade is a closed label set (the same one the AI prompt enumerates): Plumbing - PM, Plumbing - Reactive, Electrical, HVAC, Dock Doors, Doors, Signage, Carpentry, Fencing/Gates, Conveyance/MHE, Painting, Flooring, Janitorial, Fire/Life Safety, Landscaping/Yard, Roofing, Security/Locksmith, Snow Removal, PO Uplift, General Building - Emergency, General Building - Handyman, General Building - Project, General Building. Authority differs by path:

• Template path — the parser leaves all three null, so Python is authoritative: it fills them.
• AI-fallback path — the LLM value stays authoritative during the bake (Python fills only a gap the LLM left null, never overwrites), and Python additionally runs in shadow mode: enrich_parsed() emits one DerivedFieldAgreement EMF record per field comparing the two.

DerivedFieldAgreement metric — namespace Seahaven/PoIngest, metric name DerivedFieldAgreement (Unit Count, value 1), dimensioned only by Field × Agreement (cardinality fixed at 3 × 4). Agreement ∈ {agree (both non-null, equal after str-strip), disagree (both non-null, different), llm_null_python_filled (LLM null, Python supplied a value), python_null (LLM non-null, Python null)}; the record is skipped entirely when both are null. po_number, PythonValue, and LlmValue ride along as non-dimensioned Logs-Insights properties so a disagreement can be reviewed by example without inflating cardinality. Only the AI-fallback path emits these — the template path has no LLM value to shadow.

Review disagreements during the bake with this CloudWatch Logs Insights query over /aws/lambda/po-email-processor:

fields @timestamp, Field, Agreement, LlmValue, PythonValue, po_number
| filter ispresent(DerivedFieldAgreement)
| filter Agreement = "disagree"
| sort @timestamp desc
| limit 200

Or aggregate overall agreement per field: | filter ispresent(DerivedFieldAgreement) | stats count(*) by Field, Agreement.

Post-bake follow-up: once agreement is acceptable, make Python authoritative on the AI-fallback path too (stop keeping the LLM value) and drop the site_code/trade/fiscal_year rule sections from EXTRACTION_PROMPT — the LLM stays authoritative on the fallback path only until then.

Lambdas (lambdas/po/):

Function	Trigger	Purpose
po-email-processor	S3 ObjectCreated	Claude extraction + DynamoDB write
po-ingest-site-extractor	DynamoDB Streams	Site code/address extraction -> verified-sites
po-web-ui	Manual invoke (authenticated — see Setup §6)	HTML dashboard (public Function URL removed 2026-06-08, INFRA-74)

Tables:

• purchase-orders (PK: po_number, Streams: NEW_IMAGE) — read by procurement-api (see Shared Resources; the former seahaven-slack-bot reader was decommissioned 2026-07-23)
• verified-sites (PK: siteCode) — ~1,100 unique Amazon facility sites (by-state GSI removed 2026-06-03, audit M-20)
• pending-site-review (PK: po_number) — unresolvable POs for manual Payee Central verification

Work Orders (WorkorderIngestStack stack)

Amazon APM work order emails (from Hexagon EAM / HxGN SmartCloud) are received at apm@int.seahaven.com, parsed deterministic-template-first with a Claude-on-Bedrock fallback, and written to the WorkOrders DynamoDB table.

Flow:

1. Hexagon EAM sends email notifications (new assignments, comments, updates, cancellations) to amazon@seahavenind.com.
2. Gmail filter forwards APM emails to apm@int.seahaven.com (SES).
3. SES drops the raw MIME into s3://workorder-ingest-emails-{AccountId}/inbound/.
4. S3 triggers the workorder-email-processor Lambda.
5. Fail-closed sender authentication (INFRA-107): the SES-stamped Authentication-Results header must show dkim=pass for the domain that re-signs the forward (currently allowlisted as seahaven.com — see the validation caveat under Sender authentication); otherwise the email is logged and dropped.
6. Parse: a pure, offline template parser (template_parser.py) tries the two known Hexagon templates first, behind a strict fail-closed validation gate. Only on a miss/invalid result does the Lambda fall back to the Claude-on-Bedrock AI extractor. Both paths emit the identical structured-JSON contract (work order ID, site code, severity, priority, dates, assigned technician).
7. Work order upserted to WorkOrders, event/comment appended to WorkOrderComments.

Deterministic template parser. ~93.6% of WO traffic is the plain-text "AMAZON UPDATE WO DETAILS <id>" comment template (T1) and ~6.4% is the HTML "AMAZON assign Work Order <id> on building <SITE>" assignment template (T2). template_parser.try_deterministic_parse() classifies by subject, extracts the contract fields, and returns a parsed result only if it passes a strict validation gate (exact contract-key set; work_order_id matches the subject and is all-digits; the T1 Work Order: <id> double-space is literally present; email_type matches the template; site_code shape; per-type required fields; and a label-bleed guard so a value that over-ran into another field fails). Anything that fails — the rare update/cancellation shapes, Hexagon template drift, or an extractor exception — falls back to the AI extractor. The AI path is gated too: the untrusted email reaches Bedrock inside a neutralized <email> data block (tag lookalikes in the body are defanged), and the raw model output must pass the fail-closed validate_ai_fallback() schema/enum/date gate before any DynamoDB write — output that fails is dropped and paged (see the ai-fallback-rejected alarm below), a prompt-injection defence for DKIM-passing but attacker-influenced mail. Data is never corrupted; only the fallback rate rises. Every record emits one CloudWatch EMF metric (see below).

Lambdas (lambdas/wo/):

Function	Trigger	Purpose
workorder-email-processor	S3 ObjectCreated	Claude extraction + DynamoDB write
workorder-web-ui	Manual invoke (authenticated — see Setup §6)	HTML dashboard (public Function URL removed 2026-06-08, INFRA-74)
workorder-shoc-emitter	DynamoDB Streams, both WO tables (ESMs ship disabled — see SHOC webhook feed)	HMAC-signed webhook push of every WO mutation to the SHOC backend
workorder-shoc-hmac-rotator	Secrets Manager rotation schedule (30 days)	Rotates the webhook HMAC signing keys (dual-key overlap)

Tables:

• WorkOrders (PK: work_order_id, Streams: NEW_AND_OLD_IMAGES) — site-code-index and status-index GSIs removed 2026-06-03 (audit M-20)
• WorkOrderComments (PK: work_order_id, SK: comment_id, Streams: NEW_AND_OLD_IMAGES) — see the comment_id format note below

SHOC webhook feed (workorder-shoc-emitter)

Flow: WorkOrders / WorkOrderComments DynamoDB Streams (NEW_AND_OLD_IMAGES — the OLD image is what lets the emitter detect the wo_status → cancelled transition) → workorder-shoc-emitter → HMAC-signed HTTPS POST → SHOC backend. Every WO mutation becomes one webhook event (work_order.created / .updated / .cancelled / .comment_added) within seconds of the DynamoDB commit; DynamoDB stays the source of truth.

• Contract: docs/shoc-webhook-contract.md (Rev 2026-07-23) is the producer/consumer contract SHOC builds its receiver against, and docs/shoc-webhook-test-vectors.json is the shared receiver-verification vector set — both sides pin their HMAC implementation against the same vectors (producer-side via the golden-vector tests over delivery.sign_body).
• ACTIVE since 2026-07-30. Both event-source mappings run with enabled=True, flipped after the SHOC receiver on api.dev.seahaven.com passed the shared test vectors live (valid current-kid signature accepted, duplicate delivery_id deduplicated, tampered/stale/unknown-kid all rejected 401). The stack originally shipped dark (enabled=False) so it could deploy and be tested with zero deliveries while SHOC had no receiver. The ESMs start at LATEST — no historical flood; SHOC backfills history through the procurement-api read API, not the stream.
• Ordering/retry semantics. parallelization_factor=1, bisect_batch_on_error=False, retry_attempts=-1, maximum_record_age=24h: a retryable failure (429/5xx/timeout/connection error) blocks the shard and retries from the failed record — per-work-order commit order is the guarantee, and blocking is the intended behavior when SHOC is down. report_batch_item_failures keeps earlier in-batch successes from being re-delivered. Records that exhaust the 24h age are parked as ESM failure metadata (not full records) on workorder-shoc-emitter-failures (on_failure destination); a non-retryable 4xx (a contract bug, never worth blocking the shard for 24h) parks the full {envelope, response_status} payload on workorder-shoc-emitter-rejected and the loop continues. Both queues: 14-day retention, SSL-enforced, alarmed (see CloudWatch alarms); recovery is scripts/replay_shoc_webhooks.py (see Scripts).
• Secret + KMS. The HMAC signing keys live in Secrets Manager secret workorder-ingest/shoc-webhook-hmac (value {"keys": [{"kid", "secret"}, ...]}, newest first, max 2), encrypted with the dedicated CMK workorder-ingest-shoc-webhook-kms — deliberately not alias/seahaven-dynamodb, so the SHOC cross-account grant's decrypt reach covers exactly this one secret and nothing else. The secret's removal policy is DESTROY, deliberately not RETAIN: the value is machine-generated HMAC material, fully regenerable by a single rotation, so RETAIN buys nothing and would expose the fixed-name RETAIN-orphan deadlock (a failed create orphans an empty shell holding the global name; every later create fails AlreadyExists). Accidental-deletion recovery runbook: redeploy to recreate the secret, force a rotation (aws secretsmanager rotate-secret --secret-id workorder-ingest/shoc-webhook-hmac), notify the SHOC team — receivers re-fetch within their ≤5-minute cache TTL, so no coordination window is needed — then watch the -failures queue and replay the gap with the replay script.
• Rotation. workorder-shoc-hmac-rotator runs on a 30-day schedule: it prepends a fresh 64-hex-char key as keys[0] and truncates the list to 2 entries (one overlap cycle). kid format is YYYY-MM-DDTHH. The emitter always signs with keys[0] behind a 5-minute TTL cache; the receiver accepts any listed kid and re-fetches on an unknown one — there is no delivery window in which signatures can't verify.
• Cross-account grants (exact ARN only): arn:aws:iam::396287094661:role/shoc-backend-dev is granted secretsmanager:GetSecretValue on the secret's resource policy and kms:Decrypt on the CMK's key policy — both halves are required; either one alone fails silently at the receiver. Future staging/prod receiver roles are each a deliberate, individually-reviewed policy addition — no wildcard/prefix trust.

Procurement API (procurement-api stack)

A read-only REST API (API Gateway + the procurement-api Lambda, lambdas/api/) over both pipelines' tables, plus a token-gated OpenAPI docs page. Primary consumer: the SHOC backend, for which this replaces the retired SyncController cross-account DynamoDB scan as the reconciliation/backfill path (and the initial-history load for the outbound work-order webhook, which starts at LATEST).

Method + path	Auth	Backing read
GET /work-orders · /purchase-orders · /verified-sites	IAM SigV4	unordered paginated Scan (limit 1–500, opaque cursor = base64 LastEvaluatedKey; malformed cursor → 400)
GET /work-orders/{id} · /purchase-orders/{id} · /verified-sites/{siteCode}	IAM SigV4	GetItem (404 on miss)
GET /work-orders/{id}/comments	IAM SigV4	Query on the partition key, paginated
POST /work-orders/{id}/comments, PATCH /work-orders/{id}	—	phase-2 planned (x-planned in the spec); handler answers 501
GET /docs, GET /openapi.json	shared docs token (X-Auth-Token header or ?token= in a browser)	Redoc reference docs (vendored offline, no CDN) with the spec inlined / the committed spec

• Spec is source of truth: lambdas/api/openapi.json (OpenAPI 3.1). Its top-level webhooks section documents the outbound SHOC work-order feed, so one page describes both directions (call + be-called). tests/test_api_spec_drift.py pins the spec's paths to the router table, so spec and implementation cannot drift.
• Auth: data routes use API Gateway AWS_IAM (SigV4) plus a resource policy allowing exactly arn:aws:iam::396287094661:role/shoc-backend-dev on GET/*; same-account admin callers authorize via identity policy (Postman signs SigV4 natively). Docs routes are auth NONE at the gateway (resource-policy carve-out for exactly those two GETs) but the handler fails closed on the shared token (lambdas/shared/web_ui_auth.py, secret procurement-ingest/web-ui-auth-token) — not an unauthenticated data path (INFRA-74 posture).
• Custom domain: https://procurement-api.seahaven.com (REGIONAL API Gateway domain, TLS 1.2, empty base-path mapping to the prod stage, so callers hit /work-orders with no /prod segment). The stable SHOC-facing endpoint; the raw *.execute-api.us-east-1.amazonaws.com/prod URL still works. Cross-account DNS: the seahaven.com public zone is in the mgmt account (328440206208), so the ACM cert's validation record and the A-alias are added there out of band — scripts/setup_procurement_api_domain.sh cert issues the cert (DNS-validated against the mgmt zone) and writes its ARN to prod SSM /procurement-api/custom-domain/certificate-arn, which the stack reads (value_for_string_parameter, CFN-resolved at deploy); after cdk deploy procurement-api, scripts/setup_procurement_api_domain.sh alias adds the A-alias from the stack's ProcurementApiAliasTarget/ProcurementApiAliasHostedZoneId outputs. SigV4 is unaffected (same underlying API id + resource policy); the docs page injects whichever host served the request into servers[0].url.
• KMS: purchase-orders is CMK-encrypted; the imported-by-name table doesn't carry the key association, so the stack grants kms:Decrypt/DescribeKey on the CMK from SSM /seahaven/dynamodb/cmk-arn explicitly (the INFRA-104 failure class).
• No access logging in v1 (keeps the ?token= shim out of any log and avoids the account-level API Gateway CloudWatch role); rotate the docs token before ever enabling it. No CORS (server-to-server + Postman callers).
• Alarms: procurement-api-errors/-throttles/-duration (p99 ≥ 22.5 s) + gateway procurement-api-5xx, all ALARM-only → site-alerts. No 4XX alarm (401/403/404 are expected traffic).
• Docs page: /docs serves Redoc (read-only reference docs; live calls go through Postman since data routes need SigV4) as ONE token-gated response: the handler inlines the vendored redoc.standalone.js, the design-system fonts (fonts.css, data-URI @font-face — nothing may fetch from Google Fonts, test-pinned), and the spec into docs.html, with </script>/</style> breakout guards on every blob. Theme = SHOC tokens (Montserrat/DM Sans/JetBrains Mono, primary #1c75bc, 64px gradient topbar). The right-panel gradient and the topbar's Hide/Show-samples toggle target styled-components class names that are deterministic for the pinned Redoc bundle but change on any bump — re-derive them then (headless probe: elements whose computed background equals the rightPanel color); stale selectors degrade to a solid panel / inert toggle, cosmetic only. Tooling: npm run lint:api lints the spec against redocly.yaml (CI job spec-lint; deliberate exceptions live in .redocly.lint-ignore.yaml), npm run docs:preview renders the real handler output locally.

Architecture

IaC: AWS CDK (Python), three stacks in one app, region us-east-1. The cdk.Environment is deliberately account-agnostic (region-only, no account=): the stacks deploy to whichever account the deploy credentials target (011934824531 seahaven-prod since the 2026-07 account migration; formerly mgmt 328440206208), and every account-derived template value — bucket names, Lambda::Permission source account, the site-alerts SNS action ARN, the Bedrock ARN below — renders as the CloudFormation AWS::AccountId pseudo-parameter rather than a literal. Pinning account= was evaluated in Phase 4 and rejected: it would resolve those tokens to literals, and against the deployed (account-agnostic) templates CloudFormation flags the RemovalPolicy.RETAIN email buckets as requiring replacement — a data-loss risk — for no functional gain.

All Lambdas: Python 3.12, ARM64, 60-day log retention.

LLM provider — Amazon Bedrock. Both email processors call Claude Haiku 4.5 through the Bedrock inference profile us.anthropic.claude-haiku-4-5-20251001-v1:0 (bedrock-runtime InvokeModel), configured via the BEDROCK_MODEL_ID env var. There is no Anthropic API key and no Secrets Manager secret any more. Each processor role is granted bedrock:InvokeModel + bedrock:InvokeModelWithResponseStream on both the inference-profile ARN and the per-region foundation-model ARNs for us-east-1 / us-east-2 / us-west-2 (empty-account foundation-model ARNs) — the us.* profile can route cross-region, so a profile-only grant would AccessDenied at runtime under load.

Retired secrets (manual cleanup outstanding): the former secrets po-ingest/anthropic-api-key and workorder-ingest/anthropic-api-key had RemovalPolicy.RETAIN, so removing them from the CDK stacks orphans them rather than deleting them. Delete both by hand post-deploy and revoke the stored keys at the provider. The processors no longer read any ANTHROPIC_API_KEY_SECRET_ARN — authentication to Bedrock is via the Lambda execution-role IAM grant, so there is no provider API key or Secrets Manager fetch on the parse path.

SES: Both stacks add rules to the shared INBOUND_MAIL receipt rule set on int.seahaven.com.

Shared CDK helpers (cdk/common.py, Phase 4)

The ~379 lines that po_stack.py and wo_stack.py both defined identically (DynamoDB alarms, the sender-auth-rejected metric filter + alarm, the standard per-Lambda alarm set, the Bedrock InvokeModel grant, the raw-email bucket, the async-invoke DLQ, and the template-fallback-rate math alarm) are collapsed into cdk/common.py.

Logical-ID-safety rule (load-bearing). Every helper is a plain function taking (scope, id, ...) — never a Construct subclass. Each stack calls a helper with its own Stack instance as scope and the exact same literal construct id it used inline before the extraction, so every synthesized logical ID is byte-stable. A Construct subclass would insert an extra tree node, reparent every child's logical ID, and CloudFormation would attempt to replace the RemovalPolicy.RETAIN-protected purchase-orders / WorkOrders / WorkOrderComments tables and the named S3 buckets — a data-loss event. Nothing in common.py subclasses Construct, and it imports only the CDK constructs its helpers touch (dynamodb, cloudwatch/cw_actions, iam, logs, s3, sqs — no kms, ssm, or Lambda event-source imports).

Extracted helpers: add_ddb_alarms, add_sender_auth_rejected_alarm, add_standard_lambda_alarms (bespoke descriptions= dict passed through verbatim per call site — no alarm text is generated or homogenized), make_bedrock_invoke_statement, make_email_bucket, make_processor_dlq, make_fallback_rate_alarm. Per-function alarm variance is preserved exactly through call-site arguments, not baked into the helpers: PO's email-processor duration alarm uses p99, WO's uses p95; po-web-ui gets throttles + duration only (no errors, no DLQ); po-ingest-site-extractor has no DLQ alarm (it's a DynamoDB-stream consumer, not async-invoked); workorder-web-ui gets zero alarms — the helper is simply never called for it, so it cannot silently add any. The two "AI-fallback-rejected" alarms (PO's 6-hour count-floor IF(FILL(rej,0)>=1,...), WO's 5-minute/2-of-6 sparse idiom) are a different shape from make_fallback_rate_alarm and stay as distinct inline call sites in each stack rather than being forced through the shared helper.

Bedrock ARN, now account/region-derived. make_bedrock_invoke_statement(scope) builds the inference-profile ARN and the us-east-1 foundation-model ARN from Stack.of(scope).account / .region instead of the previous hardcoded 328440206208/us-east-1 literals (the us-east-2/us-west-2 foundation-model ARNs stay literal — they're fixed cross-region reach targets, not the stack's own region). Because the environment is account-agnostic (above), stack.account is the AWS::AccountId pseudo-parameter, so the derived ARN synthesizes as an Fn::Sub/Ref that resolves at deploy time to the same ARN the hardcoded literal named in-account. Versus the deployed stack this is a benign in-place IAM policy update (IAM policies never require replacement) — and it makes the grant account-portable instead of pinned to the management account. This IAM PolicyStatement change is the surface the mandatory GPT-4.1 cross-family review covers.

New CfnOutputs. Each stack now exports its Lambda function ARNs and the DynamoDB table names it owns/consumes, all net-new/additive (no existing output changes): po-ingest — EmailProcessorFunctionArn, WebUiFunctionArn, SiteExtractorFunctionArn, PurchaseOrdersTableName, PendingSiteReviewTableName (the existing VerifiedSitesTableName output is unchanged); workorder-ingest — EmailProcessorFunctionArn, WebUiFunctionArn, WorkOrdersTableName, WorkOrderCommentsTableName.

Sender authentication (INFRA-107)

The From header and any Authentication-Results header inside the raw MIME are attacker-forgeable, so neither is trusted. Instead, both email processors authenticate the sender against the verdicts SES itself stamps at delivery time, failing closed. The authenticator is single-sourced at lambdas/shared/ses_auth.py (Phase 3) — previously duplicated byte-for-byte in each pipeline's email_processor/ dir and kept in sync by a fixture-hygiene test; now one copy, so a future hardening fix to the fail-closed logic lands once instead of needing two identical edits. The CDK bundling cps it flat beside each handler so the handlers' unchanged from ses_auth import authenticate_inbound_email still resolves at runtime (see lambdas/shared/). The gate:

1. Take only the topmost Authentication-Results header (SES prepends its trace headers; any lower copies arrived inside the message and are ignored).
2. Require its authserv-id to be amazonses.com.
3. Require a dkim=pass clause whose header.d=/header.i= domain is in the pipeline's allowlist.

The allowlist is the ALLOWED_DKIM_DOMAINS Lambda environment variable (comma-separated, set per stack in CDK — no code change needed to adjust):

Pipeline	ALLOWED_DKIM_DOMAINS	Why
Work orders	seahaven.com	APM mail reaches apm@int.seahaven.com via a forward off amazon@seahavenind.com; the allowlist trusts the domain that re-signs DKIM on that forward (the original hxgnsmartcloud.com signature does not survive it). Validated against live SES-stamped headers (2026-07-16) — real APM deliveries carry dkim=pass header.i=@seahaven.com. Note a plain Gmail auto-forward re-signs under the sending Workspace domain (seahavenind.com / a *.gappssmtp.com key), not seahaven.com; only a Google Group (or Workspace routing) with "sign as seahaven.com" produces dkim=pass header.i=@seahaven.com. If the observed re-signing domain differs, update this value (do not widen it to a shared key like *.gappssmtp.com, which any Google customer's mail would pass). The workorder-email-processor-sender-auth-rejected alarm pages if this assumption is wrong instead of silently dropping every work order.
Purchase orders	amazon.coupahost.com	Coupa signs as amazon.coupahost.com. amazonses.com also passes but is deliberately not allowlisted — every SES customer's mail passes for it

Clause-injection hardening: SES echoes attacker-controlled SMTP-session tokens (envelope-from, helo, header.from) into its own Authentication-Results value, and an RFC 5321 quoted-local-part MAIL FROM may legally contain ; and spaces. The parser therefore tokenises comment- and quoted-string-aware (RFC 8601 / RFC 5322): CFWS comments (...) are stripped and clauses are split only on semicolons outside a quoted string, so a ; inside a quoted envelope-from= value can never be torn into a forged dkim=pass clause. The DKIM signer domain is read from header.d= when present (falling back to header.i=, taking the domain after the AUID's last top-level @ so a quoted local-part cannot smuggle an allowlisted domain). Two latent comment-parsing edge cases (early comment-close, no-separator-on-strip) are tracked as hardening follow-ups — see the SES-AR-01/02 issue; neither is reachable through SES's real header encoding today.

Risk acceptance — forwarder-domain binding (INFRA-107, accepted 2026-07-16): for work orders this control authenticates the domain that re-signs the apm@ forward (seahaven.com), not the Hexagon originator (hxgnsmartcloud.com, whose signature does not survive the forward). Its strength therefore rests on the apm@ Google Group's posting policy being restricted to trusted internal senders — that restriction is the load-bearing control and is accepted as documented risk. If the apm@ group is ever opened to external posting, this finding escalates to HIGH (anyone able to post to the group could inject a forged work order) and the correct fix is to bind acceptance to the originator via DMARC alignment rather than the forwarder's re-signature. The PO pipeline is unaffected — amazon.coupahost.com is an external domain an attacker cannot get SES to sign.

On any failure (env var unset, header missing/unparseable, verdict fail, unaligned domain) the processor logs a structured sender_auth_rejected warning with the reason and S3 key, skips the email, and returns normally — rejected mail never triggers Lambda retries or DLQ messages, but the <fn>-sender-auth-rejected CloudWatch alarm (see CloudWatch alarms) pages on a rejection spike so a drift-induced outage is not silent. Unit tests live in tests/test_ses_auth.py.

Failure handling (INFRA-41): Each email-processor is async-invoked (S3 → Lambda). Both have a CDK-managed SQS dead-letter queue (dead_letter_queue=, 14-day retention, SSL-enforced) so a failed parse is captured rather than silently dropped after Lambda's retries.

CloudWatch alarms

Every alarm is ALARM-only (no OK action), sends to the shared site-alerts SNS topic (imported once per stack via Topic.from_topic_arn), and uses TreatMissingData.NOT_BREACHING.

Lambda alarms (AWS/Lambda, FunctionName dimension):

Alarm	Functions	Metric / config
<fn>-errors	po-email-processor, po-ingest-site-extractor, workorder-email-processor, workorder-shoc-emitter, workorder-shoc-hmac-rotator	Errors Sum, 5 min, > 0, eval 1
<fn>-throttles	po-email-processor, po-ingest-site-extractor, po-web-ui, workorder-email-processor, workorder-shoc-emitter, workorder-shoc-hmac-rotator	Throttles Sum, 5 min, > 0, eval 1
<fn>-duration	po-email-processor, po-ingest-site-extractor, po-web-ui, workorder-shoc-emitter, workorder-shoc-hmac-rotator (p99); workorder-email-processor (p95)	Duration percentile, 5 min, >= 45000 ms (75% of the 60s timeout), eval 3 / datapoints 2
<fn>-sender-auth-rejected	po-email-processor, workorder-email-processor	Log-metric-filter count (namespace Seahaven/ProcurementIngest, default_value=0) on sender_auth_rejected warnings, Sum 5 min, >= 1, eval 3 / datapoints 2

The <fn>-sender-auth-rejected alarm closes the silent-drop gap in INFRA-107: a rejected email returns normally (no error, no retry, no DLQ message), so without a log-metric filter a signing-domain drift or a wrong allowlist would discard 100% of legitimate mail while every other alarm stayed green. It counts sender_auth_rejected warnings per 5-minute period (default_value=0 keeps the series continuous) and pages when 2 of the last 3 periods each see at least one rejection — a lone stray spoof probe to the internal ingest address self-clears, but a sustained false-reject storm pages within ~10–15 minutes even at low mail volume; the config is easy to tune in the CDK helper. (A residual gap remains for a very sparse total-reject outage — see the SES-AR-01/02 hardening issue.)

The <fn>-duration and <fn>-throttles alarms for po-email-processor and workorder-email-processor supersede the orphaned, CLI-created Lambda-Duration-* / Lambda-Throttles-* alarms (deleted post-deploy).

DLQ alarms (AWS/SQS): po-email-processor-dlq-messages and workorder-email-processor-dlq-messages fire when any message is visible on an email-processor DLQ (ApproximateNumberOfMessagesVisible Maximum, 5 min, > 0, eval 1) — a message there means an email was dropped after Lambda exhausted its async retries. Recovery from a DLQ message (no console redrive) is documented in the DLQ recovery runbook.

SHOC emitter alarms (bespoke — these don't fit the standard-Lambda-alarm helper's shape): workorder-shoc-emitter-iterator-age (IteratorAge Maximum, 5 min, >= 600000 ms, eval 3 / datapoints 2) fires when the stream lags ≥ 10 minutes — SHOC is likely down and the shard is blocking on retries, which is exactly the ordered-backpressure design working, but an operator should know. workorder-shoc-emitter-failures-messages and workorder-shoc-emitter-rejected-messages (ApproximateNumberOfMessagesVisible Maximum, 5 min, > 0, eval 1) page the replay runbook: the -failures queue receives ESM failure metadata for retry-exhausted records, the -rejected queue receives the full parked payloads of non-retryable 4xx deliveries (see the SHOC webhook feed section and scripts/replay_shoc_webhooks.py). All three: ALARM-only → site-alerts, NOT_BREACHING, per the house rules above.

Parse-outcome metric + fallback-rate alarm (workorder-ingest): the WO processor writes one CloudWatch EMF line per email to namespace Seahaven/WorkorderIngest, metric ParseOutcome (Unit Count, value 1), dimensioned by ParseMethod (template | ai_fallback | ai_fallback_rejected) and TemplateId (update_plaintext | assign_html | unknown). ai_fallback_rejected counts AI-fallback output that failed the fail-closed validate_ai_fallback() gate (schema/enum/date contract on raw Bedrock output — prompt-injection defence) and was dropped without a DynamoDB write. Non-dimension EMF properties ReasonCode and work_order_id are queryable in Logs Insights but not promoted to metrics (kept low-cardinality). EMF is used instead of PutMetricData so there is no extra sync call / latency / IAM grant on the async hot path (the role already has logs:PutLogEvents). The alarm workorder-email-processor-template-fallback-rate fires when the AI-fallback share of parses — rejected fallback parses included, so a drift outage whose AI output also fails the gate cannot lower the observed rate while dropping mail — exceeds 15% sustained (a MathExpression with FILL(...,0) and a ≥10-sample volume floor over 15-minute periods, eval 3 / datapoints 2) — catching Hexagon template-drift coverage collapse while the volume floor + FILL prevent low-volume false pages / INSUFFICIENT_DATA. ALARM-only SnsAction to site-alerts, no OK action, NOT_BREACHING. The 15-minute period is a deliberate deviation from the 5-minute house style to accumulate a stable denominator at the low ~760/day volume. A second alarm, workorder-email-processor-ai-fallback-rejected, pages on the rejected series itself (≥1 rejection per 5-min period, 2 of the last 6 periods — the sender-auth-rejected sparse-arrival idiom) because a gate rejection drops mail without error/retry/DLQ and would otherwise be silent.

WO Bedrock transport-error metric (Phase 8). A Bedrock-side transport error (throttling, malformed response, non-JSON model text) during the AI-fallback attempt previously emitted zero ParseOutcome datapoints — the only emit sites were post-gate. handler.py now wraps the extract_with_bedrock call in a try/except that emits exactly one ParseMethod=ai_fallback / ReasonCode=bedrock_error datapoint and then re-raises (the exception still propagates into the errors alarm / DLQ path unchanged). This is an except-and-reraise, not a reorder: a gate-rejected email (Bedrock returns successfully, validate_ai_fallback() then rejects it) still emits only the single ai_fallback_rejected datapoint and nothing else — the except branch never fires because Bedrock did not raise — so the workorder-email-processor-ai-fallback-rejected "a rejected email emits nothing else" alarm contract holds with no double-count.

Parse-outcome metric + fallback-rate alarm (po-ingest): the PO processor emits the same EMF shape to namespace Seahaven/PoIngest, metric ParseOutcome, dimensioned by ParseMethod (template | ai_fallback | ai_fallback_rejected) and TemplateId (coupa_new_po | coupa_cancellation | unknown), with ReasonCode (the fail-closed gate reason) and po_number as Logs-Insights ride-alongs. ai_fallback_rejected counts AI-fallback output that failed the fail-closed validate_ai_fallback() gate (nested key-set contract, po_number shape, email_type allow-list, Decimal money typing) and was dropped without a DynamoDB write. The ai_fallback_rejected emission's po_number ride-along is clamped to 64 chars (handler.py:118, pinned by the PO-DC-02 regression test) — telemetry.py's DerivedFieldAgreement PythonValue/LlmValue properties clamp the same way.

Deliberate double-count: unlike WO, PO emits ParseMethod=ai_fallback before the Bedrock call (so a Bedrock-side error still records the outcome) — a rejected email therefore always emits both an ai_fallback datapoint (pre-call) and an ai_fallback_rejected datapoint (post-gate), never just the latter. This is intentional and load-bearing, not a bug; the fallback-rate math below treats fb as already inclusive of every rejection.

The alarm po-email-processor-template-fallback-rate is deliberately retuned for PO volume — do NOT copy the WO numbers: at ~57 emails/day a 15-minute period holds ~0.6 emails, so the WO ≥10-sample floor would never be met and the alarm would be structurally dead. Instead: 6-hour periods (~14.25 expected emails each), an IF((fb+tmpl)>=8, …) volume floor (at the floor a single fallback email is 12.5% < the threshold, so one email can never breach a datapoint; a breach needs ≥2 fallbacks in one window, or ≥3 at typical volume), threshold >20% (expected baseline fallback ≈1%: comments 0.55% + multi-line 0.18% + non-USD 0), eval 4 / datapoints 2 (a 24h span — isolated noise self-clears while total template drift at 100% fallback pages within ~12h). Sparse overnight/weekend windows below the floor evaluate to 0 (non-breaching by design; accepted trade: a Friday-evening drift may not page until weekend volume accrues). The ai_fallback_rejected series (rej) is deliberately excluded from this expression's numerator, denominator, and volume floor: because the pre-call emit already counts every rejected email once inside fb, folding WO's fb+rej math in verbatim would double-count each rejection in both terms and inflate the observed rate toward 100% — fb/(fb+tmpl) alone is already exact for PO. Same idiom otherwise: ALARM-only SnsAction to site-alerts, NOT_BREACHING, and no element-wise MAX in the math expression (the post-#102 rule — the IF floor guarantees the non-zero denominator).

A second alarm, po-email-processor-ai-fallback-rejected, monitors the rejected series on its own — retuned for ~57 emails/day, not WO's 5-minute sparse idiom (which needs two rejections inside one 30-minute window and would be structurally dead at PO volume). It uses the same 6h/IF-floor/eval-4/datapoints-2 idiom as the fallback-rate alarm above, but as a plain count-floor on the rejected series itself (IF(FILL(rej,0)>=1, …), no denominator so no divide guard is needed): threshold ≥1, over 6-hour periods, eval 4 / datapoints 2 — a lone stray rejection self-clears, while ≥2 rejections landing in ≥2 distinct 6h windows within 24h (sustained prompt-injection probing, or template drift whose AI output also fails the gate) pages within ~12–24h. ALARM-only SnsAction to site-alerts, NOT_BREACHING. Accepted residual: a single isolated rejected email never pages this alarm by itself — it is still visible as an ai_fallback_rejected datapoint and in the ReasonCode log line, and it has already raised the fallback-rate numerator above via its pre-call ai_fallback emit.

DynamoDB alarms (AWS/DynamoDB): each owned table gets <table>-throttles (ThrottledRequests) and <table>-system-errors (SystemErrors). These metrics emit only at the TableName + Operation dimension set, so each alarm is a Sum math expression across the operations the table uses (Get/BatchGet/Query/Scan/Put/Update/Delete/BatchWrite). Tables covered: purchase-orders, verified-sites, pending-site-review (po-ingest); WorkOrders, WorkOrderComments (workorder-ingest).

Deploy-Pipeline Guards (Phase 0)

Goal: a broken Lambda bundle fails the deploy job, not Monday's first email.

Healthcheck direct-invoke contract. Both po-email-processor and workorder-email-processor recognize a top-level direct-invoke probe payload {"healthcheck": true}. In each handler(event, context), the very first statements — before any S3 fetch, before ses_auth, before iterating event["Records"] — are:

if isinstance(event, dict) and event.get("healthcheck") is True:
    return {"healthcheck": "ok"}

This placement is deliberate, not incidental: real mail always arrives as an S3 ObjectCreated event whose top-level keys (Records) AWS controls, so email content can never set a top-level healthcheck key — the branch creates no accept path for forged mail. It also emits no EMF metric and no log line, so it can never match the sender_auth_rejected log-metric-filter pattern that feeds the <fn>-sender-auth-rejected alarm (see CloudWatch alarms) — that alarm pages at ≥1 match in its window, so repeated healthcheck invokes across deploys (two deploys in ~30 min is routine) must never contribute to it. Unit coverage: lambdas/po/email_processor/tests/test_po_healthcheck.py and lambdas/wo/email_processor/tests/test_healthcheck.py.

Post-deploy smoke gate. scripts/post-deploy-smoke.sh is wired into the CD workflow as cd-cdk.yaml's post-deploy-script input (see CI/CD) and runs synchronously after every deploy, before the workflow is considered green. It invokes po-email-processor, workorder-email-processor, and procurement-api with aws lambda invoke --invocation-type RequestResponse --payload '{"healthcheck": true}' (region us-east-1) and asserts, per function:

1. The invoke response's FunctionError field is absent — this is the load-bearing check. A broken bundle (e.g. an ImportError at module init from a missing sibling module) still returns HTTP 200 from the Lambda Invoke API with FunctionError=Unhandled; a bare exit-code check on aws lambda invoke would false-pass on exactly the failure this gate exists to catch.
2. The returned payload is exactly {"healthcheck": "ok"}.

The script runs set -euo pipefail and exits non-zero on any invoke failure, any FunctionError, or a payload mismatch on either function, failing the deploy job.

PO bundling: glob replaces the hand-maintained allowlist. cdk/po_stack.py's asset bundling command ships PO's Lambda source with a non-recursive glob instead of a hand-maintained list of filenames (cp handler.py ses_auth.py template_parser.py derived_fields.py /asset-output/). The glob is functionally identical for today's file set — non-recursive, so tests/ and other subdirectories are still excluded — but structurally eliminates the failure mode that shipped a broken bundle twice (PR #105 omitted template_parser.py; PR #2 nearly omitted derived_fields.py): a new sibling module the handler imports now ships automatically instead of requiring someone to remember to add it to the list.

Phase 2: widened asset root, both processors on the glob. Both cdk/po_stack.py and cdk/wo_stack.py widen their bundled email-processor's Code.from_asset root from the per-pipeline dir (../lambdas/po/email_processor, ../lambdas/wo/email_processor) to the shared parent, ../lambdas — the prerequisite for the Phase 3 lambdas/shared/ extraction, which needs a bundling root able to reach a sibling shared/ package outside either pipeline's own dir (this move ships zero handler code changes — git diff -- lambdas/ is empty for this PR). With bundling present, CDK mounts the asset root as the container's working directory, so both the pip install -r path and the cp source operand became repo-relative to lambdas/: -r po/email_processor/requirements.txt (resp. wo/email_processor/requirements.txt) and cp po/email_processor/*.py /asset-output/ (resp. cp wo/email_processor/*.py /asset-output/). The pip --platform manylinux2014_aarch64 --only-binary=:all: pin — removing it once shipped x86 wheels into the ARM64 function and caused a total outage (PR #34) — is preserved byte-for-byte on both.

Both bundled from_asset calls also gain exclude=['**/__pycache__/**', '**/tests/**', '**/package/**']. This is load-bearing, not cosmetic: Code.from_asset does not honor .gitignore, and widening the root to ../lambdas means the untracked, 44 MB lambdas/po/email_processor/package/ dir (a stale vendored dependency tree; deletion is a separate, deliberate call — not part of this change) would otherwise be staged into the source fingerprint from_asset hashes to decide whether to re-bundle. Because CI never has that local-only directory, an un-excluded root would diverge the local vs. CI asset hash on every synth/deploy and force spurious redeploys; the **/tests/** and __pycache__ excludes keep the hash stable for the same reason. Note the exclude does not decouple the two pipelines' asset hashes: from_asset hashes with its default AssetHashType.SOURCE, so the fingerprint is computed over all of ../lambdas minus only the excluded __pycache__/tests/package paths — PO's and WO's first-party source (both email_processor trees, plus the two web_uis and the site_extractor) therefore both feed both email-processors' hash. Editing any non-excluded file under lambdas/ changes both email-processors' source fingerprint and redeploys both functions with byte-identical bundles. That coupling is an accepted cost of the shared-root design (the bundling cp glob still copies only each pipeline's own *.py into the zip); the excludes exist solely to strip local-only/irrelevant cruft that would diverge local vs. CI, not to isolate PO's tree from WO's — which SOURCE hashing cannot do here.

WO bundling: glob replaces the whole-dir copy — deliberate prod-zip shrinkage. WO's bundling command changes from a recursive cp -r . /asset-output/ (the entire wo/email_processor/ source dir, copied into the deployed zip) to the same scoped, non-recursive glob PO uses: cp wo/email_processor/*.py /asset-output/. This intentionally drops from the production zip:

• requirements.txt — needed only at bundle time (pip install -r ...), never at runtime;
• the entire tests/ tree (lambdas/wo/email_processor/tests/) — real scrubbed .eml fixtures, golden JSON, and test modules;
• any first-party __pycache__/*.pyc a local cp -r . would have picked up (the currently-deployed zip carries none, but the exclude keeps future local builds equally clean).

This is cleanup, not a regression: none of those file classes are imported at runtime by handler.handler, so the acceptance bar for this change on WO is "the runtime-imported module set is unchanged, plus a post-deploy smoke pass" — not a byte-identical zip diff (that stricter bar applies to PO only, whose deployed zip was already this tight before this change). All four first-party top-level .py files WO's handler needs — __init__.py, handler.py, ses_auth.py, template_parser.py — are still shipped; the glob retains __init__.py because it is itself a top-level .py file, not a special case requiring a separate copy rule.

Plain (non-bundled) from_asset calls gain exclude too. The three non-bundled Lambda assets — po-web-ui, po-ingest-site-extractor, workorder-web-ui — each add exclude=['**/__pycache__/**']. Nothing else about these three changes: each keeps its own scoped asset path (../lambdas/po/web_ui, etc.) rather than widening to ../lambdas, and none gains bundling. Without the exclude, a developer's local __pycache__ — again invisible to from_asset's .gitignore-blind staging — makes that function's asset hash nondeterministic across machines and forces spurious redeploys.

tests/test_bundle_consistency.py guards all of the above with a pure-AST check (no synth, no boto3, no handler import): it parses each handler.py's top-level first-party sibling imports, extracts the bundling command=[...] string from the corresponding CDK stack file, and asserts every required sibling module is guaranteed to ship. It recognizes both the scoped glob (cp po/email_processor/*.py / cp wo/email_processor/*.py, with or without a path prefix) and a whole-dir recursive copy (cp -r . /asset-output/) as unconditionally-safe shapes, and falls back to literal filename matching for any other (allowlist-style) shape. It pins each stack's command to the scoped-glob form specifically — a future revert to a narrowed single-file copy, a commented-out glob, or a filename allowlist missing a sibling all fail CI loudly instead of silently shipping a broken bundle. Runs in the existing pytest step, before synth.

Phase 3: shared module extraction (lambdas/shared/)

Four first-party modules that were previously duplicated per pipeline (or inlined in each handler) are now single-sourced under lambdas/shared/, following the handbook's lambdas/shared/ convention:

Module	What it is	Imported by
ses_auth.py	fail-closed SES sender-authentication (INFRA-107)	both email processors
web_ui_auth.py	fail-closed X-Auth-Token gate + token cache (INFRA-74)	both web_ui handlers
email_parsing.py	parse_raw_email (the WO superset that returns cc unconditionally; PO simply ignores cc)	both email processors
emf.py	generic CloudWatch EMF emitter (emit_metric, emit_parse_outcome) parameterized by namespace / dimension-sets / properties	both email processors (PO also uses emit_metric for DerivedFieldAgreement)

Flat-landing import rule. The shared dir has no __init__.py — the modules are consumed by bare name (from ses_auth import ..., from emf import emit_parse_outcome), exactly as when they were siblings. This works because the bundling cp lands them flat in /asset-output/ beside handler.py, so at runtime each shared module sits on the function's own sys.path under its bare name — the handler import lines are unchanged, which is what keeps the byte-identical fail-closed ses_auth behavior through the move. The load-bearing emf dimension-set list [["ParseMethod"], ["ParseMethod", "TemplateId"]] is now pinned once in emf.py (one-sided dimension drift between the two pipelines becomes structurally impossible), while the deliberate per-pipeline emission-ordering differences stay in the handlers (PO emits ai_fallback before the Bedrock call with an intentional double-count; WO emits mutually-exclusive ai_fallback/ai_fallback_rejected after its gate).

Bundling — email processors. Both email-processor commands append a second glob, cp shared/*.py /asset-output/, after their own cp <pipeline>/email_processor/*.py. This ships all four shared modules flat into each email-processor zip. web_ui_auth.py therefore rides along into both email-processor bundles even though the email handlers never import it — a harmless, deliberate consequence of the all-of-shared/ glob (PO_EXPECTED_TOP_LEVEL_MODULES and the bundle-parity expectations account for it). The base is cp-only (Phase 7 removed the pip install / manylinux pin from both email-processor commands, and this phase moves only pure first-party modules with no new dependencies, so it stays cp-only — no pip step is reintroduced).

Bundling — web UIs. Both po-web-ui and workorder-web-ui gain the same widened-root Docker bundling mechanism: their Code.from_asset root widens to ../lambdas and their command copies the function's own dir contents plus only shared/web_ui_auth.py (cp shared/web_ui_auth.py, not cp shared/*.py) — the web UIs need only the auth module, and shipping the email-processor-only modules would break the "deployed set + web_ui_auth, nothing else" parity. The per-stack INFRA-74 comments stay in each handler (their wording is deliberately pipeline-specific and is not unified). site_extractor's asset is untouched.

tests/test_bundle_consistency.py is updated in lockstep without losing teeth: _first_party_sibling_imports resolves shared-sourced imports under lambdas/shared/; the command extractor selects the email-processor command now that each stack has two bundled functions; _bundling_ships_all accumulates shipped module stems across both globs; PO_EXPECTED_TOP_LEVEL_MODULES gains the four shared modules; and a new pin + mutation test require the cp shared/*.py line to be actually executed (a commented-out or removed shared cp fails CI).

Phase 5: handler decomposition + lazy boto3 clients

Both email-processor God-handlers are decomposed along the seams that already work into flat sibling modules in the same directory (bare-name imports, exactly like the ses_auth/template_parser/derived_fields pattern), so the Phase 0/2/3 cp <pipeline>/email_processor/*.py glob ships every new sibling automatically — no bundling change beyond the exact-set pin. Every move is a pure delete-here/add-there; derived_fields.py, both template_parser.py, the validate_ai_fallback gates, lambdas/shared/, and cdk/ are byte-untouched.

PO (handler.py → 5 siblings + prompts.py): handler.py keeps the event loop, fail-closed SES auth, and email_type routing; extraction.py owns extract_with_claude + _EMAIL_TAG_RE; enrichment.py owns enrich_parsed + pad_zip (PO-only — WO has no enrichment stage) as a byte-identical move including the derived-field shadow block; telemetry.py owns the EMF ParseMethod/DerivedFieldAgreement emit wrappers; persistence.py owns _write_fields/_merge_update/save_* — with the two byte-identical save_new_po/save_revision collapsed into one _save_merge plus two thin wrappers differing only in the log verb (behavior-identical to both originals, sticky-cancel ConditionExpression guard intact; save_cancellation stays its own function). WO splits into ~5 concerns (no enrichment), keeping validate_ai_fallback and the re.fullmatch(r"[0-9]+", work_order_id) key guard in the handler loop AHEAD of both save_work_order and save_event (it protects the partition key and the #-delimited comment_id range-key segment), and keeps _header_date_iso/comment_id determinism together with save_event in persistence.py.

EXTRACTION_PROMPT (the ~181-line prompt) moves to prompts.py with a cross-reference header to the second authoritative copy of the trade/site/fiscal rule tables in derived_fields.py; handler.py keeps a from prompts import EXTRACTION_PROMPT re-export so handler.EXTRACTION_PROMPT still resolves for the tests that dereference it.

Lazy cached boto3 clients. Each I/O module initializes its client cache to None and populates it through a private _get_<client>() accessor on first call (extraction.bedrock, persistence.dynamodb, handler.s3); pure modules (enrichment, telemetry, prompts) import no boto3. The cache attribute keeps its original public name, so a test patches the same attribute — only the owning module moved (e.g. setattr(persistence, "dynamodb", fake)). Building the client at first call (deep inside a test) rather than at import also strengthens the moto-before-handler invariant.

Behavior preserved (pinned by new tests). PO emits ParseMethod=ai_fallback before the Bedrock call (a throttle that raises still leaves the pre-call datapoint), and a gate rejection is an additive second ai_fallback_rejected datapoint (PO's deliberate double-count); WO emits after its gate, mutually exclusive; the DerivedFieldAgreement shadow telemetry stays ai_fallback-only; _save_merge issues byte-identical update_item calls for both new_po and revision; and the WO key guard fires before either save.

Security

Web UI auth (defense-in-depth). The po-web-ui / workorder-web-ui handlers refuse unauthenticated requests even though their public Function URLs were removed (INFRA-74). Each requires a shared secret in the X-Auth-Token header (or Authorization: Bearer <token>), compared in constant time against the configured token. The handler fails closed if the token is unset or unreadable (denies all). The token lives in the Secrets Manager secret procurement-ingest/web-ui-auth-token; only its ARN is passed to the Lambda (WEB_UI_AUTH_TOKEN_SECRET_ARN), and the value is fetched at runtime — never embedded in the CloudFormation template or Lambda env vars. The fetched value is cached in the warm container with a short TTL (5 min) so a rotated secret propagates without waiting for the execution environment to recycle. This is a defense-in-depth floor for a detached URL, not primary auth.

Output escaping. All caller-influenced values (including prompt-injectable strings Claude may return for total_amount/line-item amounts) are HTML-escaped before interpolation to prevent stored XSS.

Shared Resources

purchase-orders table (owned here)

The purchase-orders DynamoDB table is owned by this repo's po-ingest stack (defined in cdk/po_stack.py with RemovalPolicy.RETAIN, StreamViewType.NEW_IMAGE, and SSE-KMS encryption with the shared customer-managed CMK alias/seahaven-dynamodb, INFRA-95 / M-3). The po-email-processor Lambda is the authoritative writer — it performs the merge inserts, field-level revision merges, and cancellation updates described above.

Consumers (read-only):

Consumer	How it reads	Purpose
procurement-api (this repo)	Table.from_table_name + grant_read_data	REST reads (GET /purchase-orders*)

seahaven-slack-bot, the former external consumer, was decommissioned 2026-07-23 and its cross-account grants removed. External consumers now read through the procurement-api REST contract (lambdas/api/openapi.json), never by importing the table directly.

Schema-coordination rule: Any change to the purchase-orders schema (partition key, item shape, attribute names, streams view type) must be reflected in the API's OpenAPI schemas in the same PR (the spec-drift test pins paths, not item shapes — the schema fields are maintained by hand). Treat schema changes as a contract migration, not a local edit.

Known exception (INFRA-51): amazon-po-parser currently writes directly to purchase-orders outside this stack (backfill/enrichment scripts). This second writer is being folded into the po-ingest pipeline so this stack is the sole writer; until INFRA-51 closes, coordinate any schema change with amazon-po-parser as well.

WorkOrders and WorkOrderComments tables (owned here)

Both tables are owned by this repo's WorkorderIngestStack (cdk/wo_stack.py, RemovalPolicy.RETAIN):

• WorkOrders — PK work_order_id (S).
• WorkOrderComments — PK work_order_id (S), SK comment_id (S).

comment_id format change (issue #23). The WorkOrderComments range key is now work_order_id#<comment_time|nocomment>#<sha256(s3_object_key)[:12]> (e.g. 11144580730#2026-04-27T23:51:48#a1b2c3d4e5f6, or …#nocomment#… when the source email carries no comment time). Previously it was work_order_id#<timestamp>, where two emails on the same WO with an identical/absent comment time collided and overwrote each other. The 12-hex suffix is derived from the S3 object key alone — deterministic, so a Lambda async retry of the same object produces a byte-identical key (idempotent, no duplicate row), while two distinct emails on the same WO get distinct keys. Wall-clock now() is kept out of the key. Consumers that split on # and read index [0]/[1] are unaffected; anything that treated "everything after the first #" as a bare timestamp now also captures the hash segment.

Timestamp format shift. All stored ISO timestamps (created_at, updated_at, ingested_at on WO; processed_at, cancelled_at on PO) moved from naive datetime.utcnow().isoformat() to timezone-aware datetime.now(timezone.utc).isoformat(), so they now carry a +00:00 suffix (e.g. 2026-07-15T12:00:00+00:00). Downstream parsers that assumed a naive/no-offset string must accept the offset.

Both currently use default DynamoDB encryption — they are not yet on the shared customer-managed CMK (alias/seahaven-dynamodb, INFRA-95 / M-3); that migration is tracked in INFRA-6. The workorder-email-processor role no longer holds a pre-emptive encrypt/decrypt grant on that CMK (removed in the 2026-06-17 security sweep — it was unused while the tables are unencrypted and extended the role's decrypt reach to the CMK protecting purchase-orders). Re-add the grant as part of the INFRA-6 migration, at which point grant_read_write_data on the then-encrypted tables propagates the needed key permissions automatically.

Consumers (read-only) — data contract: seahaven-slack-bot (the former external reader) was decommissioned 2026-07-23; its grants are gone. Current consumers: the procurement-api Lambda (this repo, Table.from_table_name + grant_read_data, serving GET /work-orders*), and the workorder-shoc-emitter stream consumer (this repo) — both tables now stream NEW_AND_OLD_IMAGES, and the emitter is an active consumer of those streams (ESMs enabled 2026-07-30). External readers (SHOC) consume through the REST contract (lambdas/api/openapi.json) and the webhook contract (docs/shoc-webhook-contract.md); both documents' field lists mirror lambdas/wo/email_processor/persistence.py. Any change to table name, key schema, attribute names, or encryption configuration (e.g. the INFRA-6 CMK migration) must update those two contracts in the same PR — the tables are imported by name, so there is no compile-time link and breakage surfaces at runtime.

Stream field contract (site_code). Three definitions of "is this a valid site code" have existed in this repo at once. The canonical shape is derived_fields._STRICT_CODE_RE ([A-Z][A-Z0-9]{2,4}, fullmatch) plus its skip-list semantics (a code-shaped token is only a real site code if it is not skip-listed, e.g. LLC/INC/CORP/LTD/ATTN), used by derive_site_code() (the enrich_parsed() classifier, see the Derived fields note in the Purchase Orders flow above). Honestly noted, not papered over: lambdas/po/site_extractor/handler.py's own SITE_CODE_PATTERN ([A-Z]{2,4}\d{1,2}, prefix-anchored .match, digit-requiring) still diverges from the canonical shape as of this phase — it rejects valid all-letter codes like KLAL (which surface as permanent pending-site-review rows) and accepts overlong junk like DLI6X/SNY55 that the canonical fullmatch would not. Reconciling po-ingest-site-extractor's direct-field validation onto the canonical derived_fields shape is Phase 6 scope, tracked separately — this paragraph will be updated when it lands.

verified-sites table (owned here)

Owned by this repo's po-ingest stack (cdk/po_stack.py). PK siteCode (S); default DynamoDB encryption (NOT the shared CMK).

Consumer (read-only) — data contract: seahaven-slack-bot (former reader, incl. the INFRA-138/INFRA-180 by-state GSI drift saga) was decommissioned 2026-07-23 — the GSI-drift issue died with it. Current consumer: the procurement-api Lambda (GET /verified-sites*). The API's VerifiedSite schema deliberately promises only pipeline-written fields (siteCode, address, city, state, zip, fullAddress, locationCode, poCount, sourcePOs) — latitude/longitude/notes were manually curated legacy attributes the extractor never writes, and are not part of the contract.

Documentation

The canonical map of Sea Haven's AWS infrastructure lives in Confluence. This project's po-ingest and WorkorderIngestStack stacks are represented there as Mermaid subgraphs.

• AWS Architecture Map (Confluence, IT space, page 1540098)

CI/CD

GitHub Actions with reusable workflows from Sea-Haven-Industries/.github (all pinned to a commit SHA of main):

• CI (ci.yaml, PR to main): linting + cdk synth via ci-python-sam.yaml. cdk synth's Docker-bundled asset build for po-email-processor and workorder-email-processor mounts the widened ../lambdas asset root (Phase 2, see Deploy-Pipeline Guards) as build context, not just each function's own subdirectory — the exclude list on both from_asset calls strips local-only __pycache__/package/ (and tests/) cruft from that wider mount's source fingerprint, so CI's asset hash matches a clean local checkout, and each function's scoped cp glob copies only its own pipeline's *.py into the zip. (The exclude does not, and under SOURCE hashing cannot, keep the other pipeline's tracked source out of the fingerprint (see the PO bundling note above on SOURCE hashing) — but that source is identical in CI and local, so it does not cause hash divergence.)
• CD (deploy.yaml, push to main): CDK deploy via cd-cdk.yaml (OIDC auth), followed by the synchronous post-deploy-script: scripts/post-deploy-smoke.sh healthcheck gate (see Deploy-Pipeline Guards) — cd-cdk.yaml's stack-name input only accepts one stack, so the smoke script itself enumerates po-email-processor, workorder-email-processor, and procurement-api
• Plus dependency review and PR labeler workflows on every PR

Branch protection on main — all changes through PR.

Setup

Account prerequisites — the stacks import four dependencies by name, so all of these must exist in the target account BEFORE the first deploy (in seahaven-prod they are provisioned by the seahaven-org-baseline repo and the migration Phase 0 runbook):

• SES receipt rule set INBOUND_MAIL (may be inactive; the stacks attach their receipt rules to it) + a verified int.seahaven.com domain identity
• SNS topic site-alerts (with the alias/seahaven-alarm-topics CMK)
• SSM param /seahaven/dynamodb/cmk-arn -> KMS alias/seahaven-dynamodb
• Secrets Manager secret procurement-ingest/web-ui-auth-token (step 3)
• OIDC deploy role githubdeploy-procurement-ingest (see infra/deploy-role/) with the smoke-invoke-lambda policy, or the post-deploy smoke gate fails closed

1. Bootstrap CDK: cdk bootstrap aws://{AccountId}/us-east-1
2. Ensure the Bedrock inference profile us.anthropic.claude-haiku-4-5-20251001-v1:0 is enabled in us-east-1 (it is; the CDK grants cover cross-region routing to us-east-2/us-west-2). No API key or secret to set — the processors authenticate to Bedrock via their IAM roles.
3. Create the web UI auth-gate shared secret. This secret is imported by name (Secret.from_secret_name_v2), not CDK-managed, so it must exist before deploy or the web-ui Lambdas fail closed:

   aws secretsmanager create-secret --name procurement-ingest/web-ui-auth-token --secret-string "$(openssl rand -hex 32)"
   

4. Deploy both stacks:

   cd cdk
   pip install -r requirements.txt
   cdk deploy --all
   

5. Post-deploy cleanup (one-time): the retired RETAIN-policy secrets po-ingest/anthropic-api-key and workorder-ingest/anthropic-api-key are orphaned by this deploy, not deleted. Remove them and revoke the keys at the provider:

   aws secretsmanager delete-secret --secret-id po-ingest/anthropic-api-key --force-delete-without-recovery
   aws secretsmanager delete-secret --secret-id workorder-ingest/anthropic-api-key --force-delete-without-recovery
   

6. Dashboards: po-web-ui and workorder-web-ui have no public endpoint (the Function URLs were removed 2026-06-08, INFRA-74). A bare aws lambda invoke --function-name po-web-ui /tmp/out.json with no headers in the event is guaranteed a 401 — the handler fails closed (see Web UI auth). Fetch the token and forward it in the event's headers:

   TOKEN=$(aws secretsmanager get-secret-value --secret-id procurement-ingest/web-ui-auth-token --query SecretString --output text)
   aws lambda invoke --function-name po-web-ui --payload "{\"headers\":{\"x-auth-token\":\"$TOKEN\"}}" /tmp/out.json
   

Tests

Offline unit tests (no AWS, no network) run via pytest from the repo root:

pip install -r tests/requirements.txt
pytest

Test-root consolidation (Phase 8). Both test roots are kept (tests/ for genuinely cross-pipeline suites, plus each pipeline's own lambdas/*/email_processor/tests/), but loading is now single-sourced. A new repo-root conftest.py — not tests/conftest.py, which is never an ancestor of the pipeline test roots and so cannot load for a standalone pytest lambdas/po/email_processor/tests run — sets the dummy AWS env, imports moto before any handler import (registers moto's botocore stubber hook so boto3 sessions created afterwards are stubbable), and exposes the one load_lambda_module(pipeline, name) loader (implemented in tests/support/loader.py) that every handler exec in the suite goes through, including its sys.modules save/restore dance for the per-pipeline duplicated bare names (template_parser, persistence, etc. — derived_fields/prompts/telemetry/extraction/enrichment too). tests/support/ is a small shared package (FakeTable, FakeDynamoResource, FakeS3, load_email, load_golden) used by both pipelines' _po_parser_support.py/_wo_parser_support.py shims — load_golden decodes JSON money with parse_float=Decimal unconditionally (load-bearing for PO's exact-money goldens; proven safe for WO, whose 55 golden fixtures contain zero float-typed JSON numbers). test_local.py (a root-level manual script that imported a handler at collection time, bypassing the loader gate, and globbed a nonexistent samples/ dir) is deleted — the golden suites cover its role.

Coverage:

• tests/ — shared handler + cross-pipeline tests: parse_raw_email (test_parse_raw_email.py), the fail-closed sender-authentication parser (test_ses_auth.py, INFRA-107), the Phase 0 CDK-bundling/handler-import AST consistency check (test_bundle_consistency.py — see Deploy-Pipeline Guards), the scripts/reprocess.py synthetic S3-event-shape contract (test_reprocess_contract.py, Phase 7), and, new in Phase 8: the handler-level SES-auth reject seam per pipeline (test_handler_auth_seam.py — no auth monkeypatch + an empty ALLOWED_DKIM_DOMAINS must yield zero Bedrock calls, zero writes, no raise), and both web_ui functions' first coverage — fail-closed auth (test_web_ui_auth.py) and the handlers themselves, including a 401-without-a-table-scan assertion and a hostile-field escaping regression lock (test_web_ui_handlers.py).
• lambdas/wo/email_processor/tests/ — the deterministic WO parser suite: golden-file tests over 55 real scrubbed .eml fixtures (test_parser.py), fail-closed validation-gate rules and adversarial/injection cases (test_validation_gate.py), the issue #23 comment_id idempotency invariants (test_comment_id.py), the Bedrock-fallback dispatch/EMF-metric behavior with a mocked invoke_model (test_bedrock_fallback.py, which also carries the Phase 5 behavior pins — WO's mutually-exclusive emit and the [0-9]+ key guard ahead of both saves), and the Phase 0 direct-invoke healthcheck contract (test_healthcheck.py). Golden JSON lives under tests/fixtures/expected/. New in Phase 8: test_wo_bedrock_transport.py (transport errors — throttle, missing content, empty content, non-JSON model text — plus the hand-computed no-double-count pin for the handler.py except-and-reraise metric wrap, and the multi-record failure-isolation pin) and test_wo_merge.py (a moto-backed mirror of test_po_merge.py against the WorkOrders table — null-status never clobbers wo_status, created_at immutable, status→wo_status mapping, None fields absent from SET, record_type only-when-present). After Phase 5 the suite patches accessors on the owning siblings (extraction.bedrock, persistence.dynamodb, persistence.datetime, persistence.save_*) rather than on handler.
• lambdas/po/email_processor/tests/ — the deterministic PO parser suite: golden-file tests over real scrubbed .eml fixtures (17 single-line new-PO + 8 cancellations, exact Decimal-aware golden comparison via parse_float=Decimal), fail-closed validation-gate coverage for every gate reason code (fixture-driven for body-level triggers under fixtures/adversarial/, direct validate() unit tests for candidate-level mutations), real multi-line and comment/non-Coupa fallback fixtures under fixtures/ai-fallback/, dual line-ending (CRLF/LF) parse-identity, two-path enrich_parsed/save_new_po parity (the site-extractor stream-contract guard), fixture hygiene (ses_auth pass + scrub-marker leak sweep), the Bedrock-fallback dispatch/EMF-metric behavior plus the Phase 5 behavior pins (test_po_bedrock_fallback.py — the pre-Bedrock ai_fallback emit and the additive rejected double-count), the _save_merge collapse parity to both save_new_po/save_revision (test_po_save_merge_parity.py), the ai_fallback-only shadow telemetry after the enrich_parsed move (test_po_derived_wiring.py, which gains the PO-DC-02 64-char po_number clamp regression pin in Phase 8), and the Phase 0 direct-invoke healthcheck contract (test_po_healthcheck.py). New in Phase 8: test_po_bedrock_transport.py (the same four transport-error cases as WO, asserting the pre-call ai_fallback metric survives and no partial write occurs, plus the multi-record failure-isolation pin) and, moved here from tests/: test_po_merge.py (#97, moto-backed merge-write semantics) and test_pad_zip.py (zip-code padding). After Phase 5 the suite patches accessors/constants on the owning siblings (po_extraction.bedrock/.BEDROCK_MODEL_ID/._EMAIL_TAG_RE, po_persistence.dynamodb/.PO_TABLE, po_enrichment.derive_all/.pad_zip, po_telemetry.DERIVED_METRIC_NAME) rather than on po_handler; re-exported names it calls (save_*, extract_with_claude, enrich_parsed, _emit_parse_method_metric, EXTRACTION_PROMPT) stay on po_handler.

All three roots are discovered by pytest.ini (testpaths).

Coverage floor (Phase 8). pytest.ini's addopts runs pytest-cov with an explicit --cov path per first-party package (lambdas/po/email_processor, lambdas/wo/email_processor, lambdas/po/web_ui, lambdas/wo/web_ui, lambdas/po/site_extractor, lambdas/shared) rather than relying on an __init__.py package marker — none of these dirs have one, and adding one would perturb the CDK bundling asset-hash fingerprint for zero runtime benefit; pytest-cov's path form measures by source file regardless of package markers. site_extractor is deliberately included even though it measures 0% until Phase 6 lands — coverage honesty, not a silent skip. .coveragerc omits */tests/* and */cdk.out/* so the pipeline test dirs (which sit inside their own --cov path) don't dilute the number. --cov-fail-under is the new permanent CI floor (constraint 10 — never ratcheted down).

Ruff C901/PLR floor (Phase 8). ruff.toml adds extend-select = ["C901", "PLR"] with max-complexity = 12, making the tree's pre-existing # noqa: PLR09xx suppressions load-bearing instead of inert (no config previously enabled the rules they suppressed). scripts/ is in the lint scope. Findings that could not be split or were out of this phase's file-ownership were resolved with a per-file [lint.per-file-ignores] entry carrying a written justification — most notably derived_fields.py (shadow-bake freeze: even an in-file noqa comment is a barred edit) and the other Phase 3/5/6/7 frozen modules (enrichment.py, WO template_parser.py, both ses_auth.py/emf.py, site_extractor/handler.py, scripts/reprocess.py). PO's template_parser.py — this phase's one in-scope split target — clears the ceiling by decomposing extract_new_po and _validate_new_po_values into per-rule helpers instead of an ignore.

Scripts

Reprocess emails (re-invoke an email-processor with a synthetic S3 event). reprocess.py is now pipeline-general: --pipeline po|wo selects the function + raw-email bucket. Targeted replay (--key one object, --prefix, or --since a LastModified timestamp) is the default, preferred mode; the full-prefix sweep is demoted behind an explicit --all. Every mode is dry-run unless --execute. See the DLQ recovery runbook for the targeted single-key re-invoke flow.

python scripts/reprocess.py --pipeline po --key inbound/2026/msg.eml            # targeted, dry-run
python scripts/reprocess.py --pipeline po --key inbound/2026/msg.eml --execute  # targeted, re-invoke one
python scripts/reprocess.py --pipeline wo --all --execute                       # demoted full-prefix sweep (see --all caveats)

Backfill verified sites (one-time scan of historical POs):

python scripts/backfill_sites.py

Replay SHOC webhooks (rebuild work-order webhook events from DynamoDB and re-POST them, marked "replay": true). replay_shoc_webhooks.py is the operator runbook for the workorder-shoc-emitter-failures / -rejected alarms: when deliveries were parked (SHOC down past the 24h retry window, or a contract-bug 4xx), replay re-sends the affected work orders from current table state — receivers dedupe on the deterministic delivery_id, so overlapping or repeated runs are harmless. Selection is exactly one of --work-order-id (repeatable, targeted) or --since (a full table Scan — a count banner prints per table); --events narrows to state rows, comments, or both. --url is required with no default — replay must be a deliberate act against a known receiver. Every mode is dry-run unless --execute.

python scripts/replay_shoc_webhooks.py --url https://... --work-order-id 11144580730            # targeted, dry-run first
python scripts/replay_shoc_webhooks.py --url https://... --work-order-id 11144580730 --execute  # then re-POST
python scripts/replay_shoc_webhooks.py --url https://... --since 2026-07-24T02:00:00Z --execute # everything touched since (full Scan)

Directory Structure

cdk/
  app.py               # Three stacks: po-ingest + WorkorderIngestStack + procurement-api (region-only env)
  common.py            # Phase 4: shared plain-function CDK helpers (alarms, Bedrock grant,
                        # email bucket, processor DLQ, fallback-rate alarm) -- called with
                        # each stack's own scope + literal construct ids, logical-ID-safe
  po_stack.py          # Purchase order pipeline resources
  wo_stack.py          # Work order pipeline resources + the SHOC webhook feed (secret/CMK,
                        # rotator, emitter, disabled ESMs, queues, alarms)
  procurement_api_stack.py # REST API (IAM SigV4 + resource policy) over both pipelines' tables + token-gated /docs
lambdas/               # Phase 2: shared Code.from_asset("../lambdas") bundling root for
                        # BOTH po-email-processor and workorder-email-processor (and, since
                        # Phase 3, both web_ui functions) -- each email-processor command
                        # `cp`s its own po/email_processor/*.py (or wo/...) subset PLUS
                        # `cp shared/*.py`; each web_ui command `cp`s its own dir PLUS only
                        # `shared/web_ui_auth.py`; site_extractor keeps its narrower, non-
                        # bundled asset root
  shared/              # Phase 3: single-sourced first-party modules, landed FLAT (no
                        # __init__.py -- bare-name imports) into each bundle by the cp above
    ses_auth.py          # fail-closed SES sender-auth (INFRA-107) -- one copy, one fix
    web_ui_auth.py       # fail-closed X-Auth-Token gate + token cache (INFRA-74; also gates the API /docs routes)
    email_parsing.py     # parse_raw_email (WO superset; returns cc unconditionally)
    emf.py               # generic CloudWatch EMF emitter (dimension-sets pinned once)
  api/                 # procurement-api Lambda (REST reads over both pipelines' tables)
    handler.py           # healthcheck early-return + docs-token gate + route dispatch + error mapping
    router.py            # single route table (spec-drift test pins it to openapi.json)
    pagination.py        # opaque base64(LastEvaluatedKey) cursor encode/validate (hostile cursor -> 400)
    serialization.py     # Decimal-safe JSON responses
    wo_repo.py           # WorkOrders/WorkOrderComments reads (paginated Scan / PK Query)
    po_repo.py           # purchase-orders/verified-sites reads (no VendorReplies -- dead table)
    openapi.json         # OpenAPI 3.1 source of truth (paths + outbound `webhooks` section)
    docs.html            # Redoc shell, SHOC design-system theme; handler inlines bundle/fonts/spec at request time (single token-gated request, no CDN)
    redoc.standalone.js  # vendored Redoc bundle (redoc 2.5.3, MIT), inlined into /docs; read-only docs, no try-it-out
    fonts.css            # SHOC fonts (DM Sans/Montserrat/JetBrains Mono, @fontsource latin subsets) as data URIs, inlined into /docs
  po/                  # PO pipeline Lambdas
    email_processor/     # Phase 5: God-handler decomposed into flat siblings (bare-name
                         # imports; the Phase 0/2/3 `cp <pipeline>/email_processor/*.py`
                         # glob ships every new sibling automatically)
      handler.py         # event loop + fail-closed SES auth + email_type routing + {"healthcheck": true} early-return; lazy s3 accessor; re-exports EXTRACTION_PROMPT (4 tests deref handler.EXTRACTION_PROMPT)
      extraction.py      # extract_with_claude + _EMAIL_TAG_RE + EXTRACTION_PROMPT import; lazy bedrock accessor
      enrichment.py      # enrich_parsed + pad_zip (PO-only; no boto3); derived-field shadow block
      telemetry.py       # EMF ParseMethod + DerivedFieldAgreement emit wrappers (stdout EMF; no boto3)
      persistence.py     # _write_fields/_merge_update + save_cancellation; save_new_po/save_revision collapsed into one behavior-identical _save_merge (sticky-cancel guard intact); lazy dynamodb accessor
      prompts.py         # EXTRACTION_PROMPT (~181 lines); cross-ref header to derived_fields' rule tables
      template_parser.py # pure deterministic Coupa parser + fail-closed validation gate; Phase 8:
                         # extract_new_po/_validate_new_po_values split into C901/PLR-clean per-rule
                         # helpers (behavior byte-identical; see the ruff C901/PLR floor below)
      derived_fields.py  # deterministic site_code/trade/fiscal_year classifier (untouched by Phase 3/5/8 -- shadow-bake freeze; ruff per-file-ignore instead of an in-file noqa)
      tests/             # golden-file + validation-gate + fallback-dispatch + healthcheck + save-merge-parity + behavior-pin tests + fixtures
        conftest.py                  # fake_dynamo fixture, patches persistence.dynamodb
        _po_parser_support.py        # Phase 8: thin shim -- re-exports tests.support fakes/loader; PO module handles (template_parser, prompts, telemetry, extraction, enrichment, persistence)
        test_po_derived_fields.py    # derived_fields.derive_all() unit tests
        test_po_derived_wiring.py    # ai_fallback-only shadow-telemetry wiring; Phase 8 gains the PO-DC-02 64-char po_number clamp regression pin
        test_po_bedrock_transport.py # Phase 8: Bedrock transport errors (throttle/missing-content/empty-content/non-JSON) + pre-call metric survival + multi-record failure-isolation pin
        test_po_merge.py             # Phase 8: moved from tests/ -- PO merge-write semantics tests (#97), moto-backed
        test_pad_zip.py              # Phase 8: moved from tests/ -- PO zip-code padding tests
    site_extractor/
    web_ui/              # handler.py imports `from web_ui_auth import is_authenticated` (shared)
  wo/                  # WO pipeline Lambdas
    email_processor/     # Phase 5: decomposed into ~5 flat siblings (no enrichment stage);
                         # validate_ai_fallback + the [0-9]+ work_order_id key guard stay in
                         # the handler loop AHEAD of both saves
      handler.py         # event loop + fail-closed SES auth + [0-9]+ work_order_id key guard + {"healthcheck": true} early-return; lazy s3 accessor; re-exports EXTRACTION_PROMPT; Phase 8: the Bedrock call is wrapped in try/except so a transport error emits one ai_fallback/bedrock_error datapoint then re-raises (a gate rejection still emits only ai_fallback_rejected -- no double-count)
      extraction.py      # extract_with_bedrock + _EMAIL_TAG_RE + EXTRACTION_PROMPT import; lazy bedrock accessor
      telemetry.py       # emit_parse_metric EMF wrapper (stdout EMF; no boto3)
      persistence.py     # save_work_order + _header_date_iso + save_event (#23 comment_id determinism kept WITH persistence); lazy dynamodb accessor
      prompts.py         # EXTRACTION_PROMPT; cross-ref header to template_parser.CONTRACT_KEYS
      template_parser.py # pure deterministic parser + fail-closed validation gate; Phase 8: the
                         # template-path bad-status branch now returns "invalid_status" (was a
                         # copy-paste "malformed_site_code") -- matches validate_ai_fallback's code
                         # for the identical condition
      tests/             # golden-file + validation-gate + comment_id + fallback + healthcheck + behavior-pin tests
        conftest.py                  # fake_dynamo fixture, patches persistence.dynamodb (module-top `from _wo_parser_support import wo_persistence`, Phase 8)
        _wo_parser_support.py        # Phase 8: rewritten OFF the bare `import handler` strategy -- re-exports tests.support fakes/loader; WO module handles (template_parser, persistence, extraction, telemetry)
        test_wo_merge.py             # Phase 8: moto-backed WorkOrders save_work_order merge-semantics mirror of test_po_merge (null-status never clobbers wo_status, created_at immutable, status->wo_status mapping, None fields absent, record_type only-when-present)
        test_wo_bedrock_transport.py # Phase 8: Bedrock transport errors + the hand-computed emitted-series no-double-count pin for the handler.py except-and-reraise wrap + multi-record failure-isolation pin
        fixtures/
          ses-stamped/
            auth-pass-01.eml         # Phase 8: the one new fixture allowed this phase -- a synthesized Authentication-Results header block (from WO_SES_HEADER in test_ses_auth.py), not scraped mail
    web_ui/              # handler.py imports `from web_ui_auth import is_authenticated` (shared)
    shoc_emitter/        # SHOC webhook emitter (ACTIVE since 2026-07-30; see the SHOC
                         # webhook feed section). Flat siblings, bare-name imports
      handler.py         # thin per-record event loop + partial-batch failure report + rejected-queue parking
      envelope.py        # pure stream-record -> envelope mapping + event classification (incl. echo guard)
      delivery.py        # TTL-cached secret fetch + HMAC signing (sign_body) + POST + response classification
    shoc_hmac_rotator/
      handler.py         # 30-day Secrets Manager rotation (single-user: createSecret/finishSecret;
                         # dual-key overlap, kid YYYY-MM-DDTHH)
scripts/
  reprocess.py
  backfill_sites.py
  replay_shoc_webhooks.py # rebuild + re-POST webhook events ("replay": true) -- the
                          # -failures/-rejected alarm runbook; dry-run unless --execute
  post-deploy-smoke.sh   # CD gate: synchronous healthcheck invoke of the three smoke-gated functions, checks FunctionError
conftest.py             # Phase 8: THE repo-root session-invariant conftest. rootdir is pinned by
                        # pytest.ini at repo root, so this loads for every pytest invocation shape --
                        # including a standalone `pytest lambdas/po/email_processor/tests` -- before
                        # any collection import. Sets the dummy AWS env, imports moto BEFORE any
                        # handler import (registers moto's botocore stubber hook so boto3 sessions
                        # created afterwards are stubbable), and re-exports
                        # `tests.support.loader.load_lambda_module`. Owns the session fixtures
                        # (po_handler, wo_handler, email_handler, ses_auth, po_persistence,
                        # po_enrichment, wo_persistence). Supersedes the old tests/conftest.py
                        # (deleted -- it was never an ancestor of the pipeline test roots, so it
                        # could not carry these invariants for a standalone pipeline-root run).
tests/
  requirements.txt          # Test-only deps (moto, pytest-cov)
  support/                   # Phase 8: shared test package (tests/ itself has no __init__.py --
                             # PEP-420 namespace resolution via the root conftest's sys.path insert)
    __init__.py                # load_lambda_module + REPO_ROOT re-export, FIXTURE_DIRS, stems/
                                # load_raw/load_email/load_golden (parse_float=Decimal, pinned safe
                                # for both pipelines), and the superset FakeTable/FakeDynamoResource/FakeS3
    loader.py                   # the ONE load_lambda_module + sys.modules save/restore dance
                                # (dependency-topological sibling tuple + web_ui_auth); every
                                # pipeline handler exec in the suite goes through this
  test_parse_raw_email.py   # MIME parsing tests (PO + WO handlers) -- stays at root, cross-pipeline
  test_ses_auth.py          # Sender-authentication parser tests (INFRA-107) -- stays at root, cross-pipeline
  test_handler_auth_seam.py # Phase 8: handler-level SES-auth reject seam, per pipeline -- no auth
                             # monkeypatch + empty ALLOWED_DKIM_DOMAINS -> zero Bedrock calls, zero
                             # writes, no raise (closes the "delete the gate line, tests still pass" hole)
  test_web_ui_auth.py       # Phase 8: shared/web_ui_auth.py -- fail-closed on unset ARN / Secrets
                             # Manager exception, TTL cache refresh, header-matrix case-insensitivity,
                             # non-ASCII-token documenting pin
  test_web_ui_handlers.py   # Phase 8: both web_ui handlers (0% coverage before this phase) -- 401
                             # without a table scan, authenticated render path, hostile-field escaping
                             # regression lock
  test_bundle_consistency.py  # AST check: bundling command ships every handler.py sibling import
  test_reprocess_contract.py  # reprocess.py synthetic S3-event-shape contract (Phase 7)