* fix(shoc-emitter): skip empty-text comment webhooks
Stop delivering #nocomment# WorkOrderComments rows as comment_added events
so they no longer 400-park in the rejected queue.
* style(shoc-emitter): ruff-format blank-comment handler test
* feat(webhook): ACTIVATE SHOC WO webhook emitter (enabled=True)
The deliberate one-line activation flip (plan Phase 3). Turns on both
DynamoDB stream event-source mappings for workorder-shoc-emitter, which
shipped dark (enabled=False) in PR-2. LATEST start position => the live
feed begins at deploy, no historical flood; SHOC loads history via the
procurement read API first.
DO NOT MERGE until: (1) PR-2 (feat/shoc-wo-webhook) is merged to main;
(2) SHOC's receiver passes the shared HMAC test vectors
(docs/shoc-webhook-test-vectors.json) at the target endpoint; (3) the
cross-account secret read from shoc-backend-dev is confirmed working.
Draft, gated on Luby.
* docs(webhook): stamp activation date and correct backfill wording
* docs(webhook): adjust activation comment date to 2026-07-30.
Signed-off-by: Adam Moussa <adam@seahavenind.com>
* docs(readme): mark SHOC webhook emitter active as of 2026-07-30
---------
Signed-off-by: Adam Moussa <adam@seahavenind.com>
* iac(access): temporary shoc-assessment-dynamo-reader role for Luby initial assessment
30-day, read-only (GetItem/Query/Scan/DescribeTable) cross-account role in
seahaven-prod trusting seahaven-external-dev, trust-policy hard expiry
2026-08-23. Steady state remains procurement-api + webhook; teardown script
included.
* harden(access): resolve sh-security-review findings on assessment reader
C1 (confirmed medium): DateLessThan expiry condition duplicated into both
permissions statements so in-flight sessions die at the deadline, not +1h.
C2 (confirmed medium): teardown now strips ALL inline/attached policies and
instance profiles before DeleteRole (kill-switch semantics restored,
idempotent), emergency-revocation section added to README.
Cheap hardenings: CDPATH-immune SCRIPT_DIR, MaxSessionDuration re-asserted
on update path, data-handling expectations documented.
* harden(access): address Open SWE review on #141
- Trust now requires ArnLike aws:PrincipalArn on the Identity Center role
path (human SSO sessions only; string condition survives permission-set
reprovisioning, unlike a role-ARN Principal pin)
- README extend instructions cover BOTH expiry sites (trust + permissions)
- Create script logs caller ARN + timestamp and validates the policy's KMS
ARN against SSM /seahaven/dynamodb/cmk-arn before applying
* feat(api): custom domain procurement-api.seahaven.com for the read API
Stacked on feat/shoc-wo-webhook. Gives the SHOC-facing read API a stable,
brandable endpoint instead of the opaque execute-api URL.
- procurement_api_stack.py: REGIONAL API Gateway DomainName (TLS 1.2) +
empty base-path mapping to the prod stage, so callers hit
https://procurement-api.seahaven.com/work-orders (no /prod segment). The
ACM cert ARN is read from SSM (/procurement-api/custom-domain/certificate-arn)
via value_for_string_parameter, because the seahaven.com zone is in the
mgmt account (cross-account DNS) and the cert is issued out of band. Outputs
expose the regional alias target + hosted-zone id for the mgmt A-record.
- scripts/setup_procurement_api_domain.sh: idempotent two-step runbook
(cert: request + mgmt-zone validation + wait + SSM; alias: post-deploy
A-record from stack outputs). Verifies both account identities.
- handler._base_url: omit the /{stage} segment for a custom-domain request
(the base-path mapping serves the stage at the root) so the docs never
advertise a broken server URL; execute-api hosts keep /{stage}.
- openapi.json: custom domain added as servers[0] (recommended), execute-api
kept as the direct fallback + the per-request injection target.
No IAM/auth/policy change (same API id + resource policy), so the SigV4
surface and the mandatory cross-family gates are unaffected. 751 pytest,
ruff, cdk synth, redocly lint all green.
* fix(api): use .endswith('.amazonaws.com') instead of substring check for execute-api detection
The prior '.execute-api.' in domain substring check is fragile and
triggers CodeQL incomplete-sanitization warnings. All API Gateway default
domains end with .amazonaws.com, so a suffix check is more precise and
also silences the false-positive alert.
Refs: https://github.com/Sea-Haven-Industries/procurement-ingest/security/code-scanning/6
* docs(webhook): revise SHOC webhook contract and plan for post-migration reality
Branch re-cut on main 2026-07-23 (old base carried stale PR #99 commits).
Contract Rev 2026-07-23:
- Producer account corrected: seahaven-prod (011934824531); mgmt frozen
- Reconciliation backstop is the new procurement read API, not SyncController
- wo_status "unknown" is real; SHOC must map it (checklist item added)
- write_origin forward-compat note for phase-2 write-back echo suppression
- SyncVendorReplies retirement flagged (dead table, no vendor_reply event)
Plan updates:
- Account gate: seahaven-prod only; never enable streams on mgmt tables
- Emitter ships DARK (ESMs enabled=False); activation is a deliberate flip
after the SHOC receiver passes shared HMAC vectors
- Post-refactor conventions: common.py helpers, bundle-consistency AST pins,
pytest.ini --cov additions, consolidated test roots
- Dedicated-CMK rationale, secret-ARN handooff step, consumer audit refreshed
(slack-bot decommissioned), enum golden test, write_origin skip-branch test
* feat(webhook): SHOC WO webhook emitter — dark-ship streams, HMAC secret + rotation
Implements docs/shoc-webhook-plan.md Phases 1-5 (PR-2 of the SHOC
call-and-be-called effort). Everything ships DARK: both DynamoDB event
source mappings deploy enabled=False; activation is a deliberate
one-line follow-up PR gated on the SHOC receiver passing the shared
HMAC test vectors.
- Streams: NEW_AND_OLD_IMAGES on WorkOrders + WorkOrderComments
(in-place update, RETAIN + logical IDs untouched; no existing
consumers — verified live, neither table had a stream).
- workorder-shoc-emitter (Py3.12/ARM64): stream -> envelope ->
HMAC-signed POST per docs/shoc-webhook-contract.md; strict per-shard
ordering (parallelization 1, bisect off, retry until 24h age,
ReportBatchItemFailures); 429/5xx/timeout block the shard in order,
other 4xx park to workorder-shoc-emitter-rejected; ESM failures ->
workorder-shoc-emitter-failures (metadata; replay rebuilds from
DynamoDB). Echo guard skips write_origin=shoc-write-api.
- Secret workorder-ingest/shoc-webhook-hmac on a dedicated CMK
(alias workorder-ingest-shoc-webhook-kms); cross-account
GetSecretValue/DescribeSecret + kms:Decrypt granted to exactly
arn:aws:iam::396287094661:role/shoc-backend-dev. RemovalPolicy
DESTROY deliberately (machine-generated material; avoids the
fixed-name RETAIN-orphan deadlock).
- workorder-shoc-hmac-rotator: 30-day rotation, dual-key overlap,
64-hex keys, kid = UTC %Y-%m-%dT%H.
- Alarms (ALARM-only -> site-alerts): emitter errors/throttles/
duration + iterator-age (>=10 min) + failures/rejected queue
depth; rotator standard trio.
- scripts/replay_shoc_webhooks.py: dry-run-default operator replay
(rebuilds from tables, replay:true envelopes).
- Tests: 742 passing, 85.56% aggregate; golden HMAC vectors shared
with SHOC in docs/shoc-webhook-test-vectors.json (emitter + replay
signing pinned to identical vectors); bundle-consistency AST pins
for both new bundles.
- README: WO stack + webhook feed section, alarm table, runbooks;
removed stale seahaven-slack-bot consumer references.
* fix(webhook): kms:ViaService pins, https-only delivery, cross-account principal CI pin
GPT-4.1 cross-family review of the policy surface (no BLOCK): FIX applied
to the cross-account shoc-backend-dev Decrypt statement and both Lambda
role KMS grants (the key is only ever used via Secrets Manager); its
invariant-enforcement QUESTION answered durably with
tests/test_cross_account_principal_pin.py (any new foreign IAM principal
in cdk/ fails CI). Scanner mediums fixed: delivery.py and the replay
script now refuse non-https URLs (urllib follows file:// and http://).
SQS metadata-action and dynamodb:ListStreams NITs skipped: standard CDK
grant shapes; ListStreams has no resource-level scoping. The 4 gitleaks
HIGHs on docs/shoc-webhook-test-vectors.json are deliberate non-secrets
(shared receiver-verification vectors) suppressed machine-level with
justification.
* harden(webhook): resolve /sh-security-review findings (1 confirmed medium + cheap fixes)
High-recall detector fan-out (injection/authz/secrets-crypto/iac-iam/logic)
+ proof-or-kill verifier. Gate PASSES: 1 confirmed medium, 0 confirmed
critical/high. Confirmed finding fixed; several unverified-but-cheap
hardenings applied since the emitter ships dark and activation is weeks out.
- CONFIRMED medium (confused deputy): the rotation Lambda's generated
invoke permission for secretsmanager.amazonaws.com carried no
SourceAccount/SourceArn, so any account's Secrets Manager could invoke
the rotator. Patched the generated CfnPermission in place (a second
permission would be additive, not restrictive) to pin account + this
secret ARN.
- delivery + replay: refuse to follow receiver 3xx redirects (no-redirect
opener) so live X-SH-* auth headers can't be forwarded to a
receiver-chosen Location and an http:// Location can't slip past the
https guard. Fixed the "unfollowed 3xx" comment that was factually wrong.
- delivery: classify 401/403 as retryable (invalidate key cache + retry in
order) instead of parking -- transient auth failures (rotation outran the
TTL cache, clock skew) are availability events, not contract bugs.
- envelope: build_event now genuinely total (guarded eventID /
ApproximateCreationDateTime subscripts) per its own never-raise contract.
- handler: catch-all so an unexpected per-record error (e.g. SQS park
failure) reports only that record instead of failing the whole batch
(which would re-deliver every earlier success for 24h); per-invocation
emit/skip batch summary so a systemic silent drop is queryable/alarmable.
- rotator: narrow the AWSCURRENT-read except to ResourceNotFound/JSONDecode
(transient SM/KMS errors re-raise so the overlap key isn't silently
dropped); kid uniqueness checked against ALL retained kids with a random
suffix on collision (never reissue a kid for a different secret).
- contract: skeleton-upsert required on ANY unknown work_order_id (not just
comment-before-create) + monotonicity guard (ignore older updated_at), so
a parked created or an out-of-order replay can't corrupt receiver state.
Unverified/refuted findings left as-is with rationale: the two "high" logic
claims (whole-batch crash triggers, ordering violation) were refuted on
reachability (real stream records carry required fields; persistence writes
strings only; full-state idempotent upsert absorbs the ordering gap). Signed
kid/version binding (AUTHZ-002) declined: coordinated contract change, not
cheap, no exploit with one algorithm/key.
* fix(webhook): drop kid from rotator test_ok log (CodeQL clear-text-logging FP)
GHAS CodeQL flagged py/clear-text-logging-sensitive-data (high) at
_test_secret's success log because head["kid"] is subscripted from the
same parsed-secret dict that holds head["secret"] — the taint tracker
can't tell the non-secret key id from the secret. The secret value is
never logged. Rather than dismiss the alert (fragile; re-alerts on line
moves), remove the flow: kid is already logged at stage time in
_create_secret and version_id correlates the steps, so the test_ok log
keeps only event + version_id. Also hardens against a future edit that
swaps the logged field.
* feat(api): Add @redocly/cli as a dev dependency
Signed-off-by: Adam Moussa <adam@seahavenind.com>
* feat(api): Add Redocly configuration file with custom rules
Signed-off-by: Adam Moussa <adam@seahavenind.com>
* chore(api): Redocly lint config + bring openapi.json into compliance
redocly.yaml from the Redocly guidelines builder, with three generated
rules corrected: response-contains-property had the status codes as the
required body fields (intent was the Error schema's top-level 'error';
403 exempt since API Gateway emits AWS's {message} shape, 501 not 503);
operation-4xx-problem-details-rfc7807 off (adopting RFC 7807 would be a
runtime + SHOC-contract change, decided against); the two inert casing
rules (parameter names, schema properties) removed because both name
sets are contract-pinned (gateway resource paths, DynamoDB items).
Spec changes, no runtime impact: operationIds renamed to method-prefixed
kebab-case (get-work-orders, post-work-order-comment, ...); tags added to
all 15 operations + root tags object (groups the Redoc sidebar); examples
on all six parameters; license field; server description punctuation; two
descriptions reworded to start capitalized. Real linter catches fixed:
the two x-planned ops were missing their {workOrderId} path parameter
and any 4xx response (403 added - true today, gateway rejects unsigned).
.redocly.lint-ignore.yaml pins the six deliberate exceptions: webhook
keys are the shipped SHOC contract event names (not renameable), and the
x-planned ops answer only 501 (no 2xx to document).
package.json: npm run lint:api. Verified: lint 0 errors, 675 pytest,
headless-Chrome render of the tagged docs page.
* feat(api): SHOC design-system theme for /docs (vendored fonts)
Themes the Redoc page with the canonical SHOC token set: Montserrat 600
headings / DM Sans body / JetBrains Mono code, primary #1c75bc, navy
#262262 sidebar text + right panel, #f9fafb background, 244px sidebar.
sortRequiredPropsFirst on; 200 responses pre-expanded.
Fonts ship as lambdas/api/fonts.css (latin woff2 subsets from
@fontsource 5.3.0, embedded as data URIs, ~90KB) and inline via a new
__FONTS_CSS__ placeholder with the same </style breakout guard --
the offline single-response invariant holds, nothing fetches Google
Fonts (test-pinned). Bundling cp + bundle-consistency pin + spec-drift
asset checks extended.
Verified: headless-Chrome render (theme + fonts applied), ruff, 675
pytest, cdk synth + staged-asset check.
* feat(api): SHOC gradient topbar on /docs
64px fixed header with the SHOC shell gradient token (#1b1f52 ->
#1c4f8f -> #1c75bc), Sea Haven wordmark in Montserrat 600, page name
right-aligned in DM Sans. Redoc's scrollYOffset: 64 keeps the sticky
sidebar and anchor scrolling clear of the fixed bar. Verified via
headless-Chrome render.
* style(api): normalize /docs header and right-panel blues
The right panel's #262262 is a purple-leaning navy that clashed with
the cyan-leaning gradient, and the bar's brightest point sat directly
over the dark panel. Right panel now uses #1b1f52 (the gradient's own
dark endpoint) and the gradient runs bright-to-dark so its dark end
lands flush on the panel -- no seam, one blue family. Verified via
headless-Chrome render.
* style(api): right-panel gradient on /docs via bundle-pinned override
Redoc's theme only takes solid colors (it derives shades from
rightPanel.backgroundColor), so the gradient (#1b3d79 -> #1b3068 ->
#1b1f52, continuing the topbar blend) rides as a CSS override on the
styled-components classes of the per-section right-panel divs
(.sc-iGgWBj.sc-gsFSXq + the .sc-dExYaf stub). Those names are
deterministic for the vendored 2.5.3 bundle (verified across loads) but
change on any Redoc bump: re-derive via headless probe (find elements
whose computed background equals the rightPanel color). If they stop
matching, the panel falls back to the solid #1b1f52 theme color --
cosmetic only. Verified via headless-Chrome render.
* feat(api): collapsible samples column on /docs
Redoc CE has no built-in panel toggle, so the topbar gains a Hide/Show
samples button that flips .samples-collapsed on <html>: the right-panel
divs hide (same bundle-pinned styled-components classes as the gradient
override) and each section's content half takes the full width. Choice
persists in localStorage; aria-pressed tracks state. If the pinned
classes stop matching after a Redoc bump the toggle goes inert --
cosmetic only. Both states verified via headless-Chrome render.
* ci(api): spec-lint CI gate + npm Dependabot coverage
New spec-lint job mirrors the local npm run lint:api so openapi.json
cannot drift from redocly.yaml with green CI. Dependabot gains the npm
ecosystem (package.json is new; nothing watched @redocly/cli).
* feat(api): docs finishing touches - x-tagGroups, favicon, docs:preview
x-tagGroups sections the Redoc sidebar (Read API / Meta / SHOC Feed);
inline data-URI SVG favicon (SHOC blue) stops the browser's follow-up
/favicon.ico request 403ing at the gateway; npm run docs:preview wraps
the real-handler local render (scripts/preview_docs.py); README gains a
docs-page architecture section covering the inline pattern, theme,
pinned-selector caveat, and tooling. Lint 0 errors, 675 pytest,
headless render verified.
---------
Signed-off-by: Adam Moussa <adam@seahavenind.com>
Redoc 2.5.3 standalone bundle (MIT) replaces the three swagger-ui-dist
assets: one ~1.05MB JS file instead of ~1.8MB of JS+CSS+preset, and the
layout traps (StandaloneLayout/BaseLayout) go away. Redoc is read-only by
design, which matches the existing posture: try-it-out was already
disabled since data routes need SigV4 (Postman for live calls).
Unchanged: single token-gated response, offline vendoring (no CDN),
per-request server-URL injection, script-breakout guards, cached shell
with per-request spec splice. Bundle self-containment verified: the
search worker is an inlined Blob, and the only new Worker(filename) path
is Prism's async mode, which Redoc never invokes.
Verified via headless-Chrome render of the real handler output: all
routes, the OpenAPI 3.1 webhooks section, and planned-route markers
render; no placeholder leakage.
* feat(api): use stock Swagger UI for the /docs page
Replaces the custom renderer with vendored stock Swagger UI
(swagger-ui-dist 5.17.14, Apache-2.0), kept offline (no CDN) and inlined
server-side into the single token-gated /docs response alongside the spec.
BaseLayout (topbar hidden); try-it-out disabled since data routes need SigV4
(use Postman for live calls). Breakout guards on the inlined css/js/spec.
Bundle-consistency + spec-drift + handler tests updated for the two vendored
assets. cdk diff = Lambda code asset only (no IAM/API/policy change).
* fix(api): render Swagger UI with the canonical StandaloneLayout recipe
The BaseLayout-only init (apis preset, no standalone preset) rendered
incorrectly. Switch to the canonical swagger-ui-dist recipe: vendor
swagger-ui-standalone-preset.js and init with
presets:[apis, SwaggerUIStandalonePreset] + layout:"StandaloneLayout"
(topbar hidden, try-it-out disabled). Verified via headless Chrome against
the live deployed page: all 9 endpoints + 4 webhooks + models render.
Tests/bundling updated for the third vendored asset.
* fix(api): declutter the Swagger UI docs page
The page rendered (stock Swagger UI, StandaloneLayout) but looked cramped:
a dense info.description wall of inline-code chips collided across Swagger
UI's tight default line-height, and the Servers box showed a SEE-STACK-OUTPUT
placeholder.
- Trim info.description to a few concise lines (detail lives in README + the
webhook contract doc).
- Inject the live invoke URL into servers[0].url per request (from the API
Gateway request context; committed literal is the fallback), so the Servers
box shows the real endpoint and drops the server-variables section.
- CSS: loosen description line-height + inline-code padding so chips never
overlap; tidy the scheme container spacing.
- Handler: cache the heavy CSS/JS/preset shell once; splice the (server-
injected) spec per request.
Verified via headless Chrome against the live deployed page.
* feat(api): add procurement-api stack - read API + OpenAPI docs page
Third CDK stack: API Gateway REST API (IAM SigV4) over both pipelines'
tables, replacing SHOC's retired SyncController cross-account DynamoDB
scan as the reconciliation/backfill path.
- lambdas/api/: handler (healthcheck + docs-token gate + router dispatch),
router (single route table), pagination (opaque cursor, hostile -> 400),
Decimal-safe serialization, wo_repo/po_repo reads. No VendorReplies.
- OpenAPI 3.1 spec as source of truth incl. top-level webhooks section
documenting the outbound SHOC feed; phase-2 write endpoints x-planned
(router answers 501). Self-contained /docs page, no CDN.
- Auth: AWS_IAM on data routes + resource policy scoped to exactly
arn:aws:iam::396287094661:role/shoc-backend-dev on GET/*; /docs and
/openapi.json carve-out is token-gated in the Lambda via shared
web_ui_auth (fail-closed, INFRA-74 posture).
- KMS: explicit Decrypt/DescribeKey on the DynamoDB CMK from SSM
(name-imported table drops the key association - INFRA-104 class).
- Alarms: errors/throttles/duration(p99>=22.5s) + gateway 5xx, ALARM-only
to site-alerts. No access logging in v1 (docs ?token= shim stays out of
logs); cloud_watch_role=False.
- Tests: handler auth-seam + routing + Decimal round-trip; moto cursor
pagination incl. hostile cursors; spec<->router drift gate; bundle
AST pins for the api command; pytest.ini --cov + loader siblings.
- Deploy role: third stack DescribeStacks ARN + procurement-api smoke
invoke ARN (re-run create-deploy-role.sh before merge).
* harden(api): apply sh-security-review findings to procurement-api
Fan-out (6 detectors) + review findings resolved:
Correctness / DoS:
- pagination: require EXACT key-set match (was subset) so a partial/foreign
composite cursor can't reach DynamoDB as an inconsistent ExclusiveStartKey
-> ValidationException -> 500; comments Query now pins the cursor's
work_order_id to the path entity.
- handler: map botocore ValidationException to 400 (defense in depth) so a
crafted cursor can't drive the zero-threshold 5xx alarm.
- web_ui_auth: compare tokens as bytes; a non-ASCII presented token now fails
closed (401) instead of crashing hmac.compare_digest into a 500. Resolves the
pre-existing xfail(strict) follow-up test; hardens the web UIs too.
Docs page:
- typeStr() now escapes the one spec-derived string that reached innerHTML.
- spec inlined into the docs <script> block escapes "<" -> < (</script>
breakout guard); /openapi.json still served byte-faithful.
- Cache-Control: no-store + Referrer-Policy: no-referrer on docs responses so
the ?token= URL stays out of caches/Referer.
- spec-drift test asserts the committed spec carries no "</" / "<!--".
IAM / IaC:
- resource policy enumerates the 7 data GET resources instead of GET/* so a
future GET route can't silently inherit SHOC cross-account reach.
- kms:Decrypt grant gains a kms:ViaService=dynamodb condition.
- stage throttling (50 rps / 100 burst) bounds the unauthenticated /docs blast
radius below the 10k account default.
- corrected the PATCH/POST comment (same-account callers aren't blocked by the
resource policy; 501 handler + absent write grant are the gate).
- documented the RETAIN log-group first-deploy rollback trap and the
resource-policy-needs-redeploy gotcha in-stack.
Mandatory GPT-4.1 cross-family review of the full policy surface: no BLOCK/FIX.
675 tests pass, ruff clean, cdk synth green.
First seahaven-prod deploy failed CREATE on the sender-auth MetricFilter:
it imported /aws/lambda/<fn> by name, which pre-existed in mgmt but not in
a fresh account. All 5 functions now get an explicit logs.LogGroup
(TWO_MONTHS, RETAIN) via common.make_function_log_group, and the metric
filter takes the construct so CFN orders it after the group exists.
Also removes the deprecated LogRetention custom resource and its
wildcard logs:PutRetentionPolicy role (CKV_AWS_111). Function roles keep
AWSLambdaBasicExecutionRole (verified in the synthesized template), so
log-write permissions are unchanged; cross-review's grant_write FIX was
a false positive on that basis. Supersedes PR #84, which hardcoded the
mgmt logs-CMK ARN and predates the common.py refactor.
mgmt collision note: these CREATEs would collide with the pre-existing
groups in mgmt; acceptable because the deploy secret now targets prod
and mgmt is frozen pending decommission.
* feat(migration): prepare stacks and tooling for the seahaven-prod account move
Phase 1 of the mgmt (328440206208) -> seahaven-prod (011934824531)
migration. No behavior change in-account; everything here is additive or
account-portability hygiene:
- infra/deploy-role/: reviewed OIDC deploy-role artifacts for prod
(trust main-only, cdk-hnb659fds-* AssumeRole, smoke-invoke-lambda scoped
to exactly the two email-processor fn ARNs). Codifies the previously
out-of-band smoke-invoke grant.
- Table resource policies: make_slack_bot_read_policy in cdk/common.py,
applied to purchase-orders, verified-sites, WorkOrders,
WorkOrderComments (NOT pending-site-review; no bot consumer). Grants the
mgmt-resident seahaven-slack-bot roles read-only cross-account access
post-move (bot-side identity grants land in the slack-bot repo).
- scripts/migrate_tables.py: dry-run-default backfill tool implementing
the plan's per-table semantics (superset overwrite, ingested_at cutoff
for WorkOrderComments, backup-gated truncate-and-load for the two
site tables) plus a verify subcommand (count parity, spot checks,
sticky-Cancelled drift check).
- tests/test_resource_policy_helper.py: statement-shape unit tests +
static pins that exactly the four bot-read tables carry the policy.
- Account-literal fixes: account-agnostic fixture bucket in
test_reprocess_contract; runbook/README/po-template-parser account
references updated to prod with historical mgmt notes; README gains the
account-prerequisites list (imported-by-name dependencies).
deploy.yaml is deliberately unchanged (push-to-main auto-deploy kept).
Merge is held until migration Phase 0 completes; flipping the
AWS_DEPLOY_ROLE_ARN repo secret and merging this PR IS the first prod
deploy.
* fix(migration): verify backup AVAILABLE pre-truncate; document wildcard risk acceptance (cross-review FIX/NIT)
* refactor(migration): drop cross-account read grants (slack-bot decommissioned); harden backfill + deploy role
seahaven-slack-bot was decommissioned 2026-07-23 (stack DELETE_IN_PROGRESS,
consumer Lambdas gone); its successor sh-mcp is undeployed and uses
same-account DynamoDB access. So no live consumer reads these tables
cross-account. Per Adam's call, drop the cross-account grants entirely and
re-add correctly-scoped ones if/when sh-mcp deploys to a different account.
- Remove the four table resource policies + make_slack_bot_read_policy helper
+ its constants (cdk/common.py, po_stack.py, wo_stack.py) and the helper's
unit test. Both stacks synth with zero table ResourcePolicy.
- scripts/migrate_tables.py hardening (fixes from the sh-security-review
fan-out on the destructive backfill tool):
* validate --cutoff strictly (parse ISO-8601, require aware UTC, re-emit
canonical second-precision form) so a malformed cutoff can't silently
copy dual-window rows or drop history;
* reject `copy --all` up front (must run tables individually, in order,
with the stream-drain wait) instead of writing three tables then erroring;
* truncate backup gate now also checks recency (<1h) and TableId, not just
status+name;
* verify requires --cutoff whenever a cutoff table is in scope (else it
false-flags dual-window rows as MISSING);
* sticky-cancel is now PREVENTED copy-side (a non-Cancelled source item
never overwrites a dest-Cancelled PO), and the verify comment no longer
overstates what its source-side scan covers;
* spot-check all modes (truncate_load keys are verbatim, so key-existence
is sound there too).
- Deploy role: scope cloudformation:DescribeStacks to this repo's stacks +
CDKToolkit (was Resource:*, disclosed all tenant stacks in the shared prod
account); add a drift check warning on unexpected role policies and drop the
dead SMOKE_POLICY_NAME var; document the shared-account bootstrap-role
accepted risk in the deploy-role README.
* docs(deploy-role): fold in cross-review NITs (DescribeStacks maintenance note, warn-only drift rationale)
* ci: update workflow to use new workflow tag (ruff versioning fix)
* fix(migration): address Open SWE review findings on migrate_tables.py
- Validate the truncate backup on dry-run as well as --execute so a
missing/stale/wrong-incarnation --backup-arn surfaces on the rehearsal
run (finding f_24a48b8900).
- Assert configured keys match the live key schema of both tables before
any key projection, turning config/schema drift into a descriptive
abort instead of a mid-backfill KeyError (finding f_cb6b5a6c59).
- Clarify why key-existence spot-checks are sound for WorkOrderComments:
the copy Puts source items verbatim and the sample uses the same
cutoff filter, so per-account comment_id divergence never enters the
check (finding f_390b7d6c3b is a false positive; comment hardened).
* chore(deps): bump boto3 from 1.43.31 to 1.43.51 in /lambdas/po/web_ui
* chore(deps): bump boto3 from 1.43.37 to 1.43.51 in /lambdas/wo/web_ui
* chore(deps): bump boto3 from 1.43.31 to 1.43.51 in /lambdas/po/site_extractor
* chore(deps): bump constructs from 10.6.0 to 10.7.1 in /cdk
* test: consolidate test roots — one repo-root loader, shared support package, missing-scenario suites, enforced ruff/coverage floor (refactor phase 8)
tests/conftest.py only loads for the tests/ root, not a standalone
`pytest lambdas/po/email_processor/tests` run, so it could never carry
session invariants like the dummy AWS env or the moto stubber
registration. Add a single repo-root conftest.py (pytest.ini pins
rootdir there, so it loads for every invocation) that sets the dummy
AWS credentials/region, imports moto BEFORE any handler module so
boto3 sessions pick up its stubber hook (carrying the explanatory
comment verbatim from the old _po_parser_support.py), and exposes one
load_lambda_module(pipeline, name) — the sys.modules save/restore
dance stays, since template_parser is still a duplicated bare name
across pipelines needing per-exec sibling binding.
Add tests/support/ as the shared package both pipelines' local
_*_parser_support.py modules delegate to: a superset FakeTable (PO's
update_item recording + WO's put_item and keyed single-row store),
FakeDynamoResource, load_email, and load_golden with parse_float=Decimal
kept (load-bearing for exact money comparison at PO magnitudes — WO's
prior load_golden had no parse_float and must not regress PO by losing
it). Rewrite _wo_parser_support.py off the bare `import handler` /
`from handler import parse_raw_email` strategy that was the source of
the bare-name sys.modules collision the other two loaders defend
against.
Move test_po_merge.py and test_pad_zip.py into
lambdas/po/email_processor/tests/ (PO-specific, belongs beside the
code) via git mv so history follows; test_parse_raw_email.py and
test_ses_auth.py stay at the repo root since they're genuinely
cross-pipeline, parameterized over both handlers. Delete
tests/test_local.py: it globs a nonexistent samples/ dir, is WO-only,
and imports a handler at collection time, bypassing the loader gate
entirely — the golden suites already cover its role. Its pytest.ini
exclusion comment goes with it.
New scenario coverage, all built on the single loader + support
package:
- PO+WO Bedrock transport errors (ThrottlingException, missing
'content' key, empty content list, non-JSON model text), asserting
PO's pre-call ai_fallback metric survives with no partial write and
the exception propagates; WO's no-datapoint-on-throttle behavior is
pinned with a documenting test rather than "fixed" by reordering.
- Handler-level SES-auth reject seam per pipeline: no auth
monkeypatch + empty ALLOWED_DKIM_DOMAINS asserts zero Bedrock calls,
zero writes, no raise — closing the hole where deleting the gate
line today still passes every test.
- web_ui coverage for both PO and WO (0% before this): fail-closed on
unset ARN and on a Secrets Manager exception, TTL cache refresh,
Bearer/X-Auth-Token/header-case-insensitivity, wrong-token 401 with
no table scan, non-ASCII token, and a hostile-field-escaping
regression lock. PO web_ui has no __init__.py, so these go through
the loader rather than package imports.
- A moto-backed mirror of test_po_merge for WO merge semantics
(table 'WorkOrders'): null-status never clobbers wo_status,
created_at immutable via if_not_exists, status->wo_status mapping,
None fields absent from SET, record_type only-when-present.
- Small pins: the PO-DC-02 64-char EMF clamp regression and
per-pipeline multi-record failure-isolation (all-or-retry contract).
The reprocess.py synthetic-event-shape contract test already landed
in Phase 7, so it isn't duplicated here.
Two WO product-code fixes ride along, since this is the phase that
exercises them: (a) the invalid_status reason-code fix in
template_parser.py's status check, which previously returned
malformed_site_code for the same failure validate_ai_fallback already
labels invalid_status, making one failure surface two codes depending
on path (grepped the dashboards/metric filters for
malformed_site_code first — no external references found, safe to
diverge the two codes); (b) wrapping the WO Bedrock call in
handler.py so a transport failure emits ai_fallback/bedrock_error in
an except-and-reraise. This is deliberately not a naive reorder: the
emit sits in the except block, not pre-call, so a gate-rejected email
still emits only ai_fallback_rejected and wo_stack's "a rejected
email emits nothing else" alarm contract doesn't double-count. A test
computes the emitted series by hand to pin the no-double-count
behavior. Neither change touches the handler event/return contract.
_validate_new_po_values in the PO template_parser.py is split into
per-rule helpers, and the V4 anchor-frame dataclass now carries
summary_matches/price so V13 can consume them; extract_new_po
(C901=35) is included in the split. Add ruff.toml enabling C901/PLR
so the mccabe/complexity suppressions scattered through the tree stop
being decorative; derived_fields.py is under the shadow-bake freeze
so its violations are silenced via a per-file ignore with a
justification comment instead of an in-file edit, and the handful of
other pre-existing violations surfaced by turning the config on get
the same per-file-ignore treatment with a reason, or a fix where the
file isn't frozen. scripts/ is added to the CI lint scope.
CI gains an explicit --cov module list (lambdas/po and wo
email_processor + web_ui, po/site_extractor, lambdas/shared) plus
--cov-fail-under=80, since web_ui and site_extractor lack __init__.py
markers and a bare --cov=lambdas silently skips them for the missing
package marker; .coveragerc omits the test dirs themselves from the
count. The Phase 0 AST bundle-consistency test stays in the standard
pytest run. .gitignore picks up the resulting .coverage data file.
docs/po-template-parser.md gets a small correction: the EXTRACTION_PROMPT
declares quantity/price as "number or null", not JSON strings, so
parse_float=Decimal already handles a conforming Bedrock response —
the doc previously implied the coercion path was the primary
mechanism rather than a defensive net for non-conforming responses.
* test: lock attribute-context quote escaping in web_ui hostile-field test
The escaping regression lock asserted only the element-context vector
(raw <script> absent, <script> present) while its docstring claimed
quotes were covered -- the payload's " and ' were never asserted on, so
a quote-escaping regression on the onclick row-link sink (attribute
breakout -> event-handler injection) would have passed green.
/sh-security-review finding WC-01 (confirmed medium, test-integrity).
Add assertions that the onclick sink's JSON string renders its opening
quote as " (raw " after window.location= fails), that the
payload's quote characters appear only entity-escaped, and that the
raw payload never appears anywhere in the body. Mutation-verified: the
test now fails when the sink's quote-escaping is dropped.
* test: address Open SWE review — xfail the web_ui non-ASCII auth pin, document subset coverage-floor override
- tests/test_web_ui_auth.py: replace the TypeError characterization pin with an
xfail(strict, raises=TypeError) asserting the DESIRED fail-closed (False)
behavior. Documents the intended fix and auto-fails (xpass) once web_ui_auth is
corrected, instead of requiring a passing test to be knowingly deleted. The
module stays frozen this phase; the underlying hmac.compare_digest ASCII-only
defect is tracked as a follow-up.
- pytest.ini: document that the aggregate 80% floor (enforced in CI via the
reusable workflow's bare pytest) red-exits local subset runs by design, with the
--cov-fail-under=0 override for iteration. Floor stays in addopts because the
centralized ci-python-sam workflow exposes no per-run test command.
Both email-processor God-handlers split along the seams that already
work in the flat-sibling pattern established by lambdas/shared/, so
bare-name imports keep working under the existing bundling glob.
PO (5-way split): handler.py keeps only the event loop, fail-closed
auth, and email_type routing. extraction.py holds extract_with_claude
and _EMAIL_TAG_RE, importing EXTRACTION_PROMPT from prompts.py and
parse_raw_email from shared/email_parsing.py rather than recreating a
PO-local copy. enrichment.py is a pure code move of enrich_parsed and
pad_zip (PO-only; WO has no enrichment stage) with zero behavior
change. telemetry.py holds the EMF ParseMethod emit wrappers.
persistence.py holds _write_fields/_merge_update/save_*, collapsing
the byte-identical save_new_po/save_revision bodies into one
_save_merge helper that both now call through, preserving the sticky
Cancelled ConditionExpression guard for both callers; save_cancellation
stays separate.
WO (5 concerns, no enrichment stage): the handler loop keeps
validate_ai_fallback and the re.fullmatch(r"[0-9]+", work_order_id)
key guard ahead of both save_work_order and save_event, since the
guard protects the DynamoDB partition key and the '#'-delimited
comment_id range-key segment. _header_date_iso and comment_id
determinism stay colocated with persistence.py's save_event for the
retry-idempotent event_id key.
EXTRACTION_PROMPT (PO) moves to prompts.py with cross-reference
headers to derived_fields.py's authoritative trade/site/fiscal rule
tables; handler.py re-exports it (from prompts import
EXTRACTION_PROMPT) since four tests dereference handler.EXTRACTION_
PROMPT directly. WO's prompt moves the same way.
I/O modules (extraction.py's bedrock client, persistence.py's
dynamodb resource, handler.py's s3 client) get lazy cached boto3
accessors; pure modules (enrichment.py, prompts.py, telemetry.py)
import no boto3. Test monkeypatch surfaces move to the module that
now owns the client (e.g. persistence.dynamodb) everywhere tests
patch it, and the moto-before-handler-import ordering in
_po_parser_support.py is preserved so the moto-backed suites don't
hit real AWS.
Behavior-preservation pins, verified with tests: PO still emits
ParseMethod=ai_fallback before the Bedrock call, with
ai_fallback_rejected as the additive second datapoint on rejection.
WO still emits after its gate with mutually-exclusive ai_fallback /
ai_fallback_rejected. Shadow DerivedFieldAgreement telemetry stays
ai_fallback-only. derived_fields.py is untouched (diff against
feature/phase-3-shared-extraction is empty). handler(event, context)
signatures and the save_* public contract are unchanged on both
pipelines; goldens unchanged.
PO_EXPECTED_TOP_LEVEL_MODULES and its WO equivalent in
tests/test_bundle_consistency.py are updated for the new sibling
modules so the AST bundle-consistency test still fails on an
unshipped or uncommented-out sibling.
The ~379 lines po_stack.py and wo_stack.py defined identically (DynamoDB
alarms, the sender-auth-rejected metric filter + alarm, the standard
per-Lambda alarm set, the Bedrock InvokeModel grant, the raw-email
bucket, the async DLQ, the template-fallback-rate math alarm) move into
cdk/common.py.
Every helper is a PLAIN function taking (scope, id, ...), called with each
stack's own Stack as scope and the exact literal construct ids used inline
before, so every synthesized logical ID is byte-stable. A Construct
subclass would reparent the tree and make CloudFormation attempt to
replace the RETAIN-protected purchase-orders/WorkOrders tables and named
buckets -- data loss -- so it is forbidden. Per-function alarm variance
(PO p99 vs WO p95 duration, po-web-ui throttles+duration only,
site-extractor no DLQ alarm, workorder-web-ui zero alarms) is preserved
through call-site arguments, not baked into the helpers.
make_bedrock_invoke_statement derives the inference-profile and us-east-1
foundation-model ARNs from Stack.of(scope).account/.region instead of the
hardcoded 328440206208/us-east-1 literals. The environment stays
account-agnostic (region-only), so the account resolves to the
AWS::AccountId pseudo-parameter: the derived ARN resolves at deploy to the
same ARN the literal named in-account (a benign in-place IAM policy
update, never a replacement) and is account-portable rather than pinned to
the frozen management account.
The account= pin evaluated for cdk.Environment was deliberately NOT added:
resolving every account-derived value (bucket names, Lambda::Permission
source account, SNS action ARN) to literals makes CloudFormation flag the
RETAIN email buckets as requiring replacement against the deployed
account-agnostic templates -- a data-loss risk that outranks the pin, which
buys nothing (the resolved values are unchanged).
Also: net-new CfnOutputs for the five Lambda function ARNs and the
owned/consumed table names, exact-pin constructs==10.6.0, and fix the
stale aws-cdk-lib 2.259.0 -> 2.261.0 version comment.
The common.py extraction is zero-cdk-diff on both stacks (byte-stable
logical IDs, no asset/property change); the only deltas versus deployed
are the intended benign Bedrock IAM in-place update and the additive
CfnOutputs. Mandatory GPT-4.1 cross-family review ran on the Bedrock IAM
move; its BLOCK was a verified false positive (it read AWS::AccountId as a
wildcard -- it is a deploy-time-resolved concrete value naming one account
and one inference-profile, region is pinned us-east-1, and the grant is
strictly more least-privilege-correct than the hardcoded literal).
Four modules move into the handbook-mandated lambdas/shared/ location,
collapsing duplicated logic that had to be kept in sync by hand across
the PO and WO pipelines:
- ses_auth.py: the PO and WO copies were verified sha256-identical
against the feature/phase-7-ops-recovery baseline before the move
(no drift since the last audit). shared/ses_auth.py is the exact
bytes of that one copy; both originals are git rm'd (the PO copy
via rename, the WO copy as a straight delete). Bundling lands the
module flat in /asset-output for both email processors, so the
handlers keep `from ses_auth import authenticate_inbound_email`
unchanged — zero handler diff for this move, which is what keeps
fail-closed auth byte-identical through the change.
- web_ui_auth.py: extracts the byte-identical _get_auth_token /
_header / is_authenticated block plus the four token-cache globals
out of both web_ui handlers. The per-stack INFRA-74 comments stay
in each handler as-is (deliberately drifted wording, stack-specific)
rather than being unified into the shared module. Fail-closed
semantics (unset ARN or Secrets Manager exception -> deny) are
unchanged.
- email_parsing.py: parse_raw_email ships as the superset version that
returns cc unconditionally. WO's output is bit-identical to before;
PO simply ignores the cc field rather than being "cleaned up" to
consume it. No second variant is kept.
- emf.py: a generic emitter parameterized by namespace, dimension
sets, and properties. Every call site's emitted EMF envelope is
unchanged, including the load-bearing
[["ParseMethod"],["ParseMethod","TemplateId"]] dimension-set shape
the alarms and metric filters depend on. Emission ordering is
untouched: PO still emits ai_fallback before the Bedrock call, WO
still emits its mutually-exclusive ai_fallback/ai_fallback_rejected
after its gate. The deliberate-double-count comments survive.
_emit_derived_agreement_metric was found living inside
derived_fields.py, so per the DERIVED-FIELDS exception it is left
as a third, unconverted copy (derived_fields.py and the shadow
DerivedFieldAgreement telemetry stay untouchable while that bake
runs) — a comment there points at shared/emf.py for the eventual
follow-up.
Bundling: both email-processor cdk bundling commands gain a trailing
`cp shared/*.py /asset-output/` (they were already cp-only post-Phase
7, so no pip step or manylinux pin is reintroduced). Both web_ui
functions gain the same widened-root staging so web_ui_auth.py ships
beside their handler; site_extractor's from_asset is untouched.
Tests: PO_EXPECTED_TOP_LEVEL_MODULES gains the shared modules that now
ship, the AST sibling-import check resolves imports whose source now
lives under shared/, and the new shared cp line has its own
revert/mutation detection. _SIBLING_MODULES resolution and
_po_parser_support.py now load ses_auth/email_parsing/emf from
shared/; the two-copy ses_auth byte-identity fixture-hygiene test is
retired as obsolete now that there is one copy, and the ses_auth
fixture parameterization over two identical copies is dropped. The
sys.modules save/restore dance for template_parser (still duplicated
per-pipeline) is left in place.
* feat: ops/recovery tooling + dependency hygiene (refactor phase 7)
Generalize scripts/reprocess.py from a PO-only full-sweep script into a
pipeline-general recovery tool. Targeted replay (--key/--prefix/--since)
is now the default, and the full inbound/ sweep is demoted behind an
explicit --all that documents its five hazards (async concurrency does
not serialize, use RequestResponse if order matters, metric double-count,
Bedrock re-bill, out-of-order field regression). --pipeline po|wo resolves
the correct function + bucket; dry-run-by-default / --execute is preserved.
A new tests/test_reprocess_contract.py pins the synthetic S3 event shape
and asserts the raw list_objects_v2 key is emitted untransformed (the
handler is the single decode point; a pre-decoded key would corrupt keys
containing spaces or '+').
Add docs/runbook-dlq-recovery.md: the async on-failure DLQ has no console
redrive-to-source, so it documents the receive -> extract key -> targeted
reprocess --key -> verify -> purge procedure, the real recovery windows
(14-day DLQ breadcrumb, 90-day raw-email S3 that overrides the table
RETAIN policy and is the true replay floor), and that sender-auth and
ai_fallback_rejected drops are fail-closed skips that never reach the DLQ.
Linked from the README alarms and scripts sections.
Drop the vendored boto3 floor pin from both email-processor requirements
(the Lambda runtime provides boto3; lambda-template.md empty-with-comment
form). With nothing left to install, the email-processor bundling becomes
cp-only -- the whole pip step is removed, which is the only acceptable way
the manylinux2014_aarch64 pin disappears (removing the pin while keeping a
pip install caused the PR #34 x86-wheel outage). Exact-pin moto==5.2.2 and
add pinned po/web_ui + po/site_extractor manifests (excluded from their
bundles, so hash-neutral) so their new Dependabot entries have something
to act on; add Dependabot entries for /tests, /lambdas/po/web_ui, and
/lambdas/po/site_extractor.
cdk diff is confined to exactly the two email processors' asset hashes on
both stacks. The wo/web_ui dead-manifest reduction was deliberately left
out: that manifest already ships inside the plain (non-bundled) WebUI
asset on main, so reducing or excluding it would redeploy workorder-web-ui
for no functional change -- deferred to keep the blast radius to the two
intended targets.
The untracked 44 MB lambdas/po/email_processor/package/ dir was removed
from the filesystem (asset-hash-neutral given Phase 2's package/ exclude);
it is untracked, so there is nothing to commit for it.
* Reject --all combined with --prefix/--since in reprocess.py
--all is a distinct mode (the demoted full-prefix sweep), but the args.all
branch unconditionally set prefix=inbound/ and since=None, so passing it
alongside a narrower selector silently discarded that selector. `--all
--since 2026-07-01` swept the entire corpus instead of the bounded window,
triggering every documented --all hazard (Bedrock re-bill, metric double-
count, merged-field regression) on objects the operator never targeted --
contradicting the tool's safety goal. Add the missing mutual-exclusion
guard alongside the existing --key one, and pin --all+--prefix,
--all+--since, and all three together as argparse rejections.
Both email-processor Code.from_asset calls now bundle from lambdas/
instead of their per-function subdirectory, so Phase 3's shared/
module is reachable from the asset root once it lands. The bundling
commands were rewritten for the new cwd (pip install -r <po|wo>/
email_processor/requirements.txt -t /asset-output && cp <po|wo>/
email_processor/*.py /asset-output/), preserving the ARM64
--platform manylinux2014_aarch64 --only-binary=:all: pin exactly —
its removal shipped x86 wheels into the ARM64 function and caused a
100% outage (PR #34).
All five from_asset calls (both email processors, po web_ui, po
site_extractor, wo web_ui) now exclude **/__pycache__/**; the two
widened ones also exclude **/tests/** and **/package/**. Without the
package/ exclude, the stale untracked 44 MB
lambdas/po/email_processor/package/ dir (local-only, never present
in CI) would diverge local vs CI asset hashes and force spurious
redeploys — from_asset doesn't honor .gitignore. That dir is left in
place; deleting it is Adam's call.
WO's prod zip shrinks as deliberate cleanup, not a byte-identical
match to PO: the old `cp -r .` shipped tests/ (real scrubbed .eml
fixtures), __pycache__/, and requirements.txt into production. The
acceptance bar for WO is runtime-imported module set unchanged +
smoke, not a byte-identical zip; PO keeps the byte-identical
first-party file set guarantee. tests/test_bundle_consistency.py is
updated in the same change to recognize the scoped
`cp po/email_processor/*.py` (resp. wo) glob as the new
unconditionally-safe shape, without loosening the allowlist-revert
detection, the detection-logic mutation test, or the
PO_EXPECTED_TOP_LEVEL_MODULES exact-set pin.
No code moved under lambdas/ in this change (git diff main...HEAD --
lambdas/ is empty); only CDK asset wiring and its tests changed.
