Commit graph

14 commits

Author SHA1 Message Date
Adam Moussa
97a5886139
Land safe fixes from 2026-06-17 security sweep (#97)
Some checks are pending
Deploy / deploy (push) Waiting to run
* Remove gratuitous KMS grant on shared DynamoDB CMK

wo-email-processor held grant_encrypt_decrypt on the shared
seahaven-dynamodb CMK, but the WorkOrders/WorkOrderComments tables
are not encrypted with that CMK. The grant was dead weight that
extended the WO processor's decrypt reach to the CMK protecting the
purchase-orders table (cross-stack decrypt). Drop it to restore
least privilege; re-add as part of the table CMK migration (INFRA-6).

Refs: INFRA-6

* Require Secrets Manager key for Anthropic client

Remove the silent fallback to a plaintext ANTHROPIC_API_KEY env var
in both email processors; require ANTHROPIC_API_KEY_SECRET_ARN and
raise if absent so a misconfigured deploy fails loudly instead of
using an unmanaged key.

Adapted from f175323 on security/sweep-2026-06-17. The From-header
sender-domain allowlist from that commit is intentionally dropped:
the From header is spoofable (INFRA-107, confirmed critical) and
sender authentication is being reworked in a separate PR.

Refs: INFRA-107

* Merge PO revisions and handle out-of-order events

save_revision did a full put_item overwrite, so a revision omitting
line_items/supplier permanently deleted them. save_new_po used a
conditional put that silently dropped the PO when an out-of-order
cancellation had already created a skeleton row.

Switch both to field-level merge update_items: a revision now SETs
only the fields it carries, and a new_po backfills data into a
pre-existing Cancelled skeleton while preserving the Cancelled
status. No email can now delete data established by an earlier one.

* Gate web UIs behind auth and escape currency XSS

The po-web-ui and workorder-web-ui handlers had no auth: any
invocation path returned the full PO/WO DB. Add a fail-closed
shared-secret gate (X-Auth-Token / Bearer, constant-time compared to
WEB_UI_AUTH_TOKEN) so a future re-attached Function URL cannot
re-expose the data (URLs removed under INFRA-74). Wire the token from
the SSM String param /procurement-ingest/web-ui-auth-token.

Also fix stored XSS in po-web-ui fmt_currency: the non-numeric
fallback returned str(val) unescaped, so a prompt-injected email
could make Claude emit total_amount as <script>. Escape it.

Refs: INFRA-74

* Document sweep security fixes and merge semantics

Update the README for the 2026-06-17 security sweep: required
Secrets Manager key (no plaintext env fallback), web UI auth gate +
SSM token setup step, output-escaping note, and the new PO
revision/cancellation merge behavior.

Adapted from d91f45e on security/sweep-2026-06-17; the sender
allowlist documentation is dropped along with the allowlist itself
(deferred to the INFRA-107 sender-authentication rework).

Refs: INFRA-107

* fix: resolve web UI auth token from Secrets Manager at runtime

Replace the plaintext SSM String parameter with a Secrets Manager secret
referenced by ARN only. The token is fetched and cached at module level on
first invocation, keeping shared secrets out of CloudFormation templates and
Lambda environment variables.

Refs: PR-97

* Add TTL to web UI auth token cache for rotation

The web-ui handlers cached the Secrets Manager auth token at module
level with no expiry, so a rotated secret was only picked up when the
warm container recycled — an emergency rotation could take hours to
take effect. Cache the fetched value for a 5-minute TTL instead, so a
rotated token propagates within the TTL while still avoiding a Secrets
Manager call on every request. Still fails closed when the secret is
unset or unreadable.

Refs: INFRA-74

* Log Secrets Manager failures in web UI auth token fetch

The web UI auth gate correctly fails closed when the shared token
cannot be read, but _get_auth_token() swallowed every exception
silently. A Secrets Manager permission or config error then made
every request 401 with no operational signal, leaving an outage
indistinguishable from ordinary unauthenticated traffic.

Add a module-level logger to both web_ui handlers and log the
fetch failure with logger.exception() in the except block before
returning None. Behavior is unchanged (still fails closed); the
failure is now visible in CloudWatch. The secret value is never
logged. The two handlers stay byte-consistent in the mirrored
_get_auth_token() region.

The companion finding on the CDK import of the shared
procurement-ingest/web-ui-auth-token secret was evaluated and left
as-is: the token is a single secret shared by both the PO and WO
stacks, so from_secret_name_v2 (which scopes grant_read via the
standard 6-char suffix wildcard) is correct; making it a CDK-managed
Secret in both stacks would collide the two stacks on the same
explicit secret name at deploy time.

Refs: INFRA-74

* Make Cancelled PO status sticky via atomic write

The PO merge path read status with a get_item (_is_cancelled) and then
wrote with an unconditional update_item. Two defects followed from this:

- Race (Issue A): a cancellation landing between the read and the write
  was silently un-cancelled by a revision carrying a non-cancelled
  po_status — a TOCTOU on a table with concurrent email processing.
- Over-broad strip (Issue B): save_revision dropped po_status whenever
  the PO was Cancelled, so legitimate status updates on non-cancelled
  POs and status-less revisions were affected rather than only the true
  un-cancel transition.

Enforce the invariant server-side instead. "Cancelled" is a sticky,
authoritative status: once set, later new_po/revision emails may enrich
other fields but must never move it to a non-cancelled status. When the
payload carries a non-cancelled po_status, _merge_update issues the
update_item guarded by ConditionExpression "attribute_not_exists(po_status)
OR po_status <> :marker", evaluated atomically at write time, so a
cancellation that lands first always wins. On ConditionalCheckFailedException
the same fields are re-written without po_status/cancelled_at, enriching the
record while Cancelled sticks. Payloads with no status change, or an already
-Cancelled status, take a plain merge — the status is only ever suppressed on
a real un-cancel. This removes the non-atomic get_item from the write path;
_is_cancelled is deleted. Key schema and attribute names are unchanged, so the
cross-stack purchase-orders contract (read-only by seahaven-slack-bot) holds.

Add moto-backed tests covering un-cancel suppression with field enrichment,
status-less merge onto a Cancelled PO, legitimate status updates on
non-cancelled POs, new_po backfill of a Cancelled skeleton, fresh
create/merge, and authoritative save_cancellation.

Refs: #97
2026-07-15 20:17:46 -04:00
Adam Moussa
7b90f54815
Correct stale architecture facts in README (#96)
The README had drifted from the deployed stacks: it still listed
the removed shared/ directory, the by-state / site-code-index /
status-index GSIs deleted in the 2026-06-03 audit (M-20), the
WebUIUrl CloudFormation output retired with the public Function
URLs (INFRA-74), and it attributed the shared DynamoDB CMK to the
work-order tables when only purchase-orders is migrated (the WO
migration is still tracked in INFRA-6). Setup steps also told the
reader to create the CDK-managed secrets by hand, which would
collide on deploy. Align all of it with cdk/po_stack.py,
cdk/wo_stack.py, and the actual repo layout, and document the DLQ
messages alarms that were missing from the alarms section.

Refs: INFRA-74
2026-07-15 19:00:43 -04:00
Adam Moussa
7e1b89306b
docs: document cross-stack DynamoDB data contracts (INFRA-138) (#91)
Some checks failed
Deploy / deploy (push) Has been cancelled
2026-07-08 16:22:38 -04:00
Adam Moussa
3f42078f81
docs: link Confluence AWS Architecture Map (INFRA-53) (#83)
Some checks are pending
Deploy / deploy (push) Waiting to run
2026-07-06 17:43:51 -04:00
Adam Moussa
0fdf407e1d
Add CloudWatch alarm coverage for po-ingest and workorder-ingest (#70)
Some checks are pending
Deploy / deploy (push) Waiting to run
* Add CloudWatch alarm coverage for po-ingest and workorder-ingest

Expands alarm coverage across both CDK stacks. All alarms are ALARM-only
(no OK action) to the shared site-alerts SNS topic, with TreatMissingData
NOT_BREACHING. The site-alerts topic is now imported once near the top of
each stack so every alarm reuses one Topic instance.

po-ingest (cdk/po_stack.py):
- Errors: po-ingest-site-extractor
- Throttles: po-email-processor, po-ingest-site-extractor, po-web-ui
- Duration (p99, >=45000ms, eval3/dp2): po-email-processor (orphan adoption),
  po-ingest-site-extractor, po-web-ui
- DynamoDB throttle + system-error: purchase-orders, verified-sites,
  pending-site-review

workorder-ingest (cdk/wo_stack.py):
- Throttles: workorder-email-processor
- Duration (p95, >=45000ms, eval3/dp2): workorder-email-processor (orphan adoption)
- DynamoDB throttle + system-error: WorkOrders, WorkOrderComments

DynamoDB ThrottledRequests/SystemErrors emit only at the TableName+Operation
dimension set, so each table alarm is a Sum math expression across operations
via the non-deprecated metric_*_for_operations helpers (metric_throttled_requests
is deprecated/invalid in aws-cdk-lib 2.259.0).

Refs INFRA-41 / audit H-8.

* Drop NEEDS ADAM SIGN-OFF wording from alarm comments

Duration alarm thresholds are owner-approved; remove the sign-off flag
from po_stack.py and wo_stack.py comments. Threshold values, eval config,
and orphan-delete notes are unchanged.
2026-06-17 14:46:03 -04:00
Adam Moussa
4cc049e958
Repo hygiene: PR labeler + README badges + dependabot (INFRA-56/57/66) (#57)
Some checks are pending
Deploy / deploy (push) Waiting to run
2026-06-11 14:44:27 -04:00
Adam Moussa
109c565cbf
Reconcile IaC with out-of-band DLQ + Function URL changes (INFRA-74, INFRA-41) (#50)
Some checks are pending
Deploy / deploy (push) Waiting to run
Make CDK the source of truth for two sets of changes applied out-of-band
via CLI to the po-ingest and WorkorderIngestStack stacks.

INFRA-74 (audit C-5): remove the public FunctionUrlAuthType.NONE Function
URL construct (and its auto-generated Principal:* invoke permission +
output) from both po-web-ui and workorder-web-ui. The URLs were already
deleted live via CLI; CFN's delete is idempotent.

INFRA-41 (audit H-8): add a CDK-managed SQS dead-letter queue
(dead_letter_queue=, 14d retention, SSL-enforced, CDK-generated name) and
an ALARM-only Errors alarm (Sum, threshold>0, site-alerts topic) for both
po-email-processor and workorder-email-processor, mirroring the
apm-wo-analysis-classifier DLQ and payments-payroll-batch alarm patterns.

Interim CLI resources (per-fn -dlq queues, -errors alarms, dlq-send inline
policies, OnFailure event-invoke-configs) removed post-deploy.
2026-06-08 16:02:29 -04:00
Adam Moussa
bd0682661f
Document purchase-orders table ownership (INFRA-1) (#49)
Some checks failed
Deploy / deploy (push) Has been cancelled
Add a Shared Resources section recording that this stack owns the
purchase-orders DynamoDB table and is the authoritative writer, with
payments-dashboard and seahaven-slack-bot as read-only consumers.
Note the cross-repo schema-coordination rule and the amazon-po-parser
direct-write exception being folded in under INFRA-51.
2026-06-05 17:26:22 -04:00
Adam Moussa
5112c1345b
Merge workorder-ingest into unified procurement repo (#22)
* Merge workorder-ingest pipeline into unified repo

Move PO lambdas under lambdas/po/, add WO pipeline under lambdas/wo/.
Two independent CloudFormation stacks in one CDK app. Fix WO stack
compliance: ARM64 architecture, 60-day log retention, aarch64 bundling,
RETAIN on Anthropic secret. Remove stale CodePipeline buildspec.

* Fix test_local.py import path and remove dead shared/models.py

test_local.py referenced the old lambdas/email_processor path. Updated
to lambdas/wo/email_processor. Removed shared/ directory entirely as
nothing imports from it.

* Escape HTML in both web UI dashboards to prevent XSS

Both Function URLs are public (auth_type=NONE) and render
email-derived content via f-strings. Attacker-crafted emails
could inject scripts. Added html.escape() on all interpolated
values in both PO and WO dashboards.

* Add pagination to WO web UI scan

get_work_orders() only fetched the first 1MB page from DynamoDB.
Loop on LastEvaluatedKey to match the PO web UI pattern.

* Fix esc(None) TypeError and javascript: scheme in PO web UI

Coerce supplier name through `or ""` before escaping to handle
nested None from DynamoDB. Add scheme allowlist on view_order_url
to block javascript:/data: hrefs from LLM-extracted URLs.

* Fix WO render_badge None guard, updated_at slice, and backfill path

Add null guard to WO render_badge matching the PO version. Use
`or ""` before slicing updated_at to handle explicit None values.
Fix backfill_sites.py sys.path to use new lambdas/po/site_extractor.

* Harden WO web UI and fix JS-context XSS in both dashboards

- Use json.dumps for onclick URLs to prevent JS string breakout
- Add .lower() to WO render_badge color lookup matching PO pattern
- Add pagination to get_comments query
- Cap get_work_orders to 500 results matching PO pattern

* Apply ruff formatting to web UI handlers
2026-05-12 15:21:06 -04:00
Adam Moussa
e66062c934 Align PO schema with enriched records and improve extraction prompt
Replaces extraction prompt with domain-specific rules: trade
classification taxonomy (23 categories), site_code skip list,
zip padding, revision email type, and structured extraction for
fiscal_year, trade, and coupa_category.

Handler changes:
- New "revision" email type overwrites existing PO via put_item
- enrich_parsed() adds top-level state, ship_to_raw, data_source
- pad_zip() zero-pads short zip codes (e.g., "7001" → "07001")
- Removed invoice_total/invoice_count (Payee Central only)

Web UI: added revision badge, new detail fields (site code, state,
trade, fiscal year, coupa category, data source), line item table
now shows Qty/Unit/Price columns, list view shows Site and Trade.

CDK: fixed StreamViewType to match deployed table (NEW_IMAGE).
README: documented PO record schema and revision flow.
2026-05-01 19:53:26 -04:00
Adam Moussa
36f49ae259
Add CI/CD pipeline and fix stack name to kebab-case (#3)
* Add CI/CD pipeline and fix stack name to kebab-case

CodePipeline V2 (po-ingest-pipeline) triggers CodeBuild on push
to main, running cdk deploy via buildspec.yml. Stack name changed
from PoIngestStack to po-ingest to match naming conventions.

* Add RETAIN policy to Secrets Manager secret

Prevents the Anthropic API key from being deleted if the stack
is ever removed. Matches the RETAIN policy on all other stateful
resources (DynamoDB tables, S3 bucket).
2026-05-01 19:17:19 -04:00
Adam Moussa
2d79c81e7d Update README with pending-site-review table and address fallback docs 2026-04-30 15:03:43 -04:00
Adam Moussa
194b1aec44 Update README with verified-sites pipeline and architecture changes 2026-04-30 14:30:54 -04:00
Adam Moussa
fef240695c Add README 2026-04-20 19:31:22 -04:00