From c0b68382c8bc104d6e15ff7522ac7e6bde577e60 Mon Sep 17 00:00:00 2001 From: Adam Moussa <166072409+amoussa1229@users.noreply.github.com> Date: Fri, 5 Jun 2026 13:57:53 -0400 Subject: [PATCH 1/4] chore(deps): remove blanket aws-cdk-lib dependabot ignore (#47) Per handbook Pinning Principle: exact pins are kept current by Dependabot version updates gated by CI + dependency review. Blanket ignores let pins rot (see today's fast-uri incident). --- .github/dependabot.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 0ed6f20..9717d1b 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -4,8 +4,6 @@ updates: directory: "/cdk" schedule: interval: "weekly" - ignore: - - dependency-name: aws-cdk-lib groups: minor-and-patch: update-types: From 64ff6f09ca9987ed762e331d35e6935838106a77 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 5 Jun 2026 14:49:27 -0400 Subject: [PATCH 2/4] Bump aws-cdk-lib in /cdk in the minor-and-patch group (#48) Bumps the minor-and-patch group in /cdk with 1 update: [aws-cdk-lib](https://github.com/aws/aws-cdk). Updates `aws-cdk-lib` from 2.257.0 to 2.258.0 - [Release notes](https://github.com/aws/aws-cdk/releases) - [Changelog](https://github.com/aws/aws-cdk/blob/main/CHANGELOG.v2.alpha.md) - [Commits](https://github.com/aws/aws-cdk/compare/v2.257.0...v2.258.0) --- updated-dependencies: - dependency-name: aws-cdk-lib dependency-version: 2.258.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: minor-and-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- cdk/requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cdk/requirements.txt b/cdk/requirements.txt index 392ba6a..5a1ff0d 100644 --- a/cdk/requirements.txt +++ b/cdk/requirements.txt @@ -1,2 +1,2 @@ -aws-cdk-lib==2.257.0 +aws-cdk-lib==2.258.0 constructs>=10.6.0 From bd0682661f8c902255d4a21d72202deb98fa97c5 Mon Sep 17 00:00:00 2001 From: Adam Moussa <166072409+amoussa1229@users.noreply.github.com> Date: Fri, 5 Jun 2026 17:26:22 -0400 Subject: [PATCH 3/4] Document purchase-orders table ownership (INFRA-1) (#49) Add a Shared Resources section recording that this stack owns the purchase-orders DynamoDB table and is the authoritative writer, with payments-dashboard and seahaven-slack-bot as read-only consumers. Note the cross-repo schema-coordination rule and the amazon-po-parser direct-write exception being folded in under INFRA-51. --- README.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 16f74f9..98033c3 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ Coupa PO emails are received at `amazon_po@int.seahaven.com`, parsed by Claude H | `po-web-ui` | Function URL | HTML dashboard | **Tables:** -- `purchase-orders` (PK: `po_number`, Streams: NEW_IMAGE) — shared with payments-dashboard and seahaven-slack-bot +- `purchase-orders` (PK: `po_number`, Streams: NEW_IMAGE) — shared with seahaven-slack-bot (read-only; see Shared Resources) - `verified-sites` (PK: `siteCode`, GSI: `by-state`) — ~1,100 unique Amazon facility sites - `pending-site-review` (PK: `po_number`) — unresolvable POs for manual Payee Central verification @@ -67,6 +67,24 @@ All Lambdas: Python 3.12, ARM64, 60-day log retention. **SES:** Both stacks add rules to the shared `INBOUND_MAIL` receipt rule set on `int.seahaven.com`. +## Shared Resources + +### `purchase-orders` table (owned here) + +The `purchase-orders` DynamoDB table is **owned by this repo's `po-ingest` stack** (defined in `cdk/po_stack.py` with `RemovalPolicy.RETAIN` and `StreamViewType.NEW_IMAGE`). The `po-email-processor` Lambda is the authoritative writer — it performs the conditional inserts, revision overwrites, and cancellation updates described above. + +**Consumers (read-only):** + +| Repo | How it reads | Purpose | +|---|---|---| +| `seahaven-slack-bot` | `po-sync` (DynamoDB Streams + daily scan) and `wo-po-lookup` | Daily KB sync + Bedrock agent PO lookups | + +The consumer imports the table via `Table.fromTableName(...)` and is granted read-only access (`grantReadData`); it does not own or define it. + +**Schema-coordination rule:** Any change to the `purchase-orders` schema (partition key, item shape, attribute names, streams view type) must be coordinated with `seahaven-slack-bot`. The owner here ships the change; the consumer must be updated in lockstep so its readers do not break. Treat schema changes as a cross-repo migration, not a local edit. + +**Known exception (INFRA-51):** `amazon-po-parser` currently writes directly to `purchase-orders` outside this stack (backfill/enrichment scripts). This second writer is being folded into the `po-ingest` pipeline so this stack is the sole writer; until INFRA-51 closes, coordinate any schema change with `amazon-po-parser` as well. + ## CI/CD GitHub Actions with reusable workflows from `Sea-Haven-Industries/.github`: From 109c565cbf0a24d2fef180e66f007ae8111cd379 Mon Sep 17 00:00:00 2001 From: Adam Moussa <166072409+amoussa1229@users.noreply.github.com> Date: Mon, 8 Jun 2026 16:02:29 -0400 Subject: [PATCH 4/4] Reconcile IaC with out-of-band DLQ + Function URL changes (INFRA-74, INFRA-41) (#50) Make CDK the source of truth for two sets of changes applied out-of-band via CLI to the po-ingest and WorkorderIngestStack stacks. INFRA-74 (audit C-5): remove the public FunctionUrlAuthType.NONE Function URL construct (and its auto-generated Principal:* invoke permission + output) from both po-web-ui and workorder-web-ui. The URLs were already deleted live via CLI; CFN's delete is idempotent. INFRA-41 (audit H-8): add a CDK-managed SQS dead-letter queue (dead_letter_queue=, 14d retention, SSL-enforced, CDK-generated name) and an ALARM-only Errors alarm (Sum, threshold>0, site-alerts topic) for both po-email-processor and workorder-email-processor, mirroring the apm-wo-analysis-classifier DLQ and payments-payroll-batch alarm patterns. Interim CLI resources (per-fn -dlq queues, -errors alarms, dlq-send inline policies, OnFailure event-invoke-configs) removed post-deploy. --- README.md | 6 ++++-- cdk/po_stack.py | 52 +++++++++++++++++++++++++++++++++++++++++-------- cdk/wo_stack.py | 52 +++++++++++++++++++++++++++++++++++++++++-------- 3 files changed, 92 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index 98033c3..6954744 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ Coupa PO emails are received at `amazon_po@int.seahaven.com`, parsed by Claude H |---|---|---| | `po-email-processor` | S3 ObjectCreated | Claude extraction + DynamoDB write | | `po-ingest-site-extractor` | DynamoDB Streams | Site code/address extraction -> `verified-sites` | -| `po-web-ui` | Function URL | HTML dashboard | +| `po-web-ui` | Manual invoke | HTML dashboard (public Function URL removed 2026-06-08, INFRA-74) | **Tables:** - `purchase-orders` (PK: `po_number`, Streams: NEW_IMAGE) — shared with seahaven-slack-bot (read-only; see Shared Resources) @@ -49,7 +49,7 @@ Amazon APM work order emails (from Hexagon EAM / HxGN SmartCloud) are received a | Function | Trigger | Purpose | |---|---|---| | `workorder-email-processor` | S3 ObjectCreated | Claude extraction + DynamoDB write | -| `workorder-web-ui` | Function URL | HTML dashboard | +| `workorder-web-ui` | Manual invoke | HTML dashboard (public Function URL removed 2026-06-08, INFRA-74) | **Tables:** - `WorkOrders` (PK: `work_order_id`, GSIs: `site-code-index`, `status-index`) @@ -67,6 +67,8 @@ All Lambdas: Python 3.12, ARM64, 60-day log retention. **SES:** Both stacks add rules to the shared `INBOUND_MAIL` receipt rule set on `int.seahaven.com`. +**Failure handling (INFRA-41):** Each email-processor is async-invoked (S3 → Lambda). Both have a CDK-managed SQS dead-letter queue (`dead_letter_queue=`, 14-day retention, SSL-enforced) so a failed parse is captured rather than silently dropped after Lambda's retries, plus an ALARM-only CloudWatch `Errors` alarm (Sum, threshold > 0) wired to the shared `site-alerts` SNS topic. + ## Shared Resources ### `purchase-orders` table (owned here) diff --git a/cdk/po_stack.py b/cdk/po_stack.py index cf0ea33..144d9e9 100644 --- a/cdk/po_stack.py +++ b/cdk/po_stack.py @@ -5,6 +5,8 @@ from aws_cdk import ( Duration, RemovalPolicy, Stack, + aws_cloudwatch as cloudwatch, + aws_cloudwatch_actions as cw_actions, aws_dynamodb as dynamodb, aws_lambda as lambda_, aws_lambda_event_sources as lambda_event_sources, @@ -14,6 +16,8 @@ from aws_cdk import ( aws_ses as ses, aws_ses_actions as ses_actions, aws_secretsmanager as secretsmanager, + aws_sns as sns, + aws_sqs as sqs, ) from constructs import Construct @@ -58,6 +62,18 @@ class PoIngestStack(Stack): removal_policy=RemovalPolicy.RETAIN, ) + # --- DLQ for failed async invocations (INFRA-41 / audit H-8) --- + # SES → S3 → Lambda is async; without an OnFailure destination a failed + # parse (bad email, transient error) is silently dropped after Lambda's + # retries. CDK generates the queue name to avoid colliding with the + # interim CLI-created po-email-processor-dlq (removed post-deploy). + email_processor_dlq = sqs.Queue( + self, + "EmailProcessorDlq", + retention_period=Duration.days(14), + enforce_ssl=True, + ) + # --- Lambda function --- email_processor = lambda_.Function( self, @@ -82,6 +98,7 @@ class PoIngestStack(Stack): timeout=Duration.seconds(60), memory_size=256, log_retention=logs.RetentionDays.TWO_MONTHS, + dead_letter_queue=email_processor_dlq, environment={ "PO_TABLE": "purchase-orders", "ANTHROPIC_API_KEY_SECRET_ARN": anthropic_secret.secret_arn, @@ -93,6 +110,29 @@ class PoIngestStack(Stack): po_table.grant_read_write_data(email_processor) anthropic_secret.grant_read(email_processor) + # --- Errors alarm (INFRA-41 / audit H-8) --- + # ALARM-only (no OK action, per the CloudWatch-alarm preference) to the + # shared site-alerts topic (CMK alias/seahaven-alarm-topics lives on the + # topic). Any errored invocation in a 5-min window pages. + alarm_topic = sns.Topic.from_topic_arn( + self, + "SiteAlertsTopic", + f"arn:aws:sns:{self.region}:{self.account}:site-alerts", + ) + email_processor.metric_errors( + period=Duration.minutes(5), + statistic="Sum", + ).create_alarm( + self, + "EmailProcessorErrorsAlarm", + alarm_name="po-email-processor-errors", + alarm_description="po-email-processor async invocation errors", + threshold=0, + evaluation_periods=1, + comparison_operator=cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD, + treat_missing_data=cloudwatch.TreatMissingData.NOT_BREACHING, + ).add_alarm_action(cw_actions.SnsAction(alarm_topic)) + # S3 event notification → Lambda email_bucket.add_event_notification( s3.EventType.OBJECT_CREATED, @@ -138,14 +178,10 @@ class PoIngestStack(Stack): po_table.grant_read_data(web_ui) - # Function URL for direct access - web_url = web_ui.add_function_url( - auth_type=lambda_.FunctionUrlAuthType.NONE, - ) - - cdk.CfnOutput( - self, "WebUIUrl", value=web_url.url, description="PO Dashboard URL" - ) + # Public Function URL removed 2026-06-08 (INFRA-74 / audit C-5): the + # unauthenticated FunctionUrlAuthType.NONE URL was deleted out-of-band + # via CLI. Removing the construct (and its auto-generated Principal:* + # invoke permission) reconciles IaC with the live state. # --- Verified sites table (extracted from PO ship-to addresses) --- verified_sites_table = dynamodb.Table( diff --git a/cdk/wo_stack.py b/cdk/wo_stack.py index 37355ac..4cba41b 100644 --- a/cdk/wo_stack.py +++ b/cdk/wo_stack.py @@ -5,6 +5,8 @@ from aws_cdk import ( Duration, RemovalPolicy, Stack, + aws_cloudwatch as cloudwatch, + aws_cloudwatch_actions as cw_actions, aws_dynamodb as dynamodb, aws_lambda as lambda_, aws_logs as logs, @@ -13,6 +15,8 @@ from aws_cdk import ( aws_ses as ses, aws_ses_actions as ses_actions, aws_secretsmanager as secretsmanager, + aws_sns as sns, + aws_sqs as sqs, ) from constructs import Construct @@ -73,6 +77,18 @@ class WorkorderIngestStack(Stack): removal_policy=RemovalPolicy.RETAIN, ) + # --- DLQ for failed async invocations (INFRA-41 / audit H-8) --- + # SES → S3 → Lambda is async; without an OnFailure destination a failed + # parse (bad email, transient error) is silently dropped after Lambda's + # retries. CDK generates the queue name to avoid colliding with the + # interim CLI-created workorder-email-processor-dlq (removed post-deploy). + email_processor_dlq = sqs.Queue( + self, + "EmailProcessorDlq", + retention_period=Duration.days(14), + enforce_ssl=True, + ) + # --- Lambda function --- email_processor = lambda_.Function( self, @@ -97,6 +113,7 @@ class WorkorderIngestStack(Stack): timeout=Duration.seconds(60), memory_size=256, log_retention=logs.RetentionDays.TWO_MONTHS, + dead_letter_queue=email_processor_dlq, environment={ "WORK_ORDERS_TABLE": work_orders_table.table_name, "COMMENTS_TABLE": comments_table.table_name, @@ -110,6 +127,29 @@ class WorkorderIngestStack(Stack): comments_table.grant_read_write_data(email_processor) anthropic_secret.grant_read(email_processor) + # --- Errors alarm (INFRA-41 / audit H-8) --- + # ALARM-only (no OK action, per the CloudWatch-alarm preference) to the + # shared site-alerts topic (CMK alias/seahaven-alarm-topics lives on the + # topic). Any errored invocation in a 5-min window pages. + alarm_topic = sns.Topic.from_topic_arn( + self, + "SiteAlertsTopic", + f"arn:aws:sns:{self.region}:{self.account}:site-alerts", + ) + email_processor.metric_errors( + period=Duration.minutes(5), + statistic="Sum", + ).create_alarm( + self, + "EmailProcessorErrorsAlarm", + alarm_name="workorder-email-processor-errors", + alarm_description="workorder-email-processor async invocation errors", + threshold=0, + evaluation_periods=1, + comparison_operator=cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD, + treat_missing_data=cloudwatch.TreatMissingData.NOT_BREACHING, + ).add_alarm_action(cw_actions.SnsAction(alarm_topic)) + # S3 event notification -> Lambda email_bucket.add_event_notification( s3.EventType.OBJECT_CREATED, @@ -156,11 +196,7 @@ class WorkorderIngestStack(Stack): work_orders_table.grant_read_data(web_ui) comments_table.grant_read_data(web_ui) - # Function URL for direct access - web_url = web_ui.add_function_url( - auth_type=lambda_.FunctionUrlAuthType.NONE, - ) - - cdk.CfnOutput( - self, "WebUIUrl", value=web_url.url, description="Work Order Dashboard URL" - ) + # Public Function URL removed 2026-06-08 (INFRA-74 / audit C-5): the + # unauthenticated FunctionUrlAuthType.NONE URL was deleted out-of-band + # via CLI. Removing the construct (and its auto-generated Principal:* + # invoke permission) reconciles IaC with the live state.