mirror of
https://github.com/Sea-Haven-Industries/procurement-ingest.git
synced 2026-09-30 09:33:15 +00:00
Document sweep security fixes and merge semantics
Update the README for the 2026-06-17 security sweep: sender allowlist + SES verdict checks, required Secrets Manager key, web UI auth gate + SSM token setup step, output-escaping note, and the new PO revision/cancellation merge behavior.
This commit is contained in:
parent
e97e740c5a
commit
d91f45ebc7
1 changed files with 23 additions and 7 deletions
30
README.md
30
README.md
|
|
@ -16,11 +16,12 @@ Coupa PO emails are received at `amazon_po@int.seahaven.com`, parsed by Claude H
|
||||||
1. Coupa sends a PO email (new, revision, or cancellation).
|
1. Coupa sends a PO email (new, revision, or cancellation).
|
||||||
2. SES (`INBOUND_MAIL` rule set) drops the raw MIME into `s3://po-ingest-emails-{AccountId}/inbound/`.
|
2. SES (`INBOUND_MAIL` rule set) drops the raw MIME into `s3://po-ingest-emails-{AccountId}/inbound/`.
|
||||||
3. S3 `ObjectCreated` triggers the `po-email-processor` Lambda.
|
3. S3 `ObjectCreated` triggers the `po-email-processor` Lambda.
|
||||||
4. Claude extracts structured JSON (PO number, status, supplier, site code, trade classification, line items, fiscal year).
|
4. The processor rejects the email unless the verified sender domain is on the allowlist (see Security) — anyone can email the public address, but only legitimate Coupa/Amazon senders may write data.
|
||||||
5. Conditional write to DynamoDB:
|
5. Claude extracts structured JSON (PO number, status, supplier, site code, trade classification, line items, fiscal year).
|
||||||
- `new_po` — idempotent insert (no-op if PO exists)
|
6. Merge write to DynamoDB:
|
||||||
- `revision` — unconditional overwrite
|
- `new_po` — merge insert. Creates the PO, or backfills data into a pre-existing `Cancelled` skeleton left by an out-of-order cancellation (preserving the `Cancelled` status). No longer silently dropped when a record already exists.
|
||||||
- `cancellation` — marks existing row `Cancelled`
|
- `revision` — field-level merge (`update_item` SETs only the fields present in the revision). A revision that omits `line_items`/`supplier` no longer deletes them. Will not un-cancel a `Cancelled` PO.
|
||||||
|
- `cancellation` — marks the row `Cancelled` (creating a minimal skeleton if the cancellation arrives before the `new_po`).
|
||||||
6. DynamoDB Streams feeds downstream consumers:
|
6. DynamoDB Streams feeds downstream consumers:
|
||||||
- **LedgerFlow** (`seahaven-slack-bot/po-sync`) — daily KB sync
|
- **LedgerFlow** (`seahaven-slack-bot/po-sync`) — daily KB sync
|
||||||
- **Site extractor** (`po-ingest-site-extractor`) — real-time site address extraction into `verified-sites` table
|
- **Site extractor** (`po-ingest-site-extractor`) — real-time site address extraction into `verified-sites` table
|
||||||
|
|
@ -69,8 +70,18 @@ All Lambdas: Python 3.12, ARM64, 60-day log retention.
|
||||||
- `po-ingest/anthropic-api-key` — Anthropic API key for PO parsing
|
- `po-ingest/anthropic-api-key` — Anthropic API key for PO parsing
|
||||||
- `workorder-ingest/anthropic-api-key` — Anthropic API key for WO parsing
|
- `workorder-ingest/anthropic-api-key` — Anthropic API key for WO parsing
|
||||||
|
|
||||||
|
The email processors **require** `ANTHROPIC_API_KEY_SECRET_ARN` to be set and resolve the key from Secrets Manager at runtime. There is no plaintext `ANTHROPIC_API_KEY` env fallback — a deploy missing the ARN fails loudly instead of silently running on an unmanaged key.
|
||||||
|
|
||||||
**SES:** Both stacks add rules to the shared `INBOUND_MAIL` receipt rule set on `int.seahaven.com`.
|
**SES:** Both stacks add rules to the shared `INBOUND_MAIL` receipt rule set on `int.seahaven.com`.
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
**Sender authorization (email processors).** The public receipt addresses (`amazon_po@`, `apm@`) accept mail from anyone, so each processor validates the verified sender domain against an allowlist before any DynamoDB write. The allowlist is configurable per function via the `ALLOWED_SENDER_DOMAINS` env var (comma-separated; a domain matches itself or any subdomain). Defaults: PO = `coupahost.com,amazon.com`; WO = `amazon.com,hxgnsmartcloud.com,hexagon.com`. When SES receipt-rule scanning is enabled, the processors additionally drop any email whose stored object shows an explicit SPF/DKIM/spam/virus `fail`.
|
||||||
|
|
||||||
|
**Web UI auth (defense-in-depth).** The `po-web-ui` / `workorder-web-ui` handlers refuse unauthenticated requests even though their public Function URLs were removed (INFRA-74). Each requires a shared secret in the `X-Auth-Token` header (or `Authorization: Bearer <token>`), compared in constant time against the `WEB_UI_AUTH_TOKEN` env var. The handler **fails closed** if the token is unset (denies all). The CDK wires `WEB_UI_AUTH_TOKEN` from the SSM String parameter `/procurement-ingest/web-ui-auth-token` — create it before deploy. (A plaintext String, not a SecureString, is used because `value_for_string_parameter` only resolves String params into Lambda env vars at deploy; this is a defense-in-depth floor for a detached URL, not primary auth.)
|
||||||
|
|
||||||
|
**Output escaping.** All caller-influenced values (including prompt-injectable strings Claude may return for `total_amount`/line-item amounts) are HTML-escaped before interpolation to prevent stored XSS.
|
||||||
|
|
||||||
**Failure handling (INFRA-41):** Each email-processor is async-invoked (S3 → Lambda). Both have a CDK-managed SQS dead-letter queue (`dead_letter_queue=`, 14-day retention, SSL-enforced) so a failed parse is captured rather than silently dropped after Lambda's retries, plus an ALARM-only CloudWatch `Errors` alarm (Sum, threshold > 0) wired to the shared `site-alerts` SNS topic.
|
**Failure handling (INFRA-41):** Each email-processor is async-invoked (S3 → Lambda). Both have a CDK-managed SQS dead-letter queue (`dead_letter_queue=`, 14-day retention, SSL-enforced) so a failed parse is captured rather than silently dropped after Lambda's retries, plus an ALARM-only CloudWatch `Errors` alarm (Sum, threshold > 0) wired to the shared `site-alerts` SNS topic.
|
||||||
|
|
||||||
## Shared Resources
|
## Shared Resources
|
||||||
|
|
@ -107,13 +118,18 @@ Branch protection on `main` — all changes through PR.
|
||||||
aws secretsmanager create-secret --name po-ingest/anthropic-api-key --secret-string "sk-ant-..."
|
aws secretsmanager create-secret --name po-ingest/anthropic-api-key --secret-string "sk-ant-..."
|
||||||
aws secretsmanager create-secret --name workorder-ingest/anthropic-api-key --secret-string "sk-ant-..."
|
aws secretsmanager create-secret --name workorder-ingest/anthropic-api-key --secret-string "sk-ant-..."
|
||||||
```
|
```
|
||||||
3. Deploy both stacks:
|
3. Store the web UI shared secret (the web-ui Lambdas fail closed without it):
|
||||||
|
```bash
|
||||||
|
aws ssm put-parameter --name /procurement-ingest/web-ui-auth-token --type String --value "$(openssl rand -hex 32)"
|
||||||
|
```
|
||||||
|
4. Deploy both stacks:
|
||||||
```bash
|
```bash
|
||||||
cd cdk
|
cd cdk
|
||||||
pip install -r requirements.txt
|
pip install -r requirements.txt
|
||||||
cdk deploy --all
|
cdk deploy --all
|
||||||
```
|
```
|
||||||
4. CloudFormation outputs include `WebUIUrl` for each stack's dashboard.
|
|
||||||
|
The web UI Lambdas have no public Function URL (INFRA-74); invoke them only through an authenticated path that forwards the `X-Auth-Token` header.
|
||||||
|
|
||||||
## Scripts
|
## Scripts
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue