adam/procurement-ingest
1
0
Fork 0
mirror of https://github.com/Sea-Haven-Industries/procurement-ingest.git synced 2026-09-30 04:53:12 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
Coupa PO email ingestion pipeline
3 commits 6 branches 0 tags 6.7 MiB
JavaScript 56.7%
Python 34.7%
HCL 6.7%
Shell 1.6%
HTML 0.3%
Find a file
Adam Moussa 3514e74e40 Fix stack naming to follow kebab-case convention 
Pin existing CloudFormation stack name via stack_name property so
the live stack is not affected. Construct ID now follows the org
kebab-case standard.
2026-04-28 14:43:48 -04:00
cdk Fix stack naming to follow kebab-case convention 2026-04-28 14:43:48 -04:00
lambdas Initial commit: PO email ingestion pipeline 2026-04-07 12:12:30 -04:00
scripts Initial commit: PO email ingestion pipeline 2026-04-07 12:12:30 -04:00
.gitignore Initial commit: PO email ingestion pipeline 2026-04-07 12:12:30 -04:00
README.md Add README 2026-04-20 19:31:22 -04:00

README.md

PO Ingest

Coupa purchase-order email ingestion pipeline. SES receives Amazon PO emails, Claude extracts structured data, and the result lands in the shared purchase-orders DynamoDB table.

Flow

  1. Coupa sends a PO email to amazon_po@int.seahaven.com.
  2. SES (using the shared INBOUND_MAIL rule set) drops the raw MIME into s3://po-ingest-emails-{AccountId}/inbound/.
  3. S3 ObjectCreated fires the po-email-processor Lambda.
  4. The Lambda parses the email, sends it to Claude Haiku 4.5 for structured JSON extraction, and writes to DynamoDB.
    • email_type: new_po — conditional PutItem on purchase-orders (idempotent on po_number).
    • email_type: cancellation — UpdateItem marking the existing row Cancelled.
  5. LedgerFlow consumes the table via DynamoDB Streams → po-sync. This repo only writes.

A separate po-web-ui Lambda (Function URL, unauthenticated) renders a simple HTML dashboard scanning the table.

Architecture

  • IaC: AWS CDK (Python), stack name PoIngestStack, region us-east-1.
  • Lambdas: po-email-processor (S3-triggered) and po-web-ui (Function URL). Python 3.12, 256 MB, 60s timeout.
  • Storage: S3 po-ingest-emails-{AccountId} with 90-day lifecycle expiry; DynamoDB purchase-orders (shared, not owned by this stack).
  • Secrets: Anthropic API key in Secrets Manager at po-ingest/anthropic-api-key.
  • SES: adds the PoEmailRule to the existing INBOUND_MAIL receipt rule set (shared with workorder-ingest).

Setup

  1. Bootstrap CDK in the account if you haven't already: cdk bootstrap aws://{AccountId}/us-east-1.
  2. Store the Anthropic API key: 
    aws secretsmanager create-secret \
  --name po-ingest/anthropic-api-key \
  --secret-string "sk-ant-..."
  3. Install Lambda dependencies into the deployable package directory (gitignored): 
    pip install -r lambdas/email_processor/requirements.txt -t lambdas/email_processor/package/
  4. Deploy: 
    cd cdk
pip install -r requirements.txt
cdk deploy
  5. The WebUIUrl CloudFormation output is the dashboard URL.

Reprocessing

To re-run the processor against every email still sitting in inbound/ (useful after a parser change):

python scripts/reprocess.py            # dry-run — lists keys
python scripts/reprocess.py --execute  # invokes po-email-processor for each

Inserts are conditional on po_number, so re-processing existing POs is a no-op.