mirror of
https://github.com/Sea-Haven-Industries/procurement-ingest.git
synced 2026-10-01 14:43:13 +00:00
Add README
This commit is contained in:
parent
fc690dd958
commit
fef240695c
1 changed files with 55 additions and 0 deletions
55
README.md
Normal file
55
README.md
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
# PO Ingest
|
||||
|
||||
Coupa purchase-order email ingestion pipeline. SES receives Amazon PO emails, Claude extracts structured data, and the result lands in the shared `purchase-orders` DynamoDB table.
|
||||
|
||||
## Flow
|
||||
|
||||
1. Coupa sends a PO email to `amazon_po@int.seahaven.com`.
|
||||
2. SES (using the shared `INBOUND_MAIL` rule set) drops the raw MIME into `s3://po-ingest-emails-{AccountId}/inbound/`.
|
||||
3. S3 `ObjectCreated` fires the `po-email-processor` Lambda.
|
||||
4. The Lambda parses the email, sends it to Claude Haiku 4.5 for structured JSON extraction, and writes to DynamoDB.
|
||||
- `email_type: new_po` — conditional `PutItem` on `purchase-orders` (idempotent on `po_number`).
|
||||
- `email_type: cancellation` — `UpdateItem` marking the existing row `Cancelled`.
|
||||
5. LedgerFlow consumes the table via DynamoDB Streams → `po-sync`. This repo only writes.
|
||||
|
||||
A separate `po-web-ui` Lambda (Function URL, unauthenticated) renders a simple HTML dashboard scanning the table.
|
||||
|
||||
## Architecture
|
||||
|
||||
- **IaC:** AWS CDK (Python), stack name `PoIngestStack`, region `us-east-1`.
|
||||
- **Lambdas:** `po-email-processor` (S3-triggered) and `po-web-ui` (Function URL). Python 3.12, 256 MB, 60s timeout.
|
||||
- **Storage:** S3 `po-ingest-emails-{AccountId}` with 90-day lifecycle expiry; DynamoDB `purchase-orders` (shared, not owned by this stack).
|
||||
- **Secrets:** Anthropic API key in Secrets Manager at `po-ingest/anthropic-api-key`.
|
||||
- **SES:** adds the `PoEmailRule` to the existing `INBOUND_MAIL` receipt rule set (shared with `workorder-ingest`).
|
||||
|
||||
## Setup
|
||||
|
||||
1. Bootstrap CDK in the account if you haven't already: `cdk bootstrap aws://{AccountId}/us-east-1`.
|
||||
2. Store the Anthropic API key:
|
||||
```bash
|
||||
aws secretsmanager create-secret \
|
||||
--name po-ingest/anthropic-api-key \
|
||||
--secret-string "sk-ant-..."
|
||||
```
|
||||
3. Install Lambda dependencies into the deployable package directory (gitignored):
|
||||
```bash
|
||||
pip install -r lambdas/email_processor/requirements.txt -t lambdas/email_processor/package/
|
||||
```
|
||||
4. Deploy:
|
||||
```bash
|
||||
cd cdk
|
||||
pip install -r requirements.txt
|
||||
cdk deploy
|
||||
```
|
||||
5. The `WebUIUrl` CloudFormation output is the dashboard URL.
|
||||
|
||||
## Reprocessing
|
||||
|
||||
To re-run the processor against every email still sitting in `inbound/` (useful after a parser change):
|
||||
|
||||
```bash
|
||||
python scripts/reprocess.py # dry-run — lists keys
|
||||
python scripts/reprocess.py --execute # invokes po-email-processor for each
|
||||
```
|
||||
|
||||
Inserts are conditional on `po_number`, so re-processing existing POs is a no-op.
|
||||
Loading…
Add table
Reference in a new issue