Add pytest suite and enable tests in CI (#95)

The repo had no automated tests, so regressions in the email
parsing pipeline could only be caught manually. The reusable
ci-python-sam workflow already supports a run-tests input at the
pinned SHA; enable it and add a first suite covering the pure
functions pad_zip and parse_raw_email.

tests/conftest.py sets AWS_DEFAULT_REGION and dummy credentials
before any handler import because the handlers create boto3
clients at module import time, and loads each handler.py via
importlib under a unique module name since the files share a
basename. pytest.ini scopes discovery to tests/ so the manual
test_local.py script at the repo root is not collected.

.github/workflows/ci.yaml

@@ -7,6 +7,7 @@ jobs:
   ci:
     uses: Sea-Haven-Industries/.github/.github/workflows/ci-python-sam.yaml@fd60e4c9041784f666ac0fdefb9bec3c7fbf5143 # main
     with:
-      source-dirs: "lambdas cdk"
+      source-dirs: "lambdas cdk tests"
+      run-tests: true
       run-cdk-synth: true
       run-sam-validate: false

README.md

@@ -34,7 +34,7 @@ Coupa PO emails are received at `amazon_po@int.seahaven.com`, parsed by Claude H
 
 **Tables:**
 - `purchase-orders` (PK: `po_number`, Streams: NEW_IMAGE) — shared with seahaven-slack-bot (read-only; see Shared Resources)
-- `verified-sites` (PK: `siteCode`, GSI: `by-state`) — ~1,100 unique Amazon facility sites
+- `verified-sites` (PK: `siteCode`) — ~1,100 unique Amazon facility sites (`by-state` GSI removed 2026-06-03, audit M-20)
 - `pending-site-review` (PK: `po_number`) — unresolvable POs for manual Payee Central verification
 
 ### Work Orders (`WorkorderIngestStack` stack)
@@ -56,7 +56,7 @@ Amazon APM work order emails (from Hexagon EAM / HxGN SmartCloud) are received a
 | `workorder-web-ui` | Manual invoke | HTML dashboard (public Function URL removed 2026-06-08, INFRA-74) |
 
 **Tables:**
-- `WorkOrders` (PK: `work_order_id`, GSIs: `site-code-index`, `status-index`)
+- `WorkOrders` (PK: `work_order_id`) — `site-code-index` and `status-index` GSIs removed 2026-06-03 (audit M-20)
 - `WorkOrderComments` (PK: `work_order_id`, SK: `comment_id`)
 
 ## Architecture
@@ -87,13 +87,15 @@ Every alarm is **ALARM-only** (no OK action), sends to the shared `site-alerts`
 
 The `<fn>-duration` and `<fn>-throttles` alarms for `po-email-processor` and `workorder-email-processor` supersede the orphaned, CLI-created `Lambda-Duration-*` / `Lambda-Throttles-*` alarms (deleted post-deploy).
 
+**DLQ alarms** (`AWS/SQS`): `po-email-processor-dlq-messages` and `workorder-email-processor-dlq-messages` fire when any message is visible on an email-processor DLQ (`ApproximateNumberOfMessagesVisible` Maximum, 5 min, `> 0`, eval 1) — a message there means an email was dropped after Lambda exhausted its async retries.
+
 **DynamoDB alarms** (`AWS/DynamoDB`): each owned table gets `<table>-throttles` (`ThrottledRequests`) and `<table>-system-errors` (`SystemErrors`). These metrics emit only at the `TableName` + `Operation` dimension set, so each alarm is a `Sum` math expression across the operations the table uses (Get/BatchGet/Query/Scan/Put/Update/Delete/BatchWrite). Tables covered: `purchase-orders`, `verified-sites`, `pending-site-review` (po-ingest); `WorkOrders`, `WorkOrderComments` (workorder-ingest).
 
 ## Shared Resources
 
 ### `purchase-orders` table (owned here)
 
-The `purchase-orders` DynamoDB table is **owned by this repo's `po-ingest` stack** (defined in `cdk/po_stack.py` with `RemovalPolicy.RETAIN` and `StreamViewType.NEW_IMAGE`). The `po-email-processor` Lambda is the authoritative writer — it performs the conditional inserts, revision overwrites, and cancellation updates described above.
+The `purchase-orders` DynamoDB table is **owned by this repo's `po-ingest` stack** (defined in `cdk/po_stack.py` with `RemovalPolicy.RETAIN`, `StreamViewType.NEW_IMAGE`, and SSE-KMS encryption with the shared customer-managed CMK `alias/seahaven-dynamodb`, INFRA-95 / M-3). The `po-email-processor` Lambda is the authoritative writer — it performs the conditional inserts, revision overwrites, and cancellation updates described above.
 
 **Consumers (read-only):**
 
@@ -109,16 +111,18 @@ The consumer imports the table via `Table.fromTableName(...)` and is granted rea
 
 ### `WorkOrders` and `WorkOrderComments` tables (owned here)
 
-Both tables are **owned by this repo's `WorkorderIngestStack`** (`cdk/wo_stack.py`, `RemovalPolicy.RETAIN`, shared customer-managed CMK per INFRA-95 / M-3):
+Both tables are **owned by this repo's `WorkorderIngestStack`** (`cdk/wo_stack.py`, `RemovalPolicy.RETAIN`):
 
 - `WorkOrders` — PK `work_order_id` (S).
 - `WorkOrderComments` — PK `work_order_id` (S), SK `comment_id` (S).
 
-**Consumer (read-only) — data contract:** `seahaven-slack-bot` imports both tables via `Table.fromTableName(...)` (`grantReadData` plus an explicit `kms:Decrypt` grant on the shared CMK) and reads them from two Lambdas: `workorder-sync` (daily full-table scan into the Bedrock knowledge base) and `wo-po-lookup` (the Bedrock agent's direct WO lookup action group). The bot depends on the PK/SK schema above, the CMK encryption, and these attributes: on `WorkOrders` — `description`, `wo_status`, `customer`, `site_code`, `building`, `address`, `severity`, `priority`, `assigned_to`, `date_reported`, `scheduled_start`, `due_date`, `updated_at`; on `WorkOrderComments` — `created_at` (used to sort comments), `commenter`, `text`. Any change to table name, key schema, these attribute names, or the encryption key must be coordinated with `seahaven-slack-bot` before it ships, or the Bedrock agent breaks at runtime (not at deploy — the tables are imported by name, so there is no compile-time link).
+Both currently use default DynamoDB encryption — they are **not** yet on the shared customer-managed CMK (`alias/seahaven-dynamodb`, INFRA-95 / M-3); that migration is tracked in INFRA-6. The `workorder-email-processor` role already holds a pre-emptive encrypt/decrypt grant on the CMK so the migration won't break it.
+
+**Consumer (read-only) — data contract:** `seahaven-slack-bot` imports both tables via `Table.fromTableName(...)` (`grantReadData`) and reads them from two Lambdas: `workorder-sync` (daily full-table scan into the Bedrock knowledge base) and `wo-po-lookup` (the Bedrock agent's direct WO lookup action group). The bot depends on the PK/SK schema above and these attributes: on `WorkOrders` — `description`, `wo_status`, `customer`, `site_code`, `building`, `address`, `severity`, `priority`, `assigned_to`, `date_reported`, `scheduled_start`, `due_date`, `updated_at`; on `WorkOrderComments` — `created_at` (used to sort comments), `commenter`, `text`. Any change to table name, key schema, these attribute names, or the encryption configuration (e.g. the INFRA-6 CMK migration) must be coordinated with `seahaven-slack-bot` before it ships, or the Bedrock agent breaks at runtime (not at deploy — the tables are imported by name, so there is no compile-time link).
 
 ### `verified-sites` table (owned here)
 
-Owned by this repo's `po-ingest` stack (`cdk/po_stack.py`). PK `siteCode` (S); AWS-managed encryption (NOT the shared CMK).
+Owned by this repo's `po-ingest` stack (`cdk/po_stack.py`). PK `siteCode` (S); default DynamoDB encryption (NOT the shared CMK).
 
 **Consumer (read-only) — data contract:** `seahaven-slack-bot`'s `wo-po-lookup` Lambda imports this table via `Table.fromTableName(...)` for the Bedrock agent's `lookup_site` action. It does point lookups by `siteCode` and reads `address`, `fullAddress`, `city`, `state`, `zip`, `latitude`, `longitude`, `notes`. Coordinate any change to the table name, key schema, or these attribute names with `seahaven-slack-bot`.
 
@@ -126,33 +130,34 @@ Owned by this repo's `po-ingest` stack (`cdk/po_stack.py`). PK `siteCode` (S); A
 
 ## Documentation
 
-The canonical map of Sea Haven's AWS infrastructure lives in Confluence. This project's `po-ingest` and `workorder-ingest` stacks are represented there as Mermaid subgraphs.
+The canonical map of Sea Haven's AWS infrastructure lives in Confluence. This project's `po-ingest` and `WorkorderIngestStack` stacks are represented there as Mermaid subgraphs.
 
 - **[AWS Architecture Map](https://seahaven.atlassian.net/wiki/spaces/IT/pages/1540098)** (Confluence, IT space, page 1540098)
 
 ## CI/CD
 
-GitHub Actions with reusable workflows from `Sea-Haven-Industries/.github`:
-- **CI** (PR to `main`): linting + `cdk synth` via `ci-python-sam.yaml@main`
-- **CD** (push to `main`): `cdk deploy --all` via `cd-cdk.yaml@main` (OIDC auth)
+GitHub Actions with reusable workflows from `Sea-Haven-Industries/.github` (all pinned to a commit SHA of `main`):
+- **CI** (`ci.yaml`, PR to `main`): linting + `cdk synth` via `ci-python-sam.yaml`
+- **CD** (`deploy.yaml`, push to `main`): CDK deploy via `cd-cdk.yaml` (OIDC auth)
+- Plus dependency review and PR labeler workflows on every PR
 
 Branch protection on `main` — all changes through PR.
 
 ## Setup
 
 1. Bootstrap CDK: `cdk bootstrap aws://{AccountId}/us-east-1`
-2. Store Anthropic API keys:
-   ```bash
-   aws secretsmanager create-secret --name po-ingest/anthropic-api-key --secret-string "sk-ant-..."
-   aws secretsmanager create-secret --name workorder-ingest/anthropic-api-key --secret-string "sk-ant-..."
-   ```
-3. Deploy both stacks:
+2. Deploy both stacks (this also creates the two Secrets Manager secrets — they are CDK-managed, so don't `create-secret` them by hand):
    ```bash
    cd cdk
    pip install -r requirements.txt
    cdk deploy --all
    ```
-4. CloudFormation outputs include `WebUIUrl` for each stack's dashboard.
+3. Set the Anthropic API key values:
+   ```bash
+   aws secretsmanager put-secret-value --secret-id po-ingest/anthropic-api-key --secret-string "sk-ant-..."
+   aws secretsmanager put-secret-value --secret-id workorder-ingest/anthropic-api-key --secret-string "sk-ant-..."
+   ```
+4. Dashboards: `po-web-ui` and `workorder-web-ui` have no public endpoint (the Function URLs were removed 2026-06-08, INFRA-74). Invoke them manually and render the returned HTML, e.g. `aws lambda invoke --function-name po-web-ui /tmp/out.json`.
 
 ## Scripts
 
@@ -171,7 +176,7 @@ python scripts/backfill_sites.py
 
 ```
 cdk/
-  app.py              # Two stacks: po-ingest + WorkorderIngestStack
+  app.py               # Two stacks: po-ingest + WorkorderIngestStack
   po_stack.py          # Purchase order pipeline resources
   wo_stack.py          # Work order pipeline resources
 lambdas/
@@ -182,9 +187,8 @@ lambdas/
   wo/                  # WO pipeline Lambdas
     email_processor/
     web_ui/
-shared/
-  models.py            # Work order dataclasses/enums
 scripts/
   reprocess.py
   backfill_sites.py
+test_local.py          # Parse sample emails through Claude locally (no AWS)
 ```

pytest.ini

@@ -0,0 +1,5 @@
+[pytest]
+; Restrict discovery to tests/. test_local.py at the repo root matches the
+; default test_*.py glob but is a manual script that imports a handler (and
+; therefore creates boto3 clients) at collection time.
+testpaths = tests

tests/conftest.py

@@ -0,0 +1,60 @@
+"""Shared pytest configuration for the procurement-ingest test suite.
+
+The Lambda handlers create boto3 clients at module import time, so a
+region and dummy credentials must be present in the environment before
+any handler module is imported. Setting them here at conftest import
+time guarantees they exist before test collection touches a handler.
+"""
+
+import importlib.util
+import os
+import sys
+from pathlib import Path
+
+import pytest
+
+os.environ.setdefault("AWS_DEFAULT_REGION", "us-east-1")
+os.environ.setdefault("AWS_ACCESS_KEY_ID", "testing")
+os.environ.setdefault("AWS_SECRET_ACCESS_KEY", "testing")
+os.environ.setdefault("AWS_SESSION_TOKEN", "testing")
+
+REPO_ROOT = Path(__file__).resolve().parents[1]
+
+
+def load_handler(relative_path, module_name):
+    """Load a Lambda handler module by file path under a unique name.
+
+    The handler files all share the basename ``handler.py`` and are not
+    importable as packages, so a plain ``import handler`` would collide
+    across Lambdas.
+    """
+    path = REPO_ROOT / relative_path
+    spec = importlib.util.spec_from_file_location(module_name, path)
+    module = importlib.util.module_from_spec(spec)
+    sys.modules[module_name] = module
+    spec.loader.exec_module(module)
+    return module
+
+
+@pytest.fixture(scope="session")
+def po_handler():
+    """The PO email processor handler module."""
+    return load_handler(
+        "lambdas/po/email_processor/handler.py",
+        "po_email_processor_handler",
+    )
+
+
+@pytest.fixture(scope="session")
+def wo_handler():
+    """The WO email processor handler module."""
+    return load_handler(
+        "lambdas/wo/email_processor/handler.py",
+        "wo_email_processor_handler",
+    )
+
+
+@pytest.fixture(params=["po_handler", "wo_handler"])
+def email_handler(request):
+    """Parametrized fixture yielding each email processor handler module."""
+    return request.getfixturevalue(request.param)

tests/test_pad_zip.py

@@ -0,0 +1,29 @@
+"""Tests for pad_zip in the PO email processor."""
+
+import pytest
+
+
+@pytest.mark.parametrize(
+    ("raw", "expected"),
+    [
+        # Missing values pass through untouched
+        (None, None),
+        ("", ""),
+        # Leading zeros restored (Coupa drops them: "MA 2149" -> "02149")
+        ("2149", "02149"),
+        ("7001", "07001"),
+        ("501", "00501"),
+        # Already-valid 5-digit zips are unchanged
+        ("85704", "85704"),
+        ("02149", "02149"),
+        # Zip+4: leading zeros restored on the base, suffix preserved
+        ("2149-1234", "02149-1234"),
+        ("85704-1234", "85704-1234"),
+        # Surrounding whitespace is handled
+        (" 2149 ", "02149"),
+        # Non-numeric values are left alone
+        ("ABC12", "ABC12"),
+    ],
+)
+def test_pad_zip(po_handler, raw, expected):
+    assert po_handler.pad_zip(raw) == expected

tests/test_parse_raw_email.py

@@ -0,0 +1,91 @@
+"""Tests for parse_raw_email in the PO and WO email processors.
+
+Synthetic MIME messages are built with email.message.EmailMessage so the
+tests exercise real multipart parsing without shipping .eml fixtures.
+"""
+
+from email.message import EmailMessage
+
+SUBJECT = "Amazon.com Services LLC Purchase Order 2D-18206023"
+SENDER = "do_not_reply@coupahost.com"
+TO = "orders@seahavenind.com"
+CC = "ap@seahavenind.com"
+DATE = "Tue, 14 Jul 2026 09:30:00 -0400"
+
+PLAIN_BODY = "PO 2D-18206023 issued to Sea Haven Industries."
+HTML_BODY = "<html><body><p>PO 2D-18206023 issued.</p></body></html>"
+
+
+def _base_message():
+    msg = EmailMessage()
+    msg["Subject"] = SUBJECT
+    msg["From"] = SENDER
+    msg["To"] = TO
+    msg["Cc"] = CC
+    msg["Date"] = DATE
+    return msg
+
+
+def make_plain_email():
+    msg = _base_message()
+    msg.set_content(PLAIN_BODY)
+    return msg.as_bytes()
+
+
+def make_multipart_alternative_email():
+    msg = _base_message()
+    msg.set_content(PLAIN_BODY)
+    msg.add_alternative(HTML_BODY, subtype="html")
+    return msg.as_bytes()
+
+
+def make_multipart_html_only_email():
+    msg = _base_message()
+    msg.set_content(HTML_BODY, subtype="html")
+    # An attachment forces the message to multipart/mixed
+    msg.add_attachment(
+        b"%PDF-1.4 fake",
+        maintype="application",
+        subtype="pdf",
+        filename="po.pdf",
+    )
+    return msg.as_bytes()
+
+
+def test_headers_extracted(email_handler):
+    result = email_handler.parse_raw_email(make_plain_email())
+    assert result["subject"] == SUBJECT
+    assert result["sender"] == SENDER
+    assert result["to"] == TO
+    assert result["date"] == DATE
+
+
+def test_simple_plain_body(email_handler):
+    result = email_handler.parse_raw_email(make_plain_email())
+    assert result["body"].strip() == PLAIN_BODY
+
+
+def test_multipart_prefers_text_plain(email_handler):
+    result = email_handler.parse_raw_email(make_multipart_alternative_email())
+    assert result["body"].strip() == PLAIN_BODY
+    assert "<html>" not in result["body"]
+
+
+def test_multipart_falls_back_to_html(email_handler):
+    result = email_handler.parse_raw_email(make_multipart_html_only_email())
+    assert "PO 2D-18206023 issued." in result["body"]
+    assert "<html>" in result["body"]
+
+
+def test_wo_handler_extracts_cc(wo_handler):
+    result = wo_handler.parse_raw_email(make_plain_email())
+    assert result["cc"] == CC
+
+
+def test_missing_headers_default_to_empty(email_handler):
+    msg = EmailMessage()
+    msg.set_content(PLAIN_BODY)
+    result = email_handler.parse_raw_email(msg.as_bytes())
+    assert result["subject"] == ""
+    assert result["sender"] == ""
+    assert result["to"] == ""