> Living document for making the Coupa **purchase-order** email parser *template-first with LLM fallback*, mirroring the work-order (WO) processor's PR #99. Captures the investigation, the data we gathered, the decisions made, the current scaffold, and everything still to do.
---
## 1. Context & Goal
The PO email processor (`lambdas/po/email_processor/handler.py`) sends **every** inbound Coupa/Amazon purchase-order email to an LLM (Claude Haiku 4.5 on Bedrock) for structured extraction. WO PR #99 established the pattern we want to replicate here:
> Run a **deterministic template parser first**. It returns a result only when the email provably conforms to a known template and passes a **fail-closed validation gate**; otherwise the handler falls back to the Bedrock LLM. This removes the LLM from the hot path for routine traffic while keeping full AI coverage for anything unexpected.
**Goal:** apply the same template-first + fail-closed-fallback approach to the PO processor, without regressing data quality or the downstream `po-ingest-site-extractor` pipeline (which reads `site_code`/`ship_to` off the `purchase-orders` DynamoDB stream).
- The PO contract is ~40 fields with **nested** objects (`supplier{}`, `ship_to{}`, `line_items[]`) vs WO's flat 16.
- Much of the PO extraction prompt is **derived classification** (`trade`, `site_code`, `fiscal_year`) already expressed as deterministic English rules.
-`email_type` is **not** in the subject (unlike WO's two subject templates).
---
## 2. Investigation
Two phases: a read-only multi-agent (ultracode) feasibility study over a 120-email sample, then a full-bucket triage over **all 3,448** inbound emails to get real distribution numbers.
### 2.1 Data access
- PO email bucket: `s3://po-ingest-emails-328440206208/inbound/` (AWS account **328440206208**, us-east-1).
- ~~Reached via AWS profile `amoussa-mgmt` (SSO); default CLI creds are the personal account `681986854588`~~ **Stale (corrected 2026-07-16 during the fixture harvest):** the default CLI session is now authenticated to **328440206208** directly — no `--profile` flag needed.
- ⚠️ **Corpus is aging out:**`inbound/` objects carry an S3 lifecycle expiration (~90-day rolling window; oldest object 2026-04-17 at harvest time, 3,422 objects vs 3,448 at triage). Any further harvesting should not be deferred long.
- Every per-line bullet run BEGINS with a `Supplier <name>` segment before `Need By` (the gate proves it byte-equals `supplier.name` on every item).
- Multi-item Lines blocks carry a `<qty> EA` evidence line before each description; the single-item Items summary carries an optional `<qty> <UNIT> x <price>` line (absent on some emails, e.g. new-po-14) which is the **source** for `quantity`/`unit`/`price` (the Lines-block `EA` line is a gate cross-check only).
- The Items summary contains **unit-price money tokens distinct from line amounts** (e.g. `50.0 EACH x 66.00` with line amount `8,356.00`) — money must be anchored on the `for <amt> <CCY>` / Total-block / summary-`x` contexts, never free money-shaped scanning; and line `amount ≠ qty×price` on partial quantities, so only `sum(lines) == total` is load-bearing.
- Body line endings are **decode-path dependent**: Coupa QP encodes `=0D`, so `policy.default``get_content()` yields `\r\n` even after transport normalization, while other decode paths yield bare LF — the parser matches only `_clean`-ed lines and is tested against both representations.
Subject: `<SiteName> Purchase Order #<PO#> has been cancelled`. Body is a short dashes-delimited notice — **no `Status` label**, no line items. The only field the handler's `save_cancellation()` needs is `po_number`.
---
## 3. Key Decisions
1.**Build two templates**, not one and not three:
-`coupa_new_po` (95.5%) — the main workload.
-`coupa_cancellation` (2.9%) — trivial and low-risk (just `po_number`); we have 100 real examples.
- Together ≈ **98.4%** of all mail deterministically handled/routed.
2.**No revision template** — 0 distinct revision emails in 3,448. The handler's `revision` type essentially never arrives as its own email.
3.**`email_type` never defaults to new_po.** It is emitted only when the exact new_po subject regex matches **and**`po_status` ∈ the two confirmed-safe strings. Anything else → LLM. (Misrouting a cancellation to new_po would silently defeat the sticky-`Cancelled` guard.)
4.**Fail closed, always.** Any miss / invalid / exception returns `None` and falls back to the LLM. A failure is never a parsed result. (Copies WO's `try/except` posture.)
5.**`Decimal` for all money**, with thousands-separator stripping, to match the LLM path's `parse_float=Decimal`.
6.**Derived fields (`site_code`, `trade`, `fiscal_year`) are NOT computed in the parser.** They are left `None`; a shared post-stage (`enrich_parsed()`, the `pad_zip` precedent) fills them **identically** on both the template and LLM paths, so the gate judges *extraction fidelity only* and both paths write byte-identical shapes downstream. `coupa_category` is a **verbatim** label capture, not a classifier, and IS extracted.
| **Derived — post-stage, NOT parser, NOT gate-validated** | site_code, trade, fiscal_year |
Handler enrichment metadata (`raw_s3_key`, `processed_at`, `data_source`, `email_subject`, `ship_to_raw`, top-level `state`) is added by `enrich_parsed()` and is **not** part of the parser contract.
A parity test (mirroring WO's `test_contract_keys_match_extraction_prompt`) should assert the flattened contract key set is a subset of the keys quoted in `EXTRACTION_PROMPT`.
---
## 5. Parser Design
File: `lambdas/po/email_processor/template_parser.py` — pure module (no boto3/network), mirroring WO idioms.
**Handler integration (✅ wired in PR #1):** after `authenticate_inbound_email()` + `parse_raw_email()`, `try_deterministic_parse` runs first; if `None`, `extract_with_claude`; the shared `enrich_parsed()` stage runs on `parsed` regardless of path, then the existing `save_cancellation` / `save_revision` / `save_new_po` routing (untouched). The `ParseMethod` EMF metric is emitted immediately after the deterministic attempt — before any Bedrock call — so a Bedrock-side error still records the `ai_fallback` outcome.
**Implemented (new_po):** safe-status enum (`unrecognized_status`), single-line only (`multiline_unsupported`), USD only (`non_usd`) — enforced at **both** levels: the Total-block top-level currency must be exactly `USD` (rule 8) **and** every line item's captured currency plus its re-derived `for <amt> <CCY>` body token must byte-equal it (a single non-USD line item fails closed even when the Total block reads USD).
**Implemented (new_po value-level rules V1–V13 — ✅ done, one reason code each):** the gate **re-derives every byte proof from `email_data["body"]`** (never trusting extractor-carried state), with the six value-level reason codes:
-`amount_mismatch` — money fidelity (each Decimal re-serializes byte-identically to its re-derived source token, non-digit/comma border — the `18,624.05`→`624.05` kill switch), `sum(lines) == total` (exact Decimal; deliberately **no**`qty×price == amount` rule — corpus shows partial quantities), dual-`Total` byte-identity, and quantity/unit/price coherence vs the summary and `EA` evidence lines.
-`anchor_violation` — exactly 2×`Supplier`/`Shipping`/`Total`, first `Shipping` is the literal `None` placeholder, section ordering, `supplier.name` contains `SEA HAVEN` and byte-equals the detail-block name + every item's `Supplier` bullet segment, `ship_to.name != supplier.name`.
-`address_shape_invalid` — city line fullmatches `^City, ST ZIP$` immediately before `United States`, state in the frozen USPS set, zip `^\d{5}(-\d{4})?$` on the RAW pre-enrichment value (a short zip fails closed to the LLM path where the shared `pad_zip` repairs it — the gate never predicts enrichment).
-`bullet_label_unrecognized` — every U+2022 segment carries a recognized leading label from the closed set; core labels exactly once, `Part Number` at most once.
-`unparseable_value` — recursive walk: no `_UNPARSEABLE` sentinel leaf.
-`residual_artifact` — recursive walk: no `\r`/`\xa0` in any string leaf.
Plus `missing_required_field` for the required labeled fields (per-item metadata, `Location Code:` sourcing, no invented `Attn:`) and `po_id_mismatch` for the body `PO ID` / heading / `orders/<id>`-URL identity proofs.
**Precondition (handled a layer earlier):** `authenticate_inbound_email()` (SES `dkim=pass` for `amazon.coupahost.com`) must pass before the parser runs — the 35 non-Coupa emails are rejected there.
---
## 7. Scope — In / Out
**IN (template-parsed):**
- Single-line-item, USD, `new_po` with the exact subject and a safe `Status`.
-`cancellation` (subject match → `po_number`).
**OUT (always LLM fallback / rejected):**
- **Comments** (19) — `New Comment on Purchase Order` is an `email_type` the handler enum (`new_po`/`revision`/`cancellation`) does not model. The LLM currently shoehorns these. ⚠️ **Pre-existing data-quality gap, independent of this work** — decide separately whether comments should create/update PO records at all.
- [x]`extract_new_po` — section-windowed, anchor-disciplined, label-keyed bullet split, `Decimal` money, representation-agnostic line handling. All 17 harvested single-line new-POs parse to `template/ok`; the 3 real multi-line ones extract faithfully (`sum==total`) then gate-reject with `multiline_unsupported`.
- [x] Value-level gate rules V1–V13 (§6) — every byte proof re-derived from the body.
- [x] Handler wiring: `try_deterministic_parse` first, Bedrock fallback on `None`, shared `enrich_parsed()` on both paths, `save_*` routing untouched.
- [x]`ParseMethod` EMF metric set (`Seahaven/PoIngest`/`ParseOutcome`, dims `[["ParseMethod"],["ParseMethod","TemplateId"]]`, `ReasonCode`/`po_number` ride-alongs), emitted before the Bedrock call.
- [x] Fallback-rate alarm in `cdk/po_stack.py`, retuned for ~57/day (6h periods · ≥8 volume floor · >20% · 2-of-4) — `npx cdk synth po-ingest` green.
- [x]`cdk/po_stack.py` bundling `cp` now includes `template_parser.py` (was a deploy-time ImportError waiting to happen).
- [x] Offline test suite `lambdas/po/email_processor/tests/` (132 tests): goldens for all 25 positive fixtures, every §6 reason code covered, dual line-ending identity, two-path enrich/save parity, fixture hygiene (ses_auth + leak-sweep markers), Bedrock dispatch/EMF. Full-root pytest: 354 green.
- [x] Committed sanitized `.eml` fixture corpus (header + body scrub, per-file digit cipher, narrow `.gitignore` exception scoped to `lambdas/po/email_processor/tests/fixtures/`).
- [x]`cross_review.py` (GPT-4.1) pass — run 2026-07-16 against the real handler diff: **no BLOCK, no security findings, "safe to merge with minor fixes."** Both FIX items verified as no-change-needed: the fallback log line already reports `ai_fallback` correctly (a Claude-side exception propagates before it, unchanged posture), and the non-dict-from-Claude hazard is the pre-existing issue #101 pattern this PR deliberately leaves untouched.
- [ ] Move `trade`/`site_code`/`fiscal_year` into shared `enrich_parsed()`.
- [ ] Run in **shadow mode**: compute Python values, log Python-vs-LLM disagreement via EMF, keep LLM authoritative during a bake period.
- [ ] Harden `site_code` (7-shape + skip-list, incl. multi-hop ATTN like `CBRE - RME - DLI6`) and `trade` against the full sample with per-shape fixtures.
- [ ] Only after acceptable agreement: make Python authoritative and drop the fields from `EXTRACTION_PROMPT`.
---
## 10. Open Questions
1.**Goal — cost, latency, or determinism?** Determinism (same input → same output) is the strongest justification for the derived-classifier port regardless of per-email cost.
2.**Classifier factoring now or deferred?** Recommendation: extraction template first (PR #1), factoring as shadow-logged PR #2.
3.**Strict single-line/USD-only v1 acceptable?** ✅ **Resolved (PR #1): yes.** Covers ~99.8% of new_po; multi-line and non-USD fail closed with honest reason codes.
4.**Confirm the real cancellation `Status` spelling** — `handler.py` hardcodes `CANCELLED_STATUS='Cancelled'`; the cancellation body doesn't restate it. Verify against a known-cancelled PO in the table. (Doesn't affect parser output — `save_cancellation` hardcodes the string.)
5.**Comments** — should `New Comment on Purchase Order` emails create/update PO records at all, or be dropped? (Out of scope for the parser, but a real data-quality decision.)
**Fixture-build pins recorded during PR #1 (locked by goldens):**
- **`ship_to.street`/`ship_to.address` join convention:** multi-line segments joined with `'\n'`; `address` = the lines from `ship_to.name` through `United States` inclusive (this is what `enrich_parsed` promotes to `ship_to_raw` on the site-extractor stream).
- **`quantity`/`unit`/`price` source:** the Items-summary `<qty> <UNIT> x <price>` line (e.g. `1.0 EACH x 55,206.00`); the Lines-block `<qty> EA` line is gate evidence only (V13 numeric cross-check). Emails without a summary line (e.g. new-po-14) leave all three `null` — nullable by contract.
- **V10 URL-id sweep outcome:** all 20 harvested new_po emails satisfy `orders/<id> == po_number` digit suffix — the strict V10 check stays live (no demotion needed); a corpus-sweep test locks it.
| **Cancellation misrouted to new_po** → sticky-`Cancelled` guard defeated, PO stays active | `email_type` closed-set gate; never default-to-new_po; only emit on exact subject + safe status |
| **Thousands-separator truncation** (`18,624.05`→`624.05`, ~30× too small, passes naive checks) | Amount byte-equality: re-serialize `Decimal` to source grouped format; reject if token bordered by digit/comma; `sum(lines)==total` |
| **Multi-line structure unobserved** | Hard-fail `len(line_items)!=1` → LLM; profile real multi-line before supporting |
| **Derived-field silent divergence** (wrong `trade`/`site_code` still well-formed; `site_code` rules already incomplete) | Factor into shared post-stage; keep OFF the gate; shadow-log before authoritative; per-shape fixtures |
