# PO Template-First Parser — Design, Investigation & Progress > **Status:** in progress · **Branch:** `feat/po-template-parser` (stacked on PR #99) · **Last updated:** 2026-07-16 > > Living document for making the Coupa **purchase-order** email parser *template-first with LLM fallback*, mirroring the work-order (WO) processor's PR #99. Captures the investigation, the data we gathered, the decisions made, the current scaffold, and everything still to do. --- ## 1. Context & Goal The PO email processor (`lambdas/po/email_processor/handler.py`) sends **every** inbound Coupa/Amazon purchase-order email to an LLM (Claude Haiku 4.5 on Bedrock) for structured extraction. WO PR #99 established the pattern we want to replicate here: > Run a **deterministic template parser first**. It returns a result only when the email provably conforms to a known template and passes a **fail-closed validation gate**; otherwise the handler falls back to the Bedrock LLM. This removes the LLM from the hot path for routine traffic while keeping full AI coverage for anything unexpected. **Goal:** apply the same template-first + fail-closed-fallback approach to the PO processor, without regressing data quality or the downstream `po-ingest-site-extractor` pipeline (which reads `site_code`/`ship_to` off the `purchase-orders` DynamoDB stream). **Reference implementation:** `lambdas/wo/email_processor/template_parser.py` (PR #99). ### Why this is harder than WO - The PO contract is ~40 fields with **nested** objects (`supplier{}`, `ship_to{}`, `line_items[]`) vs WO's flat 16. - Much of the PO extraction prompt is **derived classification** (`trade`, `site_code`, `fiscal_year`) already expressed as deterministic English rules. - Money fields require `Decimal` (DynamoDB rejects floats). - `email_type` is **not** in the subject (unlike WO's two subject templates). --- ## 2. Investigation Two phases: a read-only multi-agent (ultracode) feasibility study over a 120-email sample, then a full-bucket triage over **all 3,448** inbound emails to get real distribution numbers. ### 2.1 Data access - PO email bucket: `s3://po-ingest-emails-328440206208/inbound/` (AWS account **328440206208**, us-east-1). - Reached via AWS profile **`amoussa-mgmt`** (SSO). The default CLI creds are the personal account `681986854588`, which is **denied** on the ingest buckets — a recurring gotcha. ### 2.2 Full-bucket triage results (all 3,448 emails, 2026-07-16) Method: parallel ranged-GET (`bytes=0-12000`) of every object, then local classification (quoted-printable bodies, decodable offline). | Email kind | Count | % | Notes | |---|---:|---:|---| | **new_po** — `***Copy for Reference*** New Purchase Order has been issued` | 3,294 | **95.5%** | Single uniform Coupa layout | | **cancellation** — ` Purchase Order # has been cancelled` | 100 | **2.9%** | Distinct subject + minimal body | | **comment** — `New Comment on Purchase Order for Amazon` | 19 | 0.55% | Type NOT in handler enum (see §7) | | **non-Coupa senders** (human replies, WO mail, an AWS SES setup notice) | 35 | 1.0% | Rejected at ses_auth before parse | | **revision** | **0** | 0% | No distinct revision emails exist | Within **new_po**: - **Status** values: `Issued - Created` (2,223) and `Issued - Scheduled for email` (1,071) — only these two. - **Multi-line-item:** 6 / 3,294 (**0.18%**), all ≤3 items. - **Non-USD:** 0 (after excluding unit-token false positives like `EACH`/`HOUR`). - So the single-line / USD scope covers **~99.8%** of new_po traffic. ### 2.3 new_po layout (the one template that matters) `multipart/alternative`; the **text/plain** part is a stable, label-delimited flattening: - Top: `Amazon Purchase Order #`, then `Submitted By`, `On Behalf Of`, `Supplier`, `Total`, `Items`. - `More Detail` block: `PO ID`, `Department`, `Status`, `Last Opened`, `Order Date`, `Acknowledged At`, `Revision Date`, `Payment Term`, `Req #`, `Shipping`. - `Supplier` detail block (address). - `Shipping` detail block (ship-to address, `Location Code:`, `Attn:`). - `Lines` section: one block per line item; per-line metadata is **U+2022 (`•`) delimited**: `Need By` · `Category` · `Account` · `Period` [· optional `Part Number`]. **Structural hazards found in real data:** - `Supplier`, `Shipping`, `Total` labels **each appear twice** (summary placeholder + detail block); the *first* `Shipping` value is literally `None`. - Optional `Part Number` bullet segment appears in a minority of emails, inserted between `Category` and `Account` → shifts any positional splitter. - Every captured value carries a trailing `\r`; a lone `\xa0` (nbsp) line sits before `Total`. - Real amounts include thousands separators (`18,624.05`, `32,405.00`). ### 2.4 cancellation layout Subject: ` Purchase Order # has been cancelled`. Body is a short dashes-delimited notice — **no `Status` label**, no line items. The only field the handler's `save_cancellation()` needs is `po_number`. --- ## 3. Key Decisions 1. **Build two templates**, not one and not three: - `coupa_new_po` (95.5%) — the main workload. - `coupa_cancellation` (2.9%) — trivial and low-risk (just `po_number`); we have 100 real examples. - Together ≈ **98.4%** of all mail deterministically handled/routed. 2. **No revision template** — 0 distinct revision emails in 3,448. The handler's `revision` type essentially never arrives as its own email. 3. **`email_type` never defaults to new_po.** It is emitted only when the exact new_po subject regex matches **and** `po_status` ∈ the two confirmed-safe strings. Anything else → LLM. (Misrouting a cancellation to new_po would silently defeat the sticky-`Cancelled` guard.) 4. **Fail closed, always.** Any miss / invalid / exception returns `None` and falls back to the LLM. A failure is never a parsed result. (Copies WO's `try/except` posture.) 5. **`Decimal` for all money**, with thousands-separator stripping, to match the LLM path's `parse_float=Decimal`. 6. **Derived fields (`site_code`, `trade`, `fiscal_year`) are NOT computed in the parser.** They are left `None`; a shared post-stage (`enrich_parsed()`, the `pad_zip` precedent) fills them **identically** on both the template and LLM paths, so the gate judges *extraction fidelity only* and both paths write byte-identical shapes downstream. `coupa_category` is a **verbatim** label capture, not a classifier, and IS extracted. 7. **Structural nested contract** with recursive key-set validation (unlike WO's flat tuple). 8. **Two-PR split:** (1) extraction template + gate + handler wiring; (2) derived-classifier factoring, shadow-logged before it becomes authoritative. --- ## 4. Contract Top-level `CONTRACT_KEYS` (23) plus three nested sub-tuples. Taken from the `EXTRACTION_PROMPT` in `handler.py`. ``` CONTRACT_KEYS = ( email_type, po_number, po_status, source_system, submitted_by, on_behalf_of, order_date, revision_date, last_opened, acknowledged_at, payment_terms, requisition_number, department, view_order_url, supplier, site_code, ship_to, total_amount, currency, fiscal_year, trade, coupa_category, line_items, ) SUPPLIER_KEYS = (name,) SHIP_TO_KEYS = (name, address, street, city, state, zip, location_code, attn) LINE_ITEM_KEYS = (description, amount, currency, need_by, category, account_code, period, quantity, unit, price) ``` | Class | Keys | |---|---| | **Verbatim / labeled extract** | po_number, po_status, submitted_by, on_behalf_of, order_date, revision_date, last_opened, acknowledged_at, payment_terms, requisition_number, department, view_order_url, supplier.name, ship_to.location_code, ship_to.attn, coupa_category, all `line_items[*]` | | **Extract with care** | total_amount, currency, ship_to.name/street/city/state/zip (`Decimal`; sentinel-anchored address) | | **Constant** | source_system = `"coupa"` | | **Derived — post-stage, NOT parser, NOT gate-validated** | site_code, trade, fiscal_year | Handler enrichment metadata (`raw_s3_key`, `processed_at`, `data_source`, `email_subject`, `ship_to_raw`, top-level `state`) is added by `enrich_parsed()` and is **not** part of the parser contract. A parity test (mirroring WO's `test_contract_keys_match_extraction_prompt`) should assert the flattened contract key set is a subset of the keys quoted in `EXTRACTION_PROMPT`. --- ## 5. Parser Design File: `lambdas/po/email_processor/template_parser.py` — pure module (no boto3/network), mirroring WO idioms. - `classify_template(email_data) -> (template_id, reason)` — exact subject regex → `coupa_new_po` | `coupa_cancellation` | `unknown`. - `extract_new_po` / `extract_cancellation` — build the nested candidate; leave derived fields `None`. - `_empty_candidate()` / `_normalize()` — build & normalize the **nested** skeleton (`supplier={}`, `ship_to={}`, `line_items=[one item]`). - `_clean()` — strip trailing `\r` + `\xa0`, map `"None"`/empty → `None`. - `_to_decimal()` — thousands-separator-safe `Decimal`, `_UNPARSEABLE` sentinel on failure. - `validate(candidate, template_id, email_data) -> (bool, reason)` — fail closed. - `try_deterministic_parse(email_data) -> (parsed|None, method, template_id, reason)` — entry point; `try/except` fails closed on any error. **Handler integration (not yet wired):** after `authenticate_inbound_email()` + `parse_raw_email()`, call `try_deterministic_parse`; if `None`, `extract_with_claude`; then run the shared `enrich_parsed()`+classify stage on `parsed` regardless of path, then the existing `save_cancellation` / `save_revision` / `save_new_po` routing. Emit a `ParseMethod` EMF metric like WO. --- ## 6. Fail-Closed Gate Rules Reason codes currently emitted by `validate()` / `try_deterministic_parse()`: `subject_no_match` · `key_set_mismatch` · `derived_field_set` · `email_type_mismatch` · `missing_required_field` · `po_id_mismatch` · `unrecognized_status` · `multiline_unsupported` · `non_usd` · `new_po_not_implemented` · `extractor_raised` · (`ok` / `template` on success). **Implemented (both templates):** - Known template only (`subject_no_match` otherwise). - Exact nested key-set at every level (`key_set_mismatch`). - Derived fields must be unset by the parser (`derived_field_set`). - `email_type` in enum and == template's expected type (`email_type_mismatch`). - `po_number` valid shape `^[A-Z0-9]{1,6}-\d+$` **and** byte-equals the subject id (`po_id_mismatch` / `missing_required_field`). **Implemented (new_po):** safe-status enum (`unrecognized_status`), single-line only (`multiline_unsupported`), USD only (`non_usd`). **TODO (new_po value-level — see §8):** amount byte-equality + `sum(lines)==total`; duplicate-label anchoring (reject `Shipping=='None'`, require `Location Code:`, `supplier.name` contains `SEA HAVEN`, `ship_to.name != supplier.name`); bullet-metadata split by leading label; address shape (`state` null-or-USPS, `zip` null-or `^\d{5}(-\d{4})?$`); `_UNPARSEABLE` rejection; residual `\r`/`\xa0` rejection. **Precondition (handled a layer earlier):** `authenticate_inbound_email()` (SES `dkim=pass` for `amazon.coupahost.com`) must pass before the parser runs — the 35 non-Coupa emails are rejected there. --- ## 7. Scope — In / Out **IN (template-parsed):** - Single-line-item, USD, `new_po` with the exact subject and a safe `Status`. - `cancellation` (subject match → `po_number`). **OUT (always LLM fallback / rejected):** - **Comments** (19) — `New Comment on Purchase Order` is an `email_type` the handler enum (`new_po`/`revision`/`cancellation`) does not model. The LLM currently shoehorns these. ⚠️ **Pre-existing data-quality gap, independent of this work** — decide separately whether comments should create/update PO records at all. - **Revisions** — no template (0 emails). - **Multi-line-item new_po** (0.18%) — `Lines`-array structure unobserved; `multiline_unsupported`. - **Non-USD new_po** (0 observed) — path unexercised; `non_usd`. - **Non-Coupa senders** (1.0%) — rejected at ses_auth. --- ## 8. Progress - [x] Read-only ultracode feasibility investigation (8 agents, GO-with-conditions). - [x] Full-bucket triage over all 3,448 emails → distribution + scope numbers (§2.2). - [x] Confirmed cancellation & comment body layouts against real emails. - [x] **Scaffold** `template_parser.py` on `feat/po-template-parser`: - [x] Nested contract + recursive `_empty_candidate`/`_normalize`. - [x] `classify_template` for both templates (real subject regexes). - [x] `coupa_cancellation` extract + gate — **implemented & smoke-tested** (parses a real cancellation → `po_number`, `email_type=cancellation`). - [x] `coupa_new_po` classify + structural/status/single-line/USD gate. - [x] Scaffold guard: `extract_new_po` is a documented TODO → new_po returns `new_po_not_implemented` and falls back to the LLM (safe). - [x] `ruff check` + `ruff format --check` pass. - Smoke test (real corpus): cancellation → `template`; new_po → `ai_fallback/new_po_not_implemented`; comment & non-Coupa → `ai_fallback/subject_no_match`. > Not yet committed; working-tree only. --- ## 9. TODO / Path Forward **PR #1 — extraction template + wiring** - [ ] Implement `extract_new_po`: labeled single-occurrence fields; duplicate-label anchoring (`Supplier`/`Shipping`/`Total`); `ship_to` by sentinel anchors; `line_items` split on `•` by leading label; `Decimal` amounts; `coupa_category` verbatim. - [ ] Implement the new_po value-level gate rules (§6 TODO) and remove the scaffold guard. - [ ] Handler integration: `try_deterministic_parse` first, Bedrock fallback on `None`; route through existing `save_*` branches. - [ ] Emit `ParseMethod`-only EMF set (namespace `Seahaven/PoIngest`, dims `[["ParseMethod"], ["ParseMethod","TemplateId"]]`). - [ ] Fallback-rate CloudWatch alarm in `cdk/po_stack.py` — **retune thresholds for ~57/day volume** (PO is far lower than WO's ~760/day; the WO 15-min/sample-floor settings won't fit). - [ ] Offline tests + fixtures: good cancellation + good new_po; one adversarial per gate rule (thousands-sep, dup-label swap, Part-Number bullet shift, multiline, non-USD, bad status); comment + non-Coupa assertions; contract⊆prompt parity test. - [ ] README + Confluence "AWS Architecture Map" updates. - [ ] `cross_reviewer` (GPT-4.1) pass — handler contract change (Sea Haven policy). - [ ] `ruff check` + `ruff format` + `pytest` green in CI. **PR #2 — derived-classifier factoring (follow-up)** - [ ] Move `trade`/`site_code`/`fiscal_year` into shared `enrich_parsed()`. - [ ] Run in **shadow mode**: compute Python values, log Python-vs-LLM disagreement via EMF, keep LLM authoritative during a bake period. - [ ] Harden `site_code` (7-shape + skip-list, incl. multi-hop ATTN like `CBRE - RME - DLI6`) and `trade` against the full sample with per-shape fixtures. - [ ] Only after acceptable agreement: make Python authoritative and drop the fields from `EXTRACTION_PROMPT`. --- ## 10. Open Questions 1. **Goal — cost, latency, or determinism?** Determinism (same input → same output) is the strongest justification for the derived-classifier port regardless of per-email cost. 2. **Classifier factoring now or deferred?** Recommendation: extraction template first (PR #1), factoring as shadow-logged PR #2. 3. **Strict single-line/USD-only v1 acceptable?** Triage says it covers ~99.8% of new_po, so yes — but confirm. 4. **Confirm the real cancellation `Status` spelling** — `handler.py` hardcodes `CANCELLED_STATUS='Cancelled'`; the cancellation body doesn't restate it. Verify against a known-cancelled PO in the table. 5. **Comments** — should `New Comment on Purchase Order` emails create/update PO records at all, or be dropped? (Out of scope for the parser, but a real data-quality decision.) 6. **Repo fixtures** — commit a sanitized PO corpus (like WO's checked-in `.eml` fixtures), or keep it out of the repo? --- ## 11. Risk Register (high-severity) | Risk | Mitigation | |---|---| | **Cancellation misrouted to new_po** → sticky-`Cancelled` guard defeated, PO stays active | `email_type` closed-set gate; never default-to-new_po; only emit on exact subject + safe status | | **Thousands-separator truncation** (`18,624.05`→`624.05`, ~30× too small, passes naive checks) | Amount byte-equality: re-serialize `Decimal` to source grouped format; reject if token bordered by digit/comma; `sum(lines)==total` | | **Multi-line structure unobserved** | Hard-fail `len(line_items)!=1` → LLM; profile real multi-line before supporting | | **Derived-field silent divergence** (wrong `trade`/`site_code` still well-formed; `site_code` rules already incomplete) | Factor into shared post-stage; keep OFF the gate; shadow-log before authoritative; per-shape fixtures | | **Duplicate-label first-match** (`Supplier`/`Shipping`/`Total` twice; first `Shipping`=`None`) | Occurrence/sentinel anchoring; gate rejects `Shipping=='None'`, requires `Location Code:`, `supplier.name` has `SEA HAVEN`, `ship_to.name != supplier.name` | --- ## Appendix — References - WO reference parser: `lambdas/wo/email_processor/template_parser.py` (PR #99). - PO handler + `EXTRACTION_PROMPT`: `lambdas/po/email_processor/handler.py`. - PO stack (alarms, IAM, Bedrock): `cdk/po_stack.py`. - Bedrock model: inference profile `us.anthropic.claude-haiku-4-5-20251001-v1:0`. - PO email bucket: `s3://po-ingest-emails-328440206208/inbound/` (account 328440206208, profile `amoussa-mgmt`).