procurement-ingest/docs/po-template-parser.md

PO Template-First Parser — Design, Investigation & Progress

Status: in progress · Branch: feat/po-template-parser (stacked on PR #99) · Last updated: 2026-07-16

Living document for making the Coupa purchase-order email parser template-first with LLM fallback, mirroring the work-order (WO) processor's PR #99. Captures the investigation, the data we gathered, the decisions made, the current scaffold, and everything still to do.

---

1. Context & Goal

The PO email processor (lambdas/po/email_processor/handler.py) sends every inbound Coupa/Amazon purchase-order email to an LLM (Claude Haiku 4.5 on Bedrock) for structured extraction. WO PR #99 established the pattern we want to replicate here:

Run a deterministic template parser first. It returns a result only when the email provably conforms to a known template and passes a fail-closed validation gate; otherwise the handler falls back to the Bedrock LLM. This removes the LLM from the hot path for routine traffic while keeping full AI coverage for anything unexpected.

Goal: apply the same template-first + fail-closed-fallback approach to the PO processor, without regressing data quality or the downstream po-ingest-site-extractor pipeline (which reads site_code/ship_to off the purchase-orders DynamoDB stream).

Reference implementation: lambdas/wo/email_processor/template_parser.py (PR #99).

Why this is harder than WO

• The PO contract is ~40 fields with nested objects (supplier{}, ship_to{}, line_items[]) vs WO's flat 16.
• Much of the PO extraction prompt is derived classification (trade, site_code, fiscal_year) already expressed as deterministic English rules.
• Money fields require Decimal (DynamoDB rejects floats).
• email_type is not in the subject (unlike WO's two subject templates).

---

2. Investigation

Two phases: a read-only multi-agent (ultracode) feasibility study over a 120-email sample, then a full-bucket triage over all 3,448 inbound emails to get real distribution numbers.

2.1 Data access

• PO email bucket: s3://po-ingest-emails-328440206208/inbound/ (AWS account 328440206208, us-east-1).
• Reached via AWS profile amoussa-mgmt (SSO). The default CLI creds are the personal account 681986854588, which is denied on the ingest buckets — a recurring gotcha.

2.2 Full-bucket triage results (all 3,448 emails, 2026-07-16)

Method: parallel ranged-GET (bytes=0-12000) of every object, then local classification (quoted-printable bodies, decodable offline).

Email kind	Count	%	Notes
new_po — ***Copy for Reference*** New Purchase Order <PO#> has been issued	3,294	95.5%	Single uniform Coupa layout
cancellation — <Site> Purchase Order #<PO#> has been cancelled	100	2.9%	Distinct subject + minimal body
comment — New Comment on Purchase Order for Amazon	19	0.55%	Type NOT in handler enum (see §7)
non-Coupa senders (human replies, WO mail, an AWS SES setup notice)	35	1.0%	Rejected at ses_auth before parse
revision	0	0%	No distinct revision emails exist

Within new_po:

• Status values: Issued - Created (2,223) and Issued - Scheduled for email (1,071) — only these two.
• Multi-line-item: 6 / 3,294 (0.18%), all ≤3 items.
• Non-USD: 0 (after excluding unit-token false positives like EACH/HOUR).
• So the single-line / USD scope covers ~99.8% of new_po traffic.

2.3 new_po layout (the one template that matters)

multipart/alternative; the text/plain part is a stable, label-delimited flattening:

• Top: Amazon Purchase Order #<PO#>, then Submitted By, On Behalf Of, Supplier, Total, Items.
• More Detail block: PO ID, Department, Status, Last Opened, Order Date, Acknowledged At, Revision Date, Payment Term, Req #, Shipping.
• Supplier detail block (address).
• Shipping detail block (ship-to address, Location Code:, Attn:).
• Lines section: one block per line item; per-line metadata is U+2022 (•) delimited: Need By · Category · Account · Period [· optional Part Number].

Structural hazards found in real data:

• Supplier, Shipping, Total labels each appear twice (summary placeholder + detail block); the first Shipping value is literally None.
• Optional Part Number bullet segment appears in a minority of emails, inserted between Category and Account → shifts any positional splitter.
• Every captured value carries a trailing \r; a lone \xa0 (nbsp) line sits before Total.
• Real amounts include thousands separators (18,624.05, 32,405.00).

2.4 cancellation layout

Subject: <SiteName> Purchase Order #<PO#> has been cancelled. Body is a short dashes-delimited notice — no Status label, no line items. The only field the handler's save_cancellation() needs is po_number.

---

3. Key Decisions

1. Build two templates, not one and not three:
   • coupa_new_po (95.5%) — the main workload.
   • coupa_cancellation (2.9%) — trivial and low-risk (just po_number); we have 100 real examples.
   • Together ≈ 98.4% of all mail deterministically handled/routed.
2. No revision template — 0 distinct revision emails in 3,448. The handler's revision type essentially never arrives as its own email.
3. email_type never defaults to new_po. It is emitted only when the exact new_po subject regex matches and po_status ∈ the two confirmed-safe strings. Anything else → LLM. (Misrouting a cancellation to new_po would silently defeat the sticky-Cancelled guard.)
4. Fail closed, always. Any miss / invalid / exception returns None and falls back to the LLM. A failure is never a parsed result. (Copies WO's try/except posture.)
5. Decimal for all money, with thousands-separator stripping, to match the LLM path's parse_float=Decimal.
6. Derived fields (site_code, trade, fiscal_year) are NOT computed in the parser. They are left None; a shared post-stage (enrich_parsed(), the pad_zip precedent) fills them identically on both the template and LLM paths, so the gate judges extraction fidelity only and both paths write byte-identical shapes downstream. coupa_category is a verbatim label capture, not a classifier, and IS extracted.
7. Structural nested contract with recursive key-set validation (unlike WO's flat tuple).
8. Two-PR split: (1) extraction template + gate + handler wiring; (2) derived-classifier factoring, shadow-logged before it becomes authoritative.

---

4. Contract

Top-level CONTRACT_KEYS (23) plus three nested sub-tuples. Taken from the EXTRACTION_PROMPT in handler.py.

CONTRACT_KEYS = (
  email_type, po_number, po_status, source_system, submitted_by, on_behalf_of,
  order_date, revision_date, last_opened, acknowledged_at, payment_terms,
  requisition_number, department, view_order_url, supplier, site_code, ship_to,
  total_amount, currency, fiscal_year, trade, coupa_category, line_items,
)
SUPPLIER_KEYS  = (name,)
SHIP_TO_KEYS   = (name, address, street, city, state, zip, location_code, attn)
LINE_ITEM_KEYS = (description, amount, currency, need_by, category,
                  account_code, period, quantity, unit, price)

Class	Keys
Verbatim / labeled extract	po_number, po_status, submitted_by, on_behalf_of, order_date, revision_date, last_opened, acknowledged_at, payment_terms, requisition_number, department, view_order_url, supplier.name, ship_to.location_code, ship_to.attn, coupa_category, all line_items[*]
Extract with care	total_amount, currency, ship_to.name/street/city/state/zip (Decimal; sentinel-anchored address)
Constant	source_system = "coupa"
Derived — post-stage, NOT parser, NOT gate-validated	site_code, trade, fiscal_year

Handler enrichment metadata (raw_s3_key, processed_at, data_source, email_subject, ship_to_raw, top-level state) is added by enrich_parsed() and is not part of the parser contract.

A parity test (mirroring WO's test_contract_keys_match_extraction_prompt) should assert the flattened contract key set is a subset of the keys quoted in EXTRACTION_PROMPT.

---

5. Parser Design

File: lambdas/po/email_processor/template_parser.py — pure module (no boto3/network), mirroring WO idioms.

• classify_template(email_data) -> (template_id, reason) — exact subject regex → coupa_new_po | coupa_cancellation | unknown.
• extract_new_po / extract_cancellation — build the nested candidate; leave derived fields None.
• _empty_candidate() / _normalize() — build & normalize the nested skeleton (supplier={}, ship_to={}, line_items=[one item]).
• _clean() — strip trailing \r + \xa0, map "None"/empty → None.
• _to_decimal() — thousands-separator-safe Decimal, _UNPARSEABLE sentinel on failure.
• validate(candidate, template_id, email_data) -> (bool, reason) — fail closed.
• try_deterministic_parse(email_data) -> (parsed|None, method, template_id, reason) — entry point; try/except fails closed on any error.

Handler integration (not yet wired): after authenticate_inbound_email() + parse_raw_email(), call try_deterministic_parse; if None, extract_with_claude; then run the shared enrich_parsed()+classify stage on parsed regardless of path, then the existing save_cancellation / save_revision / save_new_po routing. Emit a ParseMethod EMF metric like WO.

---

6. Fail-Closed Gate Rules

Reason codes currently emitted by validate() / try_deterministic_parse():

subject_no_match · key_set_mismatch · derived_field_set · email_type_mismatch · missing_required_field · po_id_mismatch · unrecognized_status · multiline_unsupported · non_usd · new_po_not_implemented · extractor_raised · (ok / template on success).

Implemented (both templates):

• Known template only (subject_no_match otherwise).
• Exact nested key-set at every level (key_set_mismatch).
• Derived fields must be unset by the parser (derived_field_set).
• email_type in enum and == template's expected type (email_type_mismatch).
• po_number valid shape ^[A-Z0-9]{1,6}-\d+$ and byte-equals the subject id (po_id_mismatch / missing_required_field).

Implemented (new_po): safe-status enum (unrecognized_status), single-line only (multiline_unsupported), USD only (non_usd).

TODO (new_po value-level — see §8): amount byte-equality + sum(lines)==total; duplicate-label anchoring (reject Shipping=='None', require Location Code:, supplier.name contains SEA HAVEN, ship_to.name != supplier.name); bullet-metadata split by leading label; address shape (state null-or-USPS, zip null-or ^\d{5}(-\d{4})?$); _UNPARSEABLE rejection; residual \r/\xa0 rejection.

Precondition (handled a layer earlier): authenticate_inbound_email() (SES dkim=pass for amazon.coupahost.com) must pass before the parser runs — the 35 non-Coupa emails are rejected there.

---

7. Scope — In / Out

IN (template-parsed):

• Single-line-item, USD, new_po with the exact subject and a safe Status.
• cancellation (subject match → po_number).

OUT (always LLM fallback / rejected):

• Comments (19) — New Comment on Purchase Order is an email_type the handler enum (new_po/revision/cancellation) does not model. The LLM currently shoehorns these. ⚠️ Pre-existing data-quality gap, independent of this work — decide separately whether comments should create/update PO records at all.
• Revisions — no template (0 emails).
• Multi-line-item new_po (0.18%) — Lines-array structure unobserved; multiline_unsupported.
• Non-USD new_po (0 observed) — path unexercised; non_usd.
• Non-Coupa senders (1.0%) — rejected at ses_auth.

---

8. Progress

• Read-only ultracode feasibility investigation (8 agents, GO-with-conditions).
• Full-bucket triage over all 3,448 emails → distribution + scope numbers (§2.2).
• Confirmed cancellation & comment body layouts against real emails.
• Scaffold template_parser.py on feat/po-template-parser:
  • Nested contract + recursive _empty_candidate/_normalize.
  • classify_template for both templates (real subject regexes).
  • coupa_cancellation extract + gate — implemented & smoke-tested (parses a real cancellation → po_number, email_type=cancellation).
  • coupa_new_po classify + structural/status/single-line/USD gate.
  • Scaffold guard: extract_new_po is a documented TODO → new_po returns new_po_not_implemented and falls back to the LLM (safe).
  • ruff check + ruff format --check pass.
• Smoke test (real corpus): cancellation → template; new_po → ai_fallback/new_po_not_implemented; comment & non-Coupa → ai_fallback/subject_no_match.

Not yet committed; working-tree only.

---

9. TODO / Path Forward

PR #1 — extraction template + wiring

• Implement extract_new_po: labeled single-occurrence fields; duplicate-label anchoring (Supplier/Shipping/Total); ship_to by sentinel anchors; line_items split on • by leading label; Decimal amounts; coupa_category verbatim.
• Implement the new_po value-level gate rules (§6 TODO) and remove the scaffold guard.
• Handler integration: try_deterministic_parse first, Bedrock fallback on None; route through existing save_* branches.
• Emit ParseMethod-only EMF set (namespace Seahaven/PoIngest, dims [["ParseMethod"], ["ParseMethod","TemplateId"]]).
• Fallback-rate CloudWatch alarm in cdk/po_stack.py — retune thresholds for ~57/day volume (PO is far lower than WO's ~760/day; the WO 15-min/sample-floor settings won't fit).
• Offline tests + fixtures: good cancellation + good new_po; one adversarial per gate rule (thousands-sep, dup-label swap, Part-Number bullet shift, multiline, non-USD, bad status); comment + non-Coupa assertions; contract⊆prompt parity test.
• README + Confluence "AWS Architecture Map" updates.
• cross_reviewer (GPT-4.1) pass — handler contract change (Sea Haven policy).
• ruff check + ruff format + pytest green in CI.

PR #2 — derived-classifier factoring (follow-up)

• Move trade/site_code/fiscal_year into shared enrich_parsed().
• Run in shadow mode: compute Python values, log Python-vs-LLM disagreement via EMF, keep LLM authoritative during a bake period.
• Harden site_code (7-shape + skip-list, incl. multi-hop ATTN like CBRE - RME - DLI6) and trade against the full sample with per-shape fixtures.
• Only after acceptable agreement: make Python authoritative and drop the fields from EXTRACTION_PROMPT.

---

10. Open Questions

1. Goal — cost, latency, or determinism? Determinism (same input → same output) is the strongest justification for the derived-classifier port regardless of per-email cost.
2. Classifier factoring now or deferred? Recommendation: extraction template first (PR #1), factoring as shadow-logged PR #2.
3. Strict single-line/USD-only v1 acceptable? Triage says it covers ~99.8% of new_po, so yes — but confirm.
4. Confirm the real cancellation Status spelling — handler.py hardcodes CANCELLED_STATUS='Cancelled'; the cancellation body doesn't restate it. Verify against a known-cancelled PO in the table.
5. Comments — should New Comment on Purchase Order emails create/update PO records at all, or be dropped? (Out of scope for the parser, but a real data-quality decision.)
6. Repo fixtures — commit a sanitized PO corpus (like WO's checked-in .eml fixtures), or keep it out of the repo?

---

11. Risk Register (high-severity)

Risk	Mitigation
Cancellation misrouted to new_po → sticky-Cancelled guard defeated, PO stays active	email_type closed-set gate; never default-to-new_po; only emit on exact subject + safe status
Thousands-separator truncation (18,624.05→624.05, ~30× too small, passes naive checks)	Amount byte-equality: re-serialize Decimal to source grouped format; reject if token bordered by digit/comma; sum(lines)==total
Multi-line structure unobserved	Hard-fail len(line_items)!=1 → LLM; profile real multi-line before supporting
Derived-field silent divergence (wrong trade/site_code still well-formed; site_code rules already incomplete)	Factor into shared post-stage; keep OFF the gate; shadow-log before authoritative; per-shape fixtures
Duplicate-label first-match (Supplier/Shipping/Total twice; first Shipping=None)	Occurrence/sentinel anchoring; gate rejects Shipping=='None', requires Location Code:, supplier.name has SEA HAVEN, ship_to.name != supplier.name

---

Appendix — References

• WO reference parser: lambdas/wo/email_processor/template_parser.py (PR #99).
• PO handler + EXTRACTION_PROMPT: lambdas/po/email_processor/handler.py.
• PO stack (alarms, IAM, Bedrock): cdk/po_stack.py.
• Bedrock model: inference profile us.anthropic.claude-haiku-4-5-20251001-v1:0.
• PO email bucket: s3://po-ingest-emails-328440206208/inbound/ (account 328440206208, profile amoussa-mgmt).