procurement-ingest/tests/test_po_merge.py
Adam Moussa ca4f43a2cc
Some checks are pending
Deploy / deploy (push) Waiting to run
feat: decompose email-processor handlers into flat siblings + lazy boto3 clients (refactor phase 5) (#113)
Both email-processor God-handlers split along the seams that already
work in the flat-sibling pattern established by lambdas/shared/, so
bare-name imports keep working under the existing bundling glob.

PO (5-way split): handler.py keeps only the event loop, fail-closed
auth, and email_type routing. extraction.py holds extract_with_claude
and _EMAIL_TAG_RE, importing EXTRACTION_PROMPT from prompts.py and
parse_raw_email from shared/email_parsing.py rather than recreating a
PO-local copy. enrichment.py is a pure code move of enrich_parsed and
pad_zip (PO-only; WO has no enrichment stage) with zero behavior
change. telemetry.py holds the EMF ParseMethod emit wrappers.
persistence.py holds _write_fields/_merge_update/save_*, collapsing
the byte-identical save_new_po/save_revision bodies into one
_save_merge helper that both now call through, preserving the sticky
Cancelled ConditionExpression guard for both callers; save_cancellation
stays separate.

WO (5 concerns, no enrichment stage): the handler loop keeps
validate_ai_fallback and the re.fullmatch(r"[0-9]+", work_order_id)
key guard ahead of both save_work_order and save_event, since the
guard protects the DynamoDB partition key and the '#'-delimited
comment_id range-key segment. _header_date_iso and comment_id
determinism stay colocated with persistence.py's save_event for the
retry-idempotent event_id key.

EXTRACTION_PROMPT (PO) moves to prompts.py with cross-reference
headers to derived_fields.py's authoritative trade/site/fiscal rule
tables; handler.py re-exports it (from prompts import
EXTRACTION_PROMPT) since four tests dereference handler.EXTRACTION_
PROMPT directly. WO's prompt moves the same way.

I/O modules (extraction.py's bedrock client, persistence.py's
dynamodb resource, handler.py's s3 client) get lazy cached boto3
accessors; pure modules (enrichment.py, prompts.py, telemetry.py)
import no boto3. Test monkeypatch surfaces move to the module that
now owns the client (e.g. persistence.dynamodb) everywhere tests
patch it, and the moto-before-handler-import ordering in
_po_parser_support.py is preserved so the moto-backed suites don't
hit real AWS.

Behavior-preservation pins, verified with tests: PO still emits
ParseMethod=ai_fallback before the Bedrock call, with
ai_fallback_rejected as the additive second datapoint on rejection.
WO still emits after its gate with mutually-exclusive ai_fallback /
ai_fallback_rejected. Shadow DerivedFieldAgreement telemetry stays
ai_fallback-only. derived_fields.py is untouched (diff against
feature/phase-3-shared-extraction is empty). handler(event, context)
signatures and the save_* public contract are unchanged on both
pipelines; goldens unchanged.

PO_EXPECTED_TOP_LEVEL_MODULES and its WO equivalent in
tests/test_bundle_consistency.py are updated for the new sibling
modules so the AST bundle-consistency test still fails on an
unshipped or uncommented-out sibling.
2026-07-20 15:34:53 -04:00

166 lines
5.6 KiB
Python

"""Tests for the sticky-Cancelled merge semantics in the PO email processor.
These exercise the atomic conditional write in ``_merge_update`` that makes
"Cancelled" a sticky, authoritative status: once a PO is Cancelled, a later
new_po or revision may enrich its other fields but can never move it back to a
non-cancelled status. The invariant is enforced server-side with a DynamoDB
ConditionExpression, so it holds even under concurrent email processing — there
is no read-then-write TOCTOU window.
A real in-memory purchase-orders table is stood up with moto so the tests
exercise the actual DynamoDB conditional-write and ConditionalCheckFailedException
fallback path, not a stub.
"""
import boto3
import pytest
from moto import mock_aws
@pytest.fixture
def po_table(po_persistence):
"""Stand up a mock purchase-orders table matching the real key schema.
Phase 5 moved PO_TABLE onto persistence.py; key the mock table on
``po_persistence.PO_TABLE`` so the fixture and the save_* code under test
resolve the same name from one source rather than a duplicated literal.
"""
with mock_aws():
resource = boto3.resource("dynamodb", region_name="us-east-1")
table = resource.create_table(
TableName=po_persistence.PO_TABLE,
KeySchema=[{"AttributeName": "po_number", "KeyType": "HASH"}],
AttributeDefinitions=[
{"AttributeName": "po_number", "AttributeType": "S"},
],
BillingMode="PAY_PER_REQUEST",
)
table.wait_until_exists()
yield table
def _item(table, po_number):
return table.get_item(Key={"po_number": po_number}).get("Item")
def test_revision_cannot_uncancel_but_enriches(po_handler, po_table):
"""(a) Revision carrying a live status onto a Cancelled PO keeps it Cancelled
while still writing the other revision fields (no resurrection)."""
po_table.put_item(
Item={
"po_number": "PO-A",
"po_status": "Cancelled",
"cancelled_at": "2026-07-01T00:00:00",
}
)
po_handler.save_revision(
{
"po_number": "PO-A",
"po_status": "Issued",
"total_amount": 500,
"supplier": {"name": "Acme"},
}
)
item = _item(po_table, "PO-A")
assert item["po_status"] == "Cancelled" # not resurrected
assert item["cancelled_at"] == "2026-07-01T00:00:00" # marker preserved
assert item["total_amount"] == 500 # other fields still enriched
assert item["supplier"] == {"name": "Acme"}
def test_revision_without_status_merges_on_cancelled(po_handler, po_table):
"""(b) Revision with NO po_status onto a Cancelled PO merges cleanly, no
exception, status stays Cancelled."""
po_table.put_item(Item={"po_number": "PO-B", "po_status": "Cancelled"})
po_handler.save_revision({"po_number": "PO-B", "total_amount": 250})
item = _item(po_table, "PO-B")
assert item["po_status"] == "Cancelled"
assert item["total_amount"] == 250
def test_revision_updates_status_on_non_cancelled(po_handler, po_table):
"""(c) Issue B: a legitimate status update on a NON-cancelled PO is applied,
never dropped."""
po_table.put_item(Item={"po_number": "PO-C", "po_status": "Issued"})
po_handler.save_revision(
{"po_number": "PO-C", "po_status": "Revised", "total_amount": 999}
)
item = _item(po_table, "PO-C")
assert item["po_status"] == "Revised"
assert item["total_amount"] == 999
def test_new_po_backfills_cancelled_skeleton(po_handler, po_table):
"""(d) new_po onto a Cancelled skeleton backfills its fields but keeps the
Cancelled status and cancellation marker."""
po_table.put_item(
Item={
"po_number": "PO-D",
"po_status": "Cancelled",
"cancelled_at": "2026-07-02T00:00:00",
}
)
po_handler.save_new_po(
{
"po_number": "PO-D",
"po_status": "Issued",
"supplier": {"name": "Beta"},
"total_amount": 1200,
}
)
item = _item(po_table, "PO-D")
assert item["po_status"] == "Cancelled"
assert item["cancelled_at"] == "2026-07-02T00:00:00"
assert item["supplier"] == {"name": "Beta"}
assert item["total_amount"] == 1200
def test_new_po_creates_fresh(po_handler, po_table):
"""(e) new_po on a fresh PO creates it normally with its status."""
po_handler.save_new_po(
{"po_number": "PO-E", "po_status": "Issued", "total_amount": 42}
)
item = _item(po_table, "PO-E")
assert item["po_status"] == "Issued"
assert item["total_amount"] == 42
def test_revision_creates_fresh_merge(po_handler, po_table):
"""(e) revision on a fresh PO merges/creates normally with its status."""
po_handler.save_revision(
{"po_number": "PO-F", "po_status": "Revised", "total_amount": 77}
)
item = _item(po_table, "PO-F")
assert item["po_status"] == "Revised"
assert item["total_amount"] == 77
def test_save_cancellation_is_authoritative(po_handler, po_table):
"""Cancellation always wins: save_cancellation writes Cancelled over a live
status while leaving previously-established data intact."""
po_table.put_item(
Item={"po_number": "PO-G", "po_status": "Issued", "total_amount": 300}
)
po_handler.save_cancellation(
{
"po_number": "PO-G",
"processed_at": "2026-07-03T00:00:00",
"raw_s3_key": "s3://bucket/key",
}
)
item = _item(po_table, "PO-G")
assert item["po_status"] == "Cancelled"
assert item["cancelled_at"] == "2026-07-03T00:00:00"
assert item["total_amount"] == 300 # pre-existing data retained