procurement-ingest/tests/test_handler_auth_seam.py

206 lines
6.9 KiB
Python
Raw Normal View History

test: consolidate test roots — one loader, shared support, enforced CI floor (phase 8) (#118) * test: consolidate test roots — one repo-root loader, shared support package, missing-scenario suites, enforced ruff/coverage floor (refactor phase 8) tests/conftest.py only loads for the tests/ root, not a standalone `pytest lambdas/po/email_processor/tests` run, so it could never carry session invariants like the dummy AWS env or the moto stubber registration. Add a single repo-root conftest.py (pytest.ini pins rootdir there, so it loads for every invocation) that sets the dummy AWS credentials/region, imports moto BEFORE any handler module so boto3 sessions pick up its stubber hook (carrying the explanatory comment verbatim from the old _po_parser_support.py), and exposes one load_lambda_module(pipeline, name) — the sys.modules save/restore dance stays, since template_parser is still a duplicated bare name across pipelines needing per-exec sibling binding. Add tests/support/ as the shared package both pipelines' local _*_parser_support.py modules delegate to: a superset FakeTable (PO's update_item recording + WO's put_item and keyed single-row store), FakeDynamoResource, load_email, and load_golden with parse_float=Decimal kept (load-bearing for exact money comparison at PO magnitudes — WO's prior load_golden had no parse_float and must not regress PO by losing it). Rewrite _wo_parser_support.py off the bare `import handler` / `from handler import parse_raw_email` strategy that was the source of the bare-name sys.modules collision the other two loaders defend against. Move test_po_merge.py and test_pad_zip.py into lambdas/po/email_processor/tests/ (PO-specific, belongs beside the code) via git mv so history follows; test_parse_raw_email.py and test_ses_auth.py stay at the repo root since they're genuinely cross-pipeline, parameterized over both handlers. Delete tests/test_local.py: it globs a nonexistent samples/ dir, is WO-only, and imports a handler at collection time, bypassing the loader gate entirely — the golden suites already cover its role. Its pytest.ini exclusion comment goes with it. New scenario coverage, all built on the single loader + support package: - PO+WO Bedrock transport errors (ThrottlingException, missing 'content' key, empty content list, non-JSON model text), asserting PO's pre-call ai_fallback metric survives with no partial write and the exception propagates; WO's no-datapoint-on-throttle behavior is pinned with a documenting test rather than "fixed" by reordering. - Handler-level SES-auth reject seam per pipeline: no auth monkeypatch + empty ALLOWED_DKIM_DOMAINS asserts zero Bedrock calls, zero writes, no raise — closing the hole where deleting the gate line today still passes every test. - web_ui coverage for both PO and WO (0% before this): fail-closed on unset ARN and on a Secrets Manager exception, TTL cache refresh, Bearer/X-Auth-Token/header-case-insensitivity, wrong-token 401 with no table scan, non-ASCII token, and a hostile-field-escaping regression lock. PO web_ui has no __init__.py, so these go through the loader rather than package imports. - A moto-backed mirror of test_po_merge for WO merge semantics (table 'WorkOrders'): null-status never clobbers wo_status, created_at immutable via if_not_exists, status->wo_status mapping, None fields absent from SET, record_type only-when-present. - Small pins: the PO-DC-02 64-char EMF clamp regression and per-pipeline multi-record failure-isolation (all-or-retry contract). The reprocess.py synthetic-event-shape contract test already landed in Phase 7, so it isn't duplicated here. Two WO product-code fixes ride along, since this is the phase that exercises them: (a) the invalid_status reason-code fix in template_parser.py's status check, which previously returned malformed_site_code for the same failure validate_ai_fallback already labels invalid_status, making one failure surface two codes depending on path (grepped the dashboards/metric filters for malformed_site_code first — no external references found, safe to diverge the two codes); (b) wrapping the WO Bedrock call in handler.py so a transport failure emits ai_fallback/bedrock_error in an except-and-reraise. This is deliberately not a naive reorder: the emit sits in the except block, not pre-call, so a gate-rejected email still emits only ai_fallback_rejected and wo_stack's "a rejected email emits nothing else" alarm contract doesn't double-count. A test computes the emitted series by hand to pin the no-double-count behavior. Neither change touches the handler event/return contract. _validate_new_po_values in the PO template_parser.py is split into per-rule helpers, and the V4 anchor-frame dataclass now carries summary_matches/price so V13 can consume them; extract_new_po (C901=35) is included in the split. Add ruff.toml enabling C901/PLR so the mccabe/complexity suppressions scattered through the tree stop being decorative; derived_fields.py is under the shadow-bake freeze so its violations are silenced via a per-file ignore with a justification comment instead of an in-file edit, and the handful of other pre-existing violations surfaced by turning the config on get the same per-file-ignore treatment with a reason, or a fix where the file isn't frozen. scripts/ is added to the CI lint scope. CI gains an explicit --cov module list (lambdas/po and wo email_processor + web_ui, po/site_extractor, lambdas/shared) plus --cov-fail-under=80, since web_ui and site_extractor lack __init__.py markers and a bare --cov=lambdas silently skips them for the missing package marker; .coveragerc omits the test dirs themselves from the count. The Phase 0 AST bundle-consistency test stays in the standard pytest run. .gitignore picks up the resulting .coverage data file. docs/po-template-parser.md gets a small correction: the EXTRACTION_PROMPT declares quantity/price as "number or null", not JSON strings, so parse_float=Decimal already handles a conforming Bedrock response — the doc previously implied the coercion path was the primary mechanism rather than a defensive net for non-conforming responses. * test: lock attribute-context quote escaping in web_ui hostile-field test The escaping regression lock asserted only the element-context vector (raw <script> absent, &lt;script&gt; present) while its docstring claimed quotes were covered -- the payload's " and ' were never asserted on, so a quote-escaping regression on the onclick row-link sink (attribute breakout -> event-handler injection) would have passed green. /sh-security-review finding WC-01 (confirmed medium, test-integrity). Add assertions that the onclick sink's JSON string renders its opening quote as &quot; (raw " after window.location= fails), that the payload's quote characters appear only entity-escaped, and that the raw payload never appears anywhere in the body. Mutation-verified: the test now fails when the sink's quote-escaping is dropped. * test: address Open SWE review — xfail the web_ui non-ASCII auth pin, document subset coverage-floor override - tests/test_web_ui_auth.py: replace the TypeError characterization pin with an xfail(strict, raises=TypeError) asserting the DESIRED fail-closed (False) behavior. Documents the intended fix and auto-fails (xpass) once web_ui_auth is corrected, instead of requiring a passing test to be knowingly deleted. The module stays frozen this phase; the underlying hmac.compare_digest ASCII-only defect is tracked as a follow-up. - pytest.ini: document that the aggregate 80% floor (enforced in CI via the reusable workflow's bare pytest) red-exits local subset runs by design, with the --cov-fail-under=0 override for iteration. Floor stays in addopts because the centralized ci-python-sam workflow exposes no per-run test command.
2026-07-20 16:19:15 -04:00
"""Handler-level SES-auth reject seam (constraint 5b), per pipeline.
The sender-auth gate sits AFTER ``get_object`` in both handlers, and the
allowlist is read from the environment at CALL time. With an empty
``ALLOWED_DKIM_DOMAINS`` and NO auth monkeypatch, every email must be rejected:
ZERO Bedrock calls, ZERO DynamoDB writes, and NO raise (rejected mail is skipped,
never DLQ'd). This closes the hole where deleting the gate line still passed
every other test (those bypass auth via monkeypatch).
An accept-path companion per pipeline proves the gate lets good mail THROUGH when
the allowlist is set to the fixture's SES-stamped domain.
Parameterized over both handlers via the root-conftest ``email_handler`` fixture;
all wiring goes through the shared loader + support package (no bare imports; PO
web_ui lacks __init__.py -- the loader, never package imports).
"""
import sys
from tests.support import FakeDynamoResource, FakeS3, load_raw
# Per-pipeline reject fixture: any committed .eml works (the gate fails at the
# empty-allowlist check, before the DKIM verdict even matters).
_REJECT_FIXTURE = {
"po": ("po", "ai-fallback", "comment-01"),
"wo": ("wo", "ai-fallback", "unknown-subject"),
}
class _RecordingBedrock:
"""Records every invoke_model call so the test can assert ZERO were made."""
def __init__(self):
self.calls = []
def invoke_model(self, modelId, body): # noqa: N803 (boto3 kwarg name)
self.calls.append((modelId, body))
raise AssertionError("bedrock must not be called on a rejected email")
def _siblings(handler_module):
name = handler_module.__name__ # "po_email_processor_handler" / "wo_..."
pipeline = name.split("_", 1)[0]
return (
pipeline,
sys.modules[f"{name}__extraction"],
sys.modules[f"{name}__persistence"],
)
def _event(key="inbound/o1"):
return {
"Records": [
{"s3": {"bucket": {"name": "ingest-emails-x"}, "object": {"key": key}}}
]
}
def test_empty_allowlist_rejects_before_bedrock_and_writes(email_handler, monkeypatch):
pipeline, extraction, persistence = _siblings(email_handler)
raw = load_raw(*_REJECT_FIXTURE[pipeline])
# Env read at call time; the gate sits after get_object, so a fake S3 is
# still needed to reach it.
monkeypatch.setenv("ALLOWED_DKIM_DOMAINS", "")
monkeypatch.setattr(email_handler, "s3", FakeS3({"inbound/o1": raw}))
bedrock = _RecordingBedrock()
monkeypatch.setattr(extraction, "bedrock", bedrock)
fake = FakeDynamoResource()
monkeypatch.setattr(persistence, "dynamodb", fake)
# NO raise: rejected mail is skipped, and the invocation returns normally.
result = email_handler.handler(_event(), None)
assert result == {"statusCode": 200, "body": "OK"}
# ZERO bedrock calls, ZERO writes.
assert bedrock.calls == []
assert not any(table.updates or table.puts for table in fake.tables.values())
# --- Accept-path companions: the gate lets good mail THROUGH -----------------
# A full-contract PO AI payload (valid -> a write lands once past the gate).
_PO_AI_PAYLOAD = {
"email_type": "new_po",
"po_number": "2D-70000001",
"po_status": "Issued - Created",
"source_system": "coupa",
"submitted_by": None,
"on_behalf_of": None,
"order_date": None,
"revision_date": None,
"last_opened": None,
"acknowledged_at": None,
"payment_terms": None,
"requisition_number": None,
"department": None,
"view_order_url": None,
"supplier": {"name": None},
"site_code": None,
"ship_to": {
"name": None,
"address": None,
"street": None,
"city": None,
"state": None,
"zip": None,
"location_code": None,
"attn": None,
},
"total_amount": 123.45,
"currency": "USD",
"fiscal_year": None,
"trade": None,
"coupa_category": None,
"line_items": [],
}
_WO_AI_PAYLOAD = {
"email_type": "comment",
"work_order_id": "77777777777",
"description": None,
"status": None,
"site_code": None,
"building": None,
"address": None,
"severity": None,
"priority": None,
"date_reported": None,
"scheduled_start": None,
"due_date": None,
"assigned_to": None,
"commenter": None,
"comment_text": "ai extracted",
"comment_time": None,
}
class _FakeBody:
def __init__(self, data):
self._data = data
def read(self):
return self._data
class _JsonBedrock:
def __init__(self, payload):
self.calls = []
self._payload = payload
def invoke_model(self, modelId, body): # noqa: N803
import json
self.calls.append((modelId, body))
text = json.dumps(self._payload)
return {"body": _FakeBody(json.dumps({"content": [{"text": text}]}).encode())}
def test_po_accept_path_passes_gate_and_writes(po_handler, monkeypatch):
"""PO: a scrubbed Coupa fixture (dkim=pass amazon.coupahost.com) authenticates
when the allowlist names its stamped domain -- processing reaches Bedrock and
a purchase-order write lands. NO auth monkeypatch (the real gate runs)."""
extraction = sys.modules["po_email_processor_handler__extraction"]
persistence = sys.modules["po_email_processor_handler__persistence"]
monkeypatch.setenv("ALLOWED_DKIM_DOMAINS", "amazon.coupahost.com")
monkeypatch.setattr(
po_handler,
"s3",
FakeS3({"inbound/o1": load_raw("po", "ai-fallback", "new-po-05")}),
)
bedrock = _JsonBedrock(_PO_AI_PAYLOAD)
monkeypatch.setattr(extraction, "bedrock", bedrock)
fake = FakeDynamoResource()
monkeypatch.setattr(persistence, "dynamodb", fake)
result = po_handler.handler(_event(), None)
assert result == {"statusCode": 200, "body": "OK"}
assert bedrock.calls, "authenticated mail must reach the Bedrock fallback"
assert fake.tables[persistence.PO_TABLE].updates
def test_wo_accept_path_passes_gate_and_writes(wo_handler, monkeypatch):
"""WO: the one permitted new fixture (dkim=pass seahaven.com) authenticates
when the allowlist names seahaven.com -- processing reaches Bedrock and a
work-order write lands. NO auth monkeypatch (the real gate runs)."""
extraction = sys.modules["wo_email_processor_handler__extraction"]
persistence = sys.modules["wo_email_processor_handler__persistence"]
monkeypatch.setenv("ALLOWED_DKIM_DOMAINS", "seahaven.com")
monkeypatch.setattr(
wo_handler,
"s3",
FakeS3({"inbound/o1": load_raw("wo", "ses-stamped", "auth-pass-01")}),
)
bedrock = _JsonBedrock(_WO_AI_PAYLOAD)
monkeypatch.setattr(extraction, "bedrock", bedrock)
fake = FakeDynamoResource()
monkeypatch.setattr(persistence, "dynamodb", fake)
result = wo_handler.handler(_event(), None)
assert result == {"statusCode": 200, "body": "OK"}
assert bedrock.calls, "authenticated mail must reach the Bedrock fallback"
assert fake.tables[persistence.WORK_ORDERS_TABLE].updates