feat: extract lambdas/shared/ — single-source ses_auth, web_ui auth, email parsing, EMF emitter (refactor phase 3) (#111)
Four modules move into the handbook-mandated lambdas/shared/ location,
collapsing duplicated logic that had to be kept in sync by hand across
the PO and WO pipelines:
- ses_auth.py: the PO and WO copies were verified sha256-identical
against the feature/phase-7-ops-recovery baseline before the move
(no drift since the last audit). shared/ses_auth.py is the exact
bytes of that one copy; both originals are git rm'd (the PO copy
via rename, the WO copy as a straight delete). Bundling lands the
module flat in /asset-output for both email processors, so the
handlers keep `from ses_auth import authenticate_inbound_email`
unchanged — zero handler diff for this move, which is what keeps
fail-closed auth byte-identical through the change.
- web_ui_auth.py: extracts the byte-identical _get_auth_token /
_header / is_authenticated block plus the four token-cache globals
out of both web_ui handlers. The per-stack INFRA-74 comments stay
in each handler as-is (deliberately drifted wording, stack-specific)
rather than being unified into the shared module. Fail-closed
semantics (unset ARN or Secrets Manager exception -> deny) are
unchanged.
- email_parsing.py: parse_raw_email ships as the superset version that
returns cc unconditionally. WO's output is bit-identical to before;
PO simply ignores the cc field rather than being "cleaned up" to
consume it. No second variant is kept.
- emf.py: a generic emitter parameterized by namespace, dimension
sets, and properties. Every call site's emitted EMF envelope is
unchanged, including the load-bearing
[["ParseMethod"],["ParseMethod","TemplateId"]] dimension-set shape
the alarms and metric filters depend on. Emission ordering is
untouched: PO still emits ai_fallback before the Bedrock call, WO
still emits its mutually-exclusive ai_fallback/ai_fallback_rejected
after its gate. The deliberate-double-count comments survive.
_emit_derived_agreement_metric was found living inside
derived_fields.py, so per the DERIVED-FIELDS exception it is left
as a third, unconverted copy (derived_fields.py and the shadow
DerivedFieldAgreement telemetry stay untouchable while that bake
runs) — a comment there points at shared/emf.py for the eventual
follow-up.
Bundling: both email-processor cdk bundling commands gain a trailing
`cp shared/*.py /asset-output/` (they were already cp-only post-Phase
7, so no pip step or manylinux pin is reintroduced). Both web_ui
functions gain the same widened-root staging so web_ui_auth.py ships
beside their handler; site_extractor's from_asset is untouched.
Tests: PO_EXPECTED_TOP_LEVEL_MODULES gains the shared modules that now
ship, the AST sibling-import check resolves imports whose source now
lives under shared/, and the new shared cp line has its own
revert/mutation detection. _SIBLING_MODULES resolution and
_po_parser_support.py now load ses_auth/email_parsing/emf from
shared/; the two-copy ses_auth byte-identity fixture-hygiene test is
retired as obsolete now that there is one copy, and the ses_auth
fixture parameterization over two identical copies is dropped. The
sys.modules save/restore dance for template_parser (still duplicated
per-pipeline) is left in place.
2026-07-20 13:38:23 -04:00
|
|
|
import hmac
|
|
|
|
|
import logging
|
|
|
|
|
import os
|
|
|
|
|
import time
|
|
|
|
|
|
|
|
|
|
import boto3
|
|
|
|
|
|
|
|
|
|
logger = logging.getLogger()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
_WEB_UI_AUTH_TOKEN_SECRET_ARN = os.environ.get("WEB_UI_AUTH_TOKEN_SECRET_ARN")
|
|
|
|
|
# Refresh the cached token this often so a rotated secret propagates without
|
|
|
|
|
# waiting for the execution environment to recycle (emergency-rotation path).
|
|
|
|
|
_AUTH_TOKEN_CACHE_TTL_SECONDS = 300
|
|
|
|
|
_auth_token_cache = None
|
|
|
|
|
_auth_token_cached_at = 0.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _get_auth_token() -> str | None:
|
|
|
|
|
"""Fetch the shared web UI auth token from Secrets Manager.
|
|
|
|
|
|
|
|
|
|
Cached in the warm container for a short TTL so we don't hit Secrets Manager
|
|
|
|
|
on every request, while still picking up a rotated secret within the TTL
|
|
|
|
|
rather than only when the execution environment recycles. Returns None when
|
|
|
|
|
not configured or unreadable (the caller then fails closed).
|
|
|
|
|
"""
|
|
|
|
|
global _auth_token_cache, _auth_token_cached_at
|
|
|
|
|
now = time.monotonic()
|
|
|
|
|
if (
|
|
|
|
|
_auth_token_cache is not None
|
|
|
|
|
and now - _auth_token_cached_at < _AUTH_TOKEN_CACHE_TTL_SECONDS
|
|
|
|
|
):
|
|
|
|
|
return _auth_token_cache
|
|
|
|
|
if not _WEB_UI_AUTH_TOKEN_SECRET_ARN:
|
|
|
|
|
return None
|
|
|
|
|
secrets = boto3.client("secretsmanager")
|
|
|
|
|
try:
|
|
|
|
|
secret = secrets.get_secret_value(SecretId=_WEB_UI_AUTH_TOKEN_SECRET_ARN)
|
|
|
|
|
_auth_token_cache = secret["SecretString"]
|
|
|
|
|
_auth_token_cached_at = now
|
|
|
|
|
return _auth_token_cache
|
|
|
|
|
except Exception:
|
|
|
|
|
# Fail closed (return None -> caller 401s) but surface the failure: a
|
|
|
|
|
# Secrets Manager permission/config error would otherwise make every
|
|
|
|
|
# request 401 with no operational signal. The secret value is never
|
|
|
|
|
# logged.
|
|
|
|
|
logger.exception(
|
|
|
|
|
"Failed to fetch web UI auth token from Secrets Manager; "
|
|
|
|
|
"denying request (failing closed)"
|
|
|
|
|
)
|
|
|
|
|
return None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _header(event: dict, name: str) -> str:
|
|
|
|
|
"""Case-insensitive header lookup from a Lambda Function URL / APIGW event."""
|
|
|
|
|
headers = event.get("headers") or {}
|
|
|
|
|
name_lower = name.lower()
|
|
|
|
|
for key, value in headers.items():
|
|
|
|
|
if key.lower() == name_lower:
|
|
|
|
|
return value or ""
|
|
|
|
|
return ""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def is_authenticated(event: dict) -> bool:
|
|
|
|
|
"""Constant-time check of the request's shared secret against the configured
|
|
|
|
|
token. Fails closed when no token is configured."""
|
|
|
|
|
token = _get_auth_token()
|
|
|
|
|
if not token:
|
|
|
|
|
return False
|
|
|
|
|
presented = _header(event, "x-auth-token")
|
|
|
|
|
if not presented:
|
|
|
|
|
auth = _header(event, "authorization")
|
|
|
|
|
if auth.lower().startswith("bearer "):
|
|
|
|
|
presented = auth[7:].strip()
|
|
|
|
|
if not presented:
|
|
|
|
|
return False
|
feat(api): procurement-api read stack + OpenAPI docs (SHOC reconciliation path) (#127)
* feat(api): add procurement-api stack - read API + OpenAPI docs page
Third CDK stack: API Gateway REST API (IAM SigV4) over both pipelines'
tables, replacing SHOC's retired SyncController cross-account DynamoDB
scan as the reconciliation/backfill path.
- lambdas/api/: handler (healthcheck + docs-token gate + router dispatch),
router (single route table), pagination (opaque cursor, hostile -> 400),
Decimal-safe serialization, wo_repo/po_repo reads. No VendorReplies.
- OpenAPI 3.1 spec as source of truth incl. top-level webhooks section
documenting the outbound SHOC feed; phase-2 write endpoints x-planned
(router answers 501). Self-contained /docs page, no CDN.
- Auth: AWS_IAM on data routes + resource policy scoped to exactly
arn:aws:iam::396287094661:role/shoc-backend-dev on GET/*; /docs and
/openapi.json carve-out is token-gated in the Lambda via shared
web_ui_auth (fail-closed, INFRA-74 posture).
- KMS: explicit Decrypt/DescribeKey on the DynamoDB CMK from SSM
(name-imported table drops the key association - INFRA-104 class).
- Alarms: errors/throttles/duration(p99>=22.5s) + gateway 5xx, ALARM-only
to site-alerts. No access logging in v1 (docs ?token= shim stays out of
logs); cloud_watch_role=False.
- Tests: handler auth-seam + routing + Decimal round-trip; moto cursor
pagination incl. hostile cursors; spec<->router drift gate; bundle
AST pins for the api command; pytest.ini --cov + loader siblings.
- Deploy role: third stack DescribeStacks ARN + procurement-api smoke
invoke ARN (re-run create-deploy-role.sh before merge).
* harden(api): apply sh-security-review findings to procurement-api
Fan-out (6 detectors) + review findings resolved:
Correctness / DoS:
- pagination: require EXACT key-set match (was subset) so a partial/foreign
composite cursor can't reach DynamoDB as an inconsistent ExclusiveStartKey
-> ValidationException -> 500; comments Query now pins the cursor's
work_order_id to the path entity.
- handler: map botocore ValidationException to 400 (defense in depth) so a
crafted cursor can't drive the zero-threshold 5xx alarm.
- web_ui_auth: compare tokens as bytes; a non-ASCII presented token now fails
closed (401) instead of crashing hmac.compare_digest into a 500. Resolves the
pre-existing xfail(strict) follow-up test; hardens the web UIs too.
Docs page:
- typeStr() now escapes the one spec-derived string that reached innerHTML.
- spec inlined into the docs <script> block escapes "<" -> < (</script>
breakout guard); /openapi.json still served byte-faithful.
- Cache-Control: no-store + Referrer-Policy: no-referrer on docs responses so
the ?token= URL stays out of caches/Referer.
- spec-drift test asserts the committed spec carries no "</" / "<!--".
IAM / IaC:
- resource policy enumerates the 7 data GET resources instead of GET/* so a
future GET route can't silently inherit SHOC cross-account reach.
- kms:Decrypt grant gains a kms:ViaService=dynamodb condition.
- stage throttling (50 rps / 100 burst) bounds the unauthenticated /docs blast
radius below the 10k account default.
- corrected the PATCH/POST comment (same-account callers aren't blocked by the
resource policy; 501 handler + absent write grant are the gate).
- documented the RETAIN log-group first-deploy rollback trap and the
resource-policy-needs-redeploy gotcha in-stack.
Mandatory GPT-4.1 cross-family review of the full policy surface: no BLOCK/FIX.
675 tests pass, ruff clean, cdk synth green.
2026-07-23 19:32:20 -04:00
|
|
|
# Compare as bytes: hmac.compare_digest raises TypeError on non-ASCII str
|
|
|
|
|
# operands, which a crafted token (?token=%C3%A9 or a non-ASCII header)
|
|
|
|
|
# would otherwise turn into an uncaught 500. Bytes always compare in
|
|
|
|
|
# constant time, so a non-matching token fails closed (401) instead.
|
|
|
|
|
return hmac.compare_digest(presented.encode("utf-8"), token.encode("utf-8"))
|