procurement-ingest/cdk/procurement_api_stack.py

362 lines
16 KiB
Python
Raw Normal View History

feat(api): procurement-api read stack + OpenAPI docs (SHOC reconciliation path) (#127) * feat(api): add procurement-api stack - read API + OpenAPI docs page Third CDK stack: API Gateway REST API (IAM SigV4) over both pipelines' tables, replacing SHOC's retired SyncController cross-account DynamoDB scan as the reconciliation/backfill path. - lambdas/api/: handler (healthcheck + docs-token gate + router dispatch), router (single route table), pagination (opaque cursor, hostile -> 400), Decimal-safe serialization, wo_repo/po_repo reads. No VendorReplies. - OpenAPI 3.1 spec as source of truth incl. top-level webhooks section documenting the outbound SHOC feed; phase-2 write endpoints x-planned (router answers 501). Self-contained /docs page, no CDN. - Auth: AWS_IAM on data routes + resource policy scoped to exactly arn:aws:iam::396287094661:role/shoc-backend-dev on GET/*; /docs and /openapi.json carve-out is token-gated in the Lambda via shared web_ui_auth (fail-closed, INFRA-74 posture). - KMS: explicit Decrypt/DescribeKey on the DynamoDB CMK from SSM (name-imported table drops the key association - INFRA-104 class). - Alarms: errors/throttles/duration(p99>=22.5s) + gateway 5xx, ALARM-only to site-alerts. No access logging in v1 (docs ?token= shim stays out of logs); cloud_watch_role=False. - Tests: handler auth-seam + routing + Decimal round-trip; moto cursor pagination incl. hostile cursors; spec<->router drift gate; bundle AST pins for the api command; pytest.ini --cov + loader siblings. - Deploy role: third stack DescribeStacks ARN + procurement-api smoke invoke ARN (re-run create-deploy-role.sh before merge). * harden(api): apply sh-security-review findings to procurement-api Fan-out (6 detectors) + review findings resolved: Correctness / DoS: - pagination: require EXACT key-set match (was subset) so a partial/foreign composite cursor can't reach DynamoDB as an inconsistent ExclusiveStartKey -> ValidationException -> 500; comments Query now pins the cursor's work_order_id to the path entity. - handler: map botocore ValidationException to 400 (defense in depth) so a crafted cursor can't drive the zero-threshold 5xx alarm. - web_ui_auth: compare tokens as bytes; a non-ASCII presented token now fails closed (401) instead of crashing hmac.compare_digest into a 500. Resolves the pre-existing xfail(strict) follow-up test; hardens the web UIs too. Docs page: - typeStr() now escapes the one spec-derived string that reached innerHTML. - spec inlined into the docs <script> block escapes "<" -> < (</script> breakout guard); /openapi.json still served byte-faithful. - Cache-Control: no-store + Referrer-Policy: no-referrer on docs responses so the ?token= URL stays out of caches/Referer. - spec-drift test asserts the committed spec carries no "</" / "<!--". IAM / IaC: - resource policy enumerates the 7 data GET resources instead of GET/* so a future GET route can't silently inherit SHOC cross-account reach. - kms:Decrypt grant gains a kms:ViaService=dynamodb condition. - stage throttling (50 rps / 100 burst) bounds the unauthenticated /docs blast radius below the 10k account default. - corrected the PATCH/POST comment (same-account callers aren't blocked by the resource policy; 501 handler + absent write grant are the gate). - documented the RETAIN log-group first-deploy rollback trap and the resource-policy-needs-redeploy gotcha in-stack. Mandatory GPT-4.1 cross-family review of the full policy surface: no BLOCK/FIX. 675 tests pass, ruff clean, cdk synth green.
2026-07-23 19:32:20 -04:00
"""procurement-api stack: read-only REST API over both pipelines' tables.
Third stack in the app. Serves work orders + comments (WO stack tables) and
purchase orders + verified sites (PO stack tables) to SigV4 callers -- the
primary consumer is the SHOC backend, for which this API replaces the retired
SyncController cross-account DynamoDB scan as the reconciliation/backfill
path. Also hosts the token-gated OpenAPI docs page (/docs, /openapi.json).
Tables are imported by fixed physical name (Table.from_table_name), NOT
passed as cross-stack objects: object passing would synthesize CFN Exports
from the owning stacks and lock them against future changes to the tables.
The one thing name-import does NOT carry is the purchase-orders CMK
association -- see the explicit KMS grant below.
"""
import aws_cdk as cdk
from aws_cdk import (
Duration,
Stack,
aws_apigateway as apigateway,
aws_cloudwatch as cloudwatch,
aws_cloudwatch_actions as cw_actions,
aws_dynamodb as dynamodb,
aws_iam as iam,
aws_kms as kms,
aws_lambda as lambda_,
aws_secretsmanager as secretsmanager,
aws_sns as sns,
aws_ssm as ssm,
)
from constructs import Construct
import common
# The only cross-account caller. Grants are to this exact role ARN -- future
# shoc-backend-staging/-prod roles are each a deliberate, individually
# cross-reviewed addition (no wildcard/prefix trust).
SHOC_BACKEND_DEV_ROLE_ARN = "arn:aws:iam::396287094661:role/shoc-backend-dev"
STAGE_NAME = "prod"
class ProcurementApiStack(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
alarm_topic = sns.Topic.from_topic_arn(
self,
"SiteAlertsTopic",
f"arn:aws:sns:{self.region}:{self.account}:site-alerts",
)
work_orders_table = dynamodb.Table.from_table_name(
self, "WorkOrdersTable", "WorkOrders"
)
comments_table = dynamodb.Table.from_table_name(
self, "CommentsTable", "WorkOrderComments"
)
po_table = dynamodb.Table.from_table_name(
self, "PurchaseOrdersTable", "purchase-orders"
)
verified_sites_table = dynamodb.Table.from_table_name(
self, "VerifiedSitesTable", "verified-sites"
)
web_ui_auth_secret = secretsmanager.Secret.from_secret_name_v2(
self,
"WebUiAuthToken",
"procurement-ingest/web-ui-auth-token",
)
# purchase-orders is SSE-KMS encrypted with the org DynamoDB CMK. A
# name-imported Table has no encryption-key association, so
# grant_read_data alone leaves the reader without kms:Decrypt and every
# purchase-orders read AccessDenies at runtime (the INFRA-104 failure
# class). Import the key from the same SSM parameter po_stack uses and
# grant it explicitly.
dynamodb_cmk = kms.Key.from_key_arn(
self,
"DynamoDbCmk",
ssm.StringParameter.value_for_string_parameter(
self, "/seahaven/dynamodb/cmk-arn"
),
)
# RETAIN log group (INFRA-114). This is the only stateful resource in
# this otherwise-stateless stack, so it carries the fresh-deploy
# rollback trap: if the FIRST create fails after the group exists,
# rollback deletes everything else but retains the group, and the retry
# CREATE then collides on "/aws/lambda/procurement-api already exists".
# Recovery: delete that log group before re-running a failed first
# deploy (same class as the RETAIN-orphan recovery in the deploy-role
# runbook / reference_cfn_deploy_role_gotchas).
api_log_group = common.make_function_log_group(
self, "ProcurementApi", "procurement-api"
)
api_fn = lambda_.Function(
self,
"ProcurementApi",
function_name="procurement-api",
runtime=lambda_.Runtime.PYTHON_3_12,
architecture=lambda_.Architecture.ARM_64,
handler="handler.handler",
code=lambda_.Code.from_asset(
"../lambdas",
exclude=["**/__pycache__/**"],
bundling=cdk.BundlingOptions(
image=lambda_.Runtime.PYTHON_3_12.bundling_image,
command=[
"bash",
"-c",
# Non-recursive glob ships every api/ sibling (the
# allowlist-omission trap from PR #105/PR #2); the spec
# and docs page ride along because the handler serves
# them from its own package dir. web_ui_auth.py must
# land FLAT beside handler.py for the bare import.
"cp api/*.py /asset-output/ && "
"cp api/openapi.json /asset-output/ && "
"cp api/docs.html /asset-output/ && "
"cp shared/web_ui_auth.py /asset-output/ && "
"rm -rf /asset-output/__pycache__",
],
),
),
timeout=Duration.seconds(30),
memory_size=256,
log_group=api_log_group,
environment={
"WORK_ORDERS_TABLE": work_orders_table.table_name,
"COMMENTS_TABLE": comments_table.table_name,
"PO_TABLE": po_table.table_name,
"VERIFIED_SITES_TABLE": verified_sites_table.table_name,
"WEB_UI_AUTH_TOKEN_SECRET_ARN": web_ui_auth_secret.secret_arn,
},
)
work_orders_table.grant_read_data(api_fn)
comments_table.grant_read_data(api_fn)
po_table.grant_read_data(api_fn)
verified_sites_table.grant_read_data(api_fn)
web_ui_auth_secret.grant_read(api_fn)
api_fn.add_to_role_policy(
iam.PolicyStatement(
actions=["kms:Decrypt", "kms:DescribeKey"],
resources=[dynamodb_cmk.key_arn],
# Scope the grant to the DynamoDB data path only: the role can
# decrypt purchase-orders items via DynamoDB, never call
# kms:Decrypt directly on arbitrary ciphertext under the shared
# org CMK.
conditions={
"StringEquals": {
"kms:ViaService": f"dynamodb.{self.region}.amazonaws.com"
}
},
)
)
# Resource policy: once a REST API has one, anything not explicitly
# allowed is denied -- so the docs routes need their own Allow or the
# NONE-auth methods would be black-holed. The docs routes are still
# token-gated inside the Lambda (fail-closed), so this is not an
# unauthenticated data path (INFRA-74 posture).
#
# The SHOC data grant enumerates the exact GET resources rather than
# GET/*: adding a RESOURCE must be as reviewable as adding a PRINCIPAL,
# so a future GET route can't silently inherit cross-account reach
# without a policy diff. (Note: this resource policy binds CROSS-account
# callers only; a same-account principal holding execute-api:Invoke is
# authorized by its own identity policy under AWS union semantics -- it
# is NOT constrained here, including on the planned PATCH/POST methods,
# which is why those also rely on the 501 handler + absent write grant,
# not on this policy, until phase 2.)
shoc_data_resources = [
f"execute-api:/{STAGE_NAME}/GET/work-orders",
f"execute-api:/{STAGE_NAME}/GET/work-orders/*",
f"execute-api:/{STAGE_NAME}/GET/purchase-orders",
f"execute-api:/{STAGE_NAME}/GET/purchase-orders/*",
f"execute-api:/{STAGE_NAME}/GET/verified-sites",
f"execute-api:/{STAGE_NAME}/GET/verified-sites/*",
]
api_policy = iam.PolicyDocument(
statements=[
iam.PolicyStatement(
sid="ShocBackendDevDataRead",
principals=[iam.ArnPrincipal(SHOC_BACKEND_DEV_ROLE_ARN)],
actions=["execute-api:Invoke"],
resources=shoc_data_resources,
),
# SECURITY INVARIANT: this AnyPrincipal allow is safe only
# while /docs//openapi.json serve static docs (handler
# enforces the shared token, fail-closed). Widening these
# routes to dynamic data, or enabling access logging (which
# would record the docs ?token= shim), requires a security
# re-review + docs-token rotation.
iam.PolicyStatement(
sid="DocsTokenGatedRoutes",
principals=[iam.AnyPrincipal()],
actions=["execute-api:Invoke"],
resources=[
f"execute-api:/{STAGE_NAME}/GET/docs",
f"execute-api:/{STAGE_NAME}/GET/openapi.json",
],
),
]
)
# OPERATIONAL NOTE: API Gateway serves the resource policy from the
# deployed stage snapshot, and CDK's Deployment hash is computed from
# resources/methods, not the RestApi Policy. A later policy-ONLY change
# (e.g. revoking the SHOC role) will UPDATE the RestApi but keep serving
# the old policy until a new Deployment is forced (any method/resource
# change, or a salted deployment). When tightening this policy, force a
# redeploy and verify the effective policy post-deploy.
api = apigateway.RestApi(
self,
"ProcurementRestApi",
rest_api_name="procurement-api",
description=(
"Read API over procurement-ingest work orders + purchase "
"orders; token-gated OpenAPI docs at /docs"
),
endpoint_types=[apigateway.EndpointType.REGIONAL],
policy=api_policy,
deploy_options=apigateway.StageOptions(
stage_name=STAGE_NAME,
# Bound the blast radius of the unauthenticated /docs routes
# (and the whole API) below the 10k rps account default -- this
# is a low-volume reconciliation/backfill API, not a hot path.
throttling_rate_limit=50,
throttling_burst_limit=100,
),
# No access/execution logging in v1: avoids the account-level API
# Gateway CloudWatch role prerequisite AND keeps the docs ?token=
# query shim out of any log. Rotate the docs token before ever
# enabling access logging here.
cloud_watch_role=False,
)
integration = apigateway.LambdaIntegration(api_fn)
iam_auth = {"authorization_type": apigateway.AuthorizationType.IAM}
work_orders = api.root.add_resource("work-orders")
work_orders.add_method("GET", integration, **iam_auth)
wo_by_id = work_orders.add_resource("{workOrderId}")
wo_by_id.add_method("GET", integration, **iam_auth)
# Phase-2 planned write endpoints: deployed but the handler answers 501
# and the role holds no DynamoDB write grant. The resource policy denies
# these to the cross-account SHOC role (GET-only enumeration above); a
# same-account caller is NOT blocked by the resource policy, so the 501
# handler + absent write grant are the real gate until phase 2 lands the
# deliberate policy + handler + write-grant change with its own review.
wo_by_id.add_method("PATCH", integration, **iam_auth)
wo_comments = wo_by_id.add_resource("comments")
wo_comments.add_method("GET", integration, **iam_auth)
wo_comments.add_method("POST", integration, **iam_auth)
purchase_orders = api.root.add_resource("purchase-orders")
purchase_orders.add_method("GET", integration, **iam_auth)
purchase_orders.add_resource("{poNumber}").add_method(
"GET", integration, **iam_auth
)
verified_sites = api.root.add_resource("verified-sites")
verified_sites.add_method("GET", integration, **iam_auth)
verified_sites.add_resource("{siteCode}").add_method(
"GET", integration, **iam_auth
)
api.root.add_resource("docs").add_method(
"GET",
integration,
authorization_type=apigateway.AuthorizationType.NONE,
)
api.root.add_resource("openapi.json").add_method(
"GET",
integration,
authorization_type=apigateway.AuthorizationType.NONE,
)
# --- Alarms (ALARM-only -> site-alerts, NOT_BREACHING) ---
# common.add_standard_lambda_alarms is NOT used here: its duration
# threshold is a fixed 45000 ms (75% of the processors' 60 s timeout),
# which this function's 30 s timeout can never reach. Same idiom,
# right-sized thresholds.
api_fn.metric_errors(period=Duration.minutes(5), statistic="Sum").create_alarm(
self,
"ProcurementApiErrorsAlarm",
alarm_name="procurement-api-errors",
alarm_description=(
"procurement-api Lambda raised (bundle/init failures; the "
"handler catches request errors, so any signal here is "
"structural)"
),
threshold=0,
evaluation_periods=1,
comparison_operator=cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD,
treat_missing_data=cloudwatch.TreatMissingData.NOT_BREACHING,
).add_alarm_action(cw_actions.SnsAction(alarm_topic))
api_fn.metric_throttles(
period=Duration.minutes(5), statistic="Sum"
).create_alarm(
self,
"ProcurementApiThrottlesAlarm",
alarm_name="procurement-api-throttles",
alarm_description="procurement-api Lambda throttled",
threshold=0,
evaluation_periods=1,
comparison_operator=cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD,
treat_missing_data=cloudwatch.TreatMissingData.NOT_BREACHING,
).add_alarm_action(cw_actions.SnsAction(alarm_topic))
api_fn.metric_duration(
period=Duration.minutes(5), statistic="p99"
).create_alarm(
self,
"ProcurementApiDurationAlarm",
alarm_name="procurement-api-duration",
alarm_description=(
"procurement-api p99 duration >= 22.5s (75% of the 30s "
"timeout; scans degrading toward timeout)"
),
threshold=22500,
evaluation_periods=3,
datapoints_to_alarm=2,
comparison_operator=cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD,
treat_missing_data=cloudwatch.TreatMissingData.NOT_BREACHING,
).add_alarm_action(cw_actions.SnsAction(alarm_topic))
# Gateway-side 5xx: catches what the Lambda's own Errors metric can't
# (the handler returns clean 500s; integration faults surface here).
# No 4XX alarm -- 401/403/404 are expected traffic.
cloudwatch.Metric(
namespace="AWS/ApiGateway",
metric_name="5XXError",
dimensions_map={"ApiName": "procurement-api", "Stage": STAGE_NAME},
period=Duration.minutes(5),
statistic="Sum",
).create_alarm(
self,
"ProcurementApi5xxAlarm",
alarm_name="procurement-api-5xx",
alarm_description="procurement-api gateway 5XX responses",
threshold=0,
evaluation_periods=1,
comparison_operator=cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD,
treat_missing_data=cloudwatch.TreatMissingData.NOT_BREACHING,
).add_alarm_action(cw_actions.SnsAction(alarm_topic))
cdk.CfnOutput(
self,
"ApiEndpointUrl",
value=api.url,
description="procurement-api invoke URL (stage prod)",
)
cdk.CfnOutput(
self,
"ProcurementApiFunctionArn",
value=api_fn.function_arn,
description="ARN of the procurement-api Lambda",
)