mirror of
https://github.com/Sea-Haven-Industries/procurement-ingest.git
synced 2026-09-30 08:23:14 +00:00
14 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Adam Moussa
|
62f004f730
|
ci: add merge_group trigger for required ci / ci (#193) | ||
|
dependabot[bot]
|
b0678edc2e
|
chore(deps): bump the minor-and-patch group with 4 updates
Bumps the minor-and-patch group with 4 updates: [Sea-Haven-Industries/.github/.github/workflows/ci-python-sam.yaml](https://github.com/sea-haven-industries/.github), [Sea-Haven-Industries/.github/.github/workflows/callable-dependency-review.yaml](https://github.com/sea-haven-industries/.github), [Sea-Haven-Industries/.github/.github/workflows/callable-labeler.yaml](https://github.com/sea-haven-industries/.github) and [Sea-Haven-Industries/.github/.github/workflows/callable-pr-policy.yaml](https://github.com/sea-haven-industries/.github). Updates `Sea-Haven-Industries/.github/.github/workflows/ci-python-sam.yaml` from 1.0.6 to 1.0.7 - [Release notes](https://github.com/sea-haven-industries/.github/releases) - [Commits]( |
||
|
dependabot[bot]
|
48b7f16bbd
|
chore(deps): bump the minor-and-patch group with 4 updates (#165)
Bumps the minor-and-patch group with 4 updates: [Sea-Haven-Industries/.github/.github/workflows/ci-python-sam.yaml](https://github.com/sea-haven-industries/.github), [Sea-Haven-Industries/.github/.github/workflows/callable-dependency-review.yaml](https://github.com/sea-haven-industries/.github), [Sea-Haven-Industries/.github/.github/workflows/callable-labeler.yaml](https://github.com/sea-haven-industries/.github) and [Sea-Haven-Industries/.github/.github/workflows/callable-pr-policy.yaml](https://github.com/sea-haven-industries/.github). Updates `Sea-Haven-Industries/.github/.github/workflows/ci-python-sam.yaml` from 1.0.3 to 1.0.6 - [Release notes](https://github.com/sea-haven-industries/.github/releases) - [Commits]( |
||
|
Adam Moussa
|
5a3729c583
|
chore(infra): remove cdk tree after hcp cutover (PLAT-89) (#164)
* chore(infra): remove cdk tree after hcp cutover Delete retired CDK sources, retarget bundle/principal contract tests to Terraform packaging, disable CDK synth in CI, and scrub deploy-adjacent docs. * fix(test): restore exact SHOC principal pin in terraform Pin shoc_consumer_role_arn's Terraform default and example to the trusted ARN, and require grant sites to consume local.shoc_consumer_role_arn only. |
||
|
dependabot[bot]
|
235fe66bbd
|
chore(deps): bump the minor-and-patch group with 4 updates (#150)
Bumps the minor-and-patch group with 4 updates: [Sea-Haven-Industries/.github/.github/workflows/ci-python-sam.yaml](https://github.com/sea-haven-industries/.github), [Sea-Haven-Industries/.github/.github/workflows/callable-dependency-review.yaml](https://github.com/sea-haven-industries/.github), [Sea-Haven-Industries/.github/.github/workflows/cd-cdk.yaml](https://github.com/sea-haven-industries/.github) and [Sea-Haven-Industries/.github/.github/workflows/callable-labeler.yaml](https://github.com/sea-haven-industries/.github). Updates `Sea-Haven-Industries/.github/.github/workflows/ci-python-sam.yaml` from 1.0.2 to 1.0.3 - [Release notes](https://github.com/sea-haven-industries/.github/releases) - [Commits]( |
||
|
Adam Moussa
|
61ccc68877
|
ci(deps): pin org reusable workflows to v1.0.2 (#148)
Some checks failed
Deploy / deploy (push) Has been cancelled
* ci(deps): pin org reusable workflows to v1.0.2 * style(ci): normalize workflow block spacing |
||
|
Adam Moussa
|
09946c0448
|
chore(deps): consolidated Dependabot bumps (checkout v7, setup-node v7, aws-cdk-lib 2.262.1, boto3 1.43.55) (#142)
Some checks are pending
Deploy / deploy (push) Waiting to run
* chore(deps): bump actions/checkout from 4.2.2 to 7.0.1 Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.2 to 7.0.1. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits]( |
||
|
Adam Moussa
|
b89e98a879
|
feat(api): Redocly lint gate + SHOC-themed /docs (Redoc theming, topbar, collapsible samples) (#130)
Some checks are pending
Deploy / deploy (push) Waiting to run
* feat(api): Add @redocly/cli as a dev dependency
Signed-off-by: Adam Moussa <adam@seahavenind.com>
* feat(api): Add Redocly configuration file with custom rules
Signed-off-by: Adam Moussa <adam@seahavenind.com>
* chore(api): Redocly lint config + bring openapi.json into compliance
redocly.yaml from the Redocly guidelines builder, with three generated
rules corrected: response-contains-property had the status codes as the
required body fields (intent was the Error schema's top-level 'error';
403 exempt since API Gateway emits AWS's {message} shape, 501 not 503);
operation-4xx-problem-details-rfc7807 off (adopting RFC 7807 would be a
runtime + SHOC-contract change, decided against); the two inert casing
rules (parameter names, schema properties) removed because both name
sets are contract-pinned (gateway resource paths, DynamoDB items).
Spec changes, no runtime impact: operationIds renamed to method-prefixed
kebab-case (get-work-orders, post-work-order-comment, ...); tags added to
all 15 operations + root tags object (groups the Redoc sidebar); examples
on all six parameters; license field; server description punctuation; two
descriptions reworded to start capitalized. Real linter catches fixed:
the two x-planned ops were missing their {workOrderId} path parameter
and any 4xx response (403 added - true today, gateway rejects unsigned).
.redocly.lint-ignore.yaml pins the six deliberate exceptions: webhook
keys are the shipped SHOC contract event names (not renameable), and the
x-planned ops answer only 501 (no 2xx to document).
package.json: npm run lint:api. Verified: lint 0 errors, 675 pytest,
headless-Chrome render of the tagged docs page.
* feat(api): SHOC design-system theme for /docs (vendored fonts)
Themes the Redoc page with the canonical SHOC token set: Montserrat 600
headings / DM Sans body / JetBrains Mono code, primary #1c75bc, navy
#262262 sidebar text + right panel, #f9fafb background, 244px sidebar.
sortRequiredPropsFirst on; 200 responses pre-expanded.
Fonts ship as lambdas/api/fonts.css (latin woff2 subsets from
@fontsource 5.3.0, embedded as data URIs, ~90KB) and inline via a new
__FONTS_CSS__ placeholder with the same </style breakout guard --
the offline single-response invariant holds, nothing fetches Google
Fonts (test-pinned). Bundling cp + bundle-consistency pin + spec-drift
asset checks extended.
Verified: headless-Chrome render (theme + fonts applied), ruff, 675
pytest, cdk synth + staged-asset check.
* feat(api): SHOC gradient topbar on /docs
64px fixed header with the SHOC shell gradient token (#1b1f52 ->
#1c4f8f -> #1c75bc), Sea Haven wordmark in Montserrat 600, page name
right-aligned in DM Sans. Redoc's scrollYOffset: 64 keeps the sticky
sidebar and anchor scrolling clear of the fixed bar. Verified via
headless-Chrome render.
* style(api): normalize /docs header and right-panel blues
The right panel's #262262 is a purple-leaning navy that clashed with
the cyan-leaning gradient, and the bar's brightest point sat directly
over the dark panel. Right panel now uses #1b1f52 (the gradient's own
dark endpoint) and the gradient runs bright-to-dark so its dark end
lands flush on the panel -- no seam, one blue family. Verified via
headless-Chrome render.
* style(api): right-panel gradient on /docs via bundle-pinned override
Redoc's theme only takes solid colors (it derives shades from
rightPanel.backgroundColor), so the gradient (#1b3d79 -> #1b3068 ->
#1b1f52, continuing the topbar blend) rides as a CSS override on the
styled-components classes of the per-section right-panel divs
(.sc-iGgWBj.sc-gsFSXq + the .sc-dExYaf stub). Those names are
deterministic for the vendored 2.5.3 bundle (verified across loads) but
change on any Redoc bump: re-derive via headless probe (find elements
whose computed background equals the rightPanel color). If they stop
matching, the panel falls back to the solid #1b1f52 theme color --
cosmetic only. Verified via headless-Chrome render.
* feat(api): collapsible samples column on /docs
Redoc CE has no built-in panel toggle, so the topbar gains a Hide/Show
samples button that flips .samples-collapsed on <html>: the right-panel
divs hide (same bundle-pinned styled-components classes as the gradient
override) and each section's content half takes the full width. Choice
persists in localStorage; aria-pressed tracks state. If the pinned
classes stop matching after a Redoc bump the toggle goes inert --
cosmetic only. Both states verified via headless-Chrome render.
* ci(api): spec-lint CI gate + npm Dependabot coverage
New spec-lint job mirrors the local npm run lint:api so openapi.json
cannot drift from redocly.yaml with green CI. Dependabot gains the npm
ecosystem (package.json is new; nothing watched @redocly/cli).
* feat(api): docs finishing touches - x-tagGroups, favicon, docs:preview
x-tagGroups sections the Redoc sidebar (Read API / Meta / SHOC Feed);
inline data-URI SVG favicon (SHOC blue) stops the browser's follow-up
/favicon.ico request 403ing at the gateway; npm run docs:preview wraps
the real-handler local render (scripts/preview_docs.py); README gains a
docs-page architecture section covering the inline pattern, theme,
pinned-selector caveat, and tooling. Lint 0 errors, 675 pytest,
headless render verified.
---------
Signed-off-by: Adam Moussa <adam@seahavenind.com>
|
||
|
Adam Moussa
|
073201f633
|
Migrate to seahaven-prod: deploy role, backfill tooling, account-portability fixes (#125)
* feat(migration): prepare stacks and tooling for the seahaven-prod account move
Phase 1 of the mgmt (328440206208) -> seahaven-prod (011934824531)
migration. No behavior change in-account; everything here is additive or
account-portability hygiene:
- infra/deploy-role/: reviewed OIDC deploy-role artifacts for prod
(trust main-only, cdk-hnb659fds-* AssumeRole, smoke-invoke-lambda scoped
to exactly the two email-processor fn ARNs). Codifies the previously
out-of-band smoke-invoke grant.
- Table resource policies: make_slack_bot_read_policy in cdk/common.py,
applied to purchase-orders, verified-sites, WorkOrders,
WorkOrderComments (NOT pending-site-review; no bot consumer). Grants the
mgmt-resident seahaven-slack-bot roles read-only cross-account access
post-move (bot-side identity grants land in the slack-bot repo).
- scripts/migrate_tables.py: dry-run-default backfill tool implementing
the plan's per-table semantics (superset overwrite, ingested_at cutoff
for WorkOrderComments, backup-gated truncate-and-load for the two
site tables) plus a verify subcommand (count parity, spot checks,
sticky-Cancelled drift check).
- tests/test_resource_policy_helper.py: statement-shape unit tests +
static pins that exactly the four bot-read tables carry the policy.
- Account-literal fixes: account-agnostic fixture bucket in
test_reprocess_contract; runbook/README/po-template-parser account
references updated to prod with historical mgmt notes; README gains the
account-prerequisites list (imported-by-name dependencies).
deploy.yaml is deliberately unchanged (push-to-main auto-deploy kept).
Merge is held until migration Phase 0 completes; flipping the
AWS_DEPLOY_ROLE_ARN repo secret and merging this PR IS the first prod
deploy.
* fix(migration): verify backup AVAILABLE pre-truncate; document wildcard risk acceptance (cross-review FIX/NIT)
* refactor(migration): drop cross-account read grants (slack-bot decommissioned); harden backfill + deploy role
seahaven-slack-bot was decommissioned 2026-07-23 (stack DELETE_IN_PROGRESS,
consumer Lambdas gone); its successor sh-mcp is undeployed and uses
same-account DynamoDB access. So no live consumer reads these tables
cross-account. Per Adam's call, drop the cross-account grants entirely and
re-add correctly-scoped ones if/when sh-mcp deploys to a different account.
- Remove the four table resource policies + make_slack_bot_read_policy helper
+ its constants (cdk/common.py, po_stack.py, wo_stack.py) and the helper's
unit test. Both stacks synth with zero table ResourcePolicy.
- scripts/migrate_tables.py hardening (fixes from the sh-security-review
fan-out on the destructive backfill tool):
* validate --cutoff strictly (parse ISO-8601, require aware UTC, re-emit
canonical second-precision form) so a malformed cutoff can't silently
copy dual-window rows or drop history;
* reject `copy --all` up front (must run tables individually, in order,
with the stream-drain wait) instead of writing three tables then erroring;
* truncate backup gate now also checks recency (<1h) and TableId, not just
status+name;
* verify requires --cutoff whenever a cutoff table is in scope (else it
false-flags dual-window rows as MISSING);
* sticky-cancel is now PREVENTED copy-side (a non-Cancelled source item
never overwrites a dest-Cancelled PO), and the verify comment no longer
overstates what its source-side scan covers;
* spot-check all modes (truncate_load keys are verbatim, so key-existence
is sound there too).
- Deploy role: scope cloudformation:DescribeStacks to this repo's stacks +
CDKToolkit (was Resource:*, disclosed all tenant stacks in the shared prod
account); add a drift check warning on unexpected role policies and drop the
dead SMOKE_POLICY_NAME var; document the shared-account bootstrap-role
accepted risk in the deploy-role README.
* docs(deploy-role): fold in cross-review NITs (DescribeStacks maintenance note, warn-only drift rationale)
* ci: update workflow to use new workflow tag (ruff versioning fix)
* fix(migration): address Open SWE review findings on migrate_tables.py
- Validate the truncate backup on dry-run as well as --execute so a
missing/stale/wrong-incarnation --backup-arn surfaces on the rehearsal
run (finding f_24a48b8900).
- Assert configured keys match the live key schema of both tables before
any key projection, turning config/schema drift into a descriptive
abort instead of a mid-backfill KeyError (finding f_cb6b5a6c59).
- Clarify why key-existence spot-checks are sound for WorkOrderComments:
the copy Puts source items verbatim and the sample uses the same
cutoff filter, so per-account comment_id divergence never enters the
check (finding f_390b7d6c3b is a false positive; comment hardened).
|
||
|
Adam Moussa
|
65feb3a4d9
|
chore: add explicit workflow permissions to satisfy least privilege principal. (#119) | ||
|
Adam Moussa
|
ff368beaa4
|
test: consolidate test roots — one loader, shared support, enforced CI floor (phase 8) (#118)
* test: consolidate test roots — one repo-root loader, shared support package, missing-scenario suites, enforced ruff/coverage floor (refactor phase 8) tests/conftest.py only loads for the tests/ root, not a standalone `pytest lambdas/po/email_processor/tests` run, so it could never carry session invariants like the dummy AWS env or the moto stubber registration. Add a single repo-root conftest.py (pytest.ini pins rootdir there, so it loads for every invocation) that sets the dummy AWS credentials/region, imports moto BEFORE any handler module so boto3 sessions pick up its stubber hook (carrying the explanatory comment verbatim from the old _po_parser_support.py), and exposes one load_lambda_module(pipeline, name) — the sys.modules save/restore dance stays, since template_parser is still a duplicated bare name across pipelines needing per-exec sibling binding. Add tests/support/ as the shared package both pipelines' local _*_parser_support.py modules delegate to: a superset FakeTable (PO's update_item recording + WO's put_item and keyed single-row store), FakeDynamoResource, load_email, and load_golden with parse_float=Decimal kept (load-bearing for exact money comparison at PO magnitudes — WO's prior load_golden had no parse_float and must not regress PO by losing it). Rewrite _wo_parser_support.py off the bare `import handler` / `from handler import parse_raw_email` strategy that was the source of the bare-name sys.modules collision the other two loaders defend against. Move test_po_merge.py and test_pad_zip.py into lambdas/po/email_processor/tests/ (PO-specific, belongs beside the code) via git mv so history follows; test_parse_raw_email.py and test_ses_auth.py stay at the repo root since they're genuinely cross-pipeline, parameterized over both handlers. Delete tests/test_local.py: it globs a nonexistent samples/ dir, is WO-only, and imports a handler at collection time, bypassing the loader gate entirely — the golden suites already cover its role. Its pytest.ini exclusion comment goes with it. New scenario coverage, all built on the single loader + support package: - PO+WO Bedrock transport errors (ThrottlingException, missing 'content' key, empty content list, non-JSON model text), asserting PO's pre-call ai_fallback metric survives with no partial write and the exception propagates; WO's no-datapoint-on-throttle behavior is pinned with a documenting test rather than "fixed" by reordering. - Handler-level SES-auth reject seam per pipeline: no auth monkeypatch + empty ALLOWED_DKIM_DOMAINS asserts zero Bedrock calls, zero writes, no raise — closing the hole where deleting the gate line today still passes every test. - web_ui coverage for both PO and WO (0% before this): fail-closed on unset ARN and on a Secrets Manager exception, TTL cache refresh, Bearer/X-Auth-Token/header-case-insensitivity, wrong-token 401 with no table scan, non-ASCII token, and a hostile-field-escaping regression lock. PO web_ui has no __init__.py, so these go through the loader rather than package imports. - A moto-backed mirror of test_po_merge for WO merge semantics (table 'WorkOrders'): null-status never clobbers wo_status, created_at immutable via if_not_exists, status->wo_status mapping, None fields absent from SET, record_type only-when-present. - Small pins: the PO-DC-02 64-char EMF clamp regression and per-pipeline multi-record failure-isolation (all-or-retry contract). The reprocess.py synthetic-event-shape contract test already landed in Phase 7, so it isn't duplicated here. Two WO product-code fixes ride along, since this is the phase that exercises them: (a) the invalid_status reason-code fix in template_parser.py's status check, which previously returned malformed_site_code for the same failure validate_ai_fallback already labels invalid_status, making one failure surface two codes depending on path (grepped the dashboards/metric filters for malformed_site_code first — no external references found, safe to diverge the two codes); (b) wrapping the WO Bedrock call in handler.py so a transport failure emits ai_fallback/bedrock_error in an except-and-reraise. This is deliberately not a naive reorder: the emit sits in the except block, not pre-call, so a gate-rejected email still emits only ai_fallback_rejected and wo_stack's "a rejected email emits nothing else" alarm contract doesn't double-count. A test computes the emitted series by hand to pin the no-double-count behavior. Neither change touches the handler event/return contract. _validate_new_po_values in the PO template_parser.py is split into per-rule helpers, and the V4 anchor-frame dataclass now carries summary_matches/price so V13 can consume them; extract_new_po (C901=35) is included in the split. Add ruff.toml enabling C901/PLR so the mccabe/complexity suppressions scattered through the tree stop being decorative; derived_fields.py is under the shadow-bake freeze so its violations are silenced via a per-file ignore with a justification comment instead of an in-file edit, and the handful of other pre-existing violations surfaced by turning the config on get the same per-file-ignore treatment with a reason, or a fix where the file isn't frozen. scripts/ is added to the CI lint scope. CI gains an explicit --cov module list (lambdas/po and wo email_processor + web_ui, po/site_extractor, lambdas/shared) plus --cov-fail-under=80, since web_ui and site_extractor lack __init__.py markers and a bare --cov=lambdas silently skips them for the missing package marker; .coveragerc omits the test dirs themselves from the count. The Phase 0 AST bundle-consistency test stays in the standard pytest run. .gitignore picks up the resulting .coverage data file. docs/po-template-parser.md gets a small correction: the EXTRACTION_PROMPT declares quantity/price as "number or null", not JSON strings, so parse_float=Decimal already handles a conforming Bedrock response — the doc previously implied the coercion path was the primary mechanism rather than a defensive net for non-conforming responses. * test: lock attribute-context quote escaping in web_ui hostile-field test The escaping regression lock asserted only the element-context vector (raw <script> absent, <script> present) while its docstring claimed quotes were covered -- the payload's " and ' were never asserted on, so a quote-escaping regression on the onclick row-link sink (attribute breakout -> event-handler injection) would have passed green. /sh-security-review finding WC-01 (confirmed medium, test-integrity). Add assertions that the onclick sink's JSON string renders its opening quote as " (raw " after window.location= fails), that the payload's quote characters appear only entity-escaped, and that the raw payload never appears anywhere in the body. Mutation-verified: the test now fails when the sink's quote-escaping is dropped. * test: address Open SWE review — xfail the web_ui non-ASCII auth pin, document subset coverage-floor override - tests/test_web_ui_auth.py: replace the TypeError characterization pin with an xfail(strict, raises=TypeError) asserting the DESIRED fail-closed (False) behavior. Documents the intended fix and auto-fails (xpass) once web_ui_auth is corrected, instead of requiring a passing test to be knowingly deleted. The module stays frozen this phase; the underlying hmac.compare_digest ASCII-only defect is tracked as a follow-up. - pytest.ini: document that the aggregate 80% floor (enforced in CI via the reusable workflow's bare pytest) red-exits local subset runs by design, with the --cov-fail-under=0 override for iteration. Floor stays in addopts because the centralized ci-python-sam workflow exposes no per-run test command. |
||
|
Adam Moussa
|
941a0ec0fe
|
Add pytest suite and enable tests in CI (#95)
Some checks are pending
Deploy / deploy (push) Waiting to run
The repo had no automated tests, so regressions in the email parsing pipeline could only be caught manually. The reusable ci-python-sam workflow already supports a run-tests input at the pinned SHA; enable it and add a first suite covering the pure functions pad_zip and parse_raw_email. tests/conftest.py sets AWS_DEFAULT_REGION and dummy credentials before any handler import because the handlers create boto3 clients at module import time, and loads each handler.py via importlib under a unique module name since the files share a basename. pytest.ini scopes discovery to tests/ so the manual test_local.py script at the repo root is not collected. |
||
|
Adam Moussa
|
1e5c73938e
|
chore(ci): SHA-pin org reusable-workflow caller refs (INFRA-50) (#85)
Some checks are pending
Deploy / deploy (push) Waiting to run
|
||
|
Adam Moussa
|
abdf2aa035
|
Add CI workflow (#18)
* Add CI workflow and apply ruff formatting * Disable cdk synth — email_processor uses pre-built package dir The email_processor Lambda bundles deps into a gitignored package/ directory. cdk synth fails in CI without a build step to recreate it. Disabling until packaging is standardized. * Use CDK BundlingOptions for email_processor Lambda packaging Replaces the pre-built gitignored package/ directory with CDK's built-in bundling. Deps are now installed inside a Docker container during cdk synth, so the build works identically locally and in CI. Re-enables run-cdk-synth in the CI workflow. |