mirror of
https://github.com/Sea-Haven-Industries/procurement-ingest.git
synced 2026-09-30 07:13:13 +00:00
Some checks are pending
Deploy / deploy (push) Waiting to run
Four modules move into the handbook-mandated lambdas/shared/ location, collapsing duplicated logic that had to be kept in sync by hand across the PO and WO pipelines: - ses_auth.py: the PO and WO copies were verified sha256-identical against the feature/phase-7-ops-recovery baseline before the move (no drift since the last audit). shared/ses_auth.py is the exact bytes of that one copy; both originals are git rm'd (the PO copy via rename, the WO copy as a straight delete). Bundling lands the module flat in /asset-output for both email processors, so the handlers keep `from ses_auth import authenticate_inbound_email` unchanged — zero handler diff for this move, which is what keeps fail-closed auth byte-identical through the change. - web_ui_auth.py: extracts the byte-identical _get_auth_token / _header / is_authenticated block plus the four token-cache globals out of both web_ui handlers. The per-stack INFRA-74 comments stay in each handler as-is (deliberately drifted wording, stack-specific) rather than being unified into the shared module. Fail-closed semantics (unset ARN or Secrets Manager exception -> deny) are unchanged. - email_parsing.py: parse_raw_email ships as the superset version that returns cc unconditionally. WO's output is bit-identical to before; PO simply ignores the cc field rather than being "cleaned up" to consume it. No second variant is kept. - emf.py: a generic emitter parameterized by namespace, dimension sets, and properties. Every call site's emitted EMF envelope is unchanged, including the load-bearing [["ParseMethod"],["ParseMethod","TemplateId"]] dimension-set shape the alarms and metric filters depend on. Emission ordering is untouched: PO still emits ai_fallback before the Bedrock call, WO still emits its mutually-exclusive ai_fallback/ai_fallback_rejected after its gate. The deliberate-double-count comments survive. _emit_derived_agreement_metric was found living inside derived_fields.py, so per the DERIVED-FIELDS exception it is left as a third, unconverted copy (derived_fields.py and the shadow DerivedFieldAgreement telemetry stay untouchable while that bake runs) — a comment there points at shared/emf.py for the eventual follow-up. Bundling: both email-processor cdk bundling commands gain a trailing `cp shared/*.py /asset-output/` (they were already cp-only post-Phase 7, so no pip step or manylinux pin is reintroduced). Both web_ui functions gain the same widened-root staging so web_ui_auth.py ships beside their handler; site_extractor's from_asset is untouched. Tests: PO_EXPECTED_TOP_LEVEL_MODULES gains the shared modules that now ship, the AST sibling-import check resolves imports whose source now lives under shared/, and the new shared cp line has its own revert/mutation detection. _SIBLING_MODULES resolution and _po_parser_support.py now load ses_auth/email_parsing/emf from shared/; the two-copy ses_auth byte-identity fixture-hygiene test is retired as obsolete now that there is one copy, and the ses_auth fixture parameterization over two identical copies is dropped. The sys.modules save/restore dance for template_parser (still duplicated per-pipeline) is left in place.
679 lines
38 KiB
JavaScript
679 lines
38 KiB
JavaScript
export const meta = {
|
|
name: 'phase-3-shared-extraction',
|
|
description: 'Phase 3 of the procurement-ingest refactor (docs/refactor-evaluation.md): extract lambdas/shared/ — ses_auth.py (byte-identical move, zero handler diff), web_ui_auth.py (byte-identical auth block from both web_ui handlers), email_parsing.py (parse_raw_email superset, cc unconditional), emf.py (parameterized emitter preserving every envelope byte-for-byte). Bundling gains cp shared/*.py in both email-processor commands + the same staging mechanism for both web_ui functions. Requires Phases 0-2 on the base branch. Auth code moves, so push is gated on /sh-security-review in the main loop. Committed locally, never pushed.',
|
|
phases: [
|
|
{ title: 'Setup', detail: 'verify Phases 0+1+2 on base, branch feature/phase-3-shared-extraction', model: 'haiku' },
|
|
{ title: 'Recon', detail: '4 mappers: the four duplicated modules, post-Phase-2 bundling sites, test-loader plumbing, deployed-zip baseline (read-only AWS)' },
|
|
{ title: 'Spec', detail: 'serial fable spec: pin shared-module contents, handler edit lists, web_ui staging, bundling strings, EMF design, test plumbing' },
|
|
{ title: 'Implement', detail: 'opus: shared/ + four handlers; sonnet: both CDK stacks; opus: all test plumbing + README — disjoint files', model: 'opus' },
|
|
{ title: 'Verify', detail: 'mechanical gates + bundle-parity verifier + 3 fable lenses (auth integrity, EMF/telemetry, loader integrity)' },
|
|
{ title: 'Fix', detail: 'opus fixer, full re-verify, max 3 rounds', model: 'opus' },
|
|
{ title: 'Package', detail: 'single commit via -F (no push)', model: 'sonnet' },
|
|
],
|
|
}
|
|
|
|
// ---------------------------------------------------------------- constants
|
|
|
|
const REPO = '/Users/adammoussa/Documents/repositories/seahaven/procurement-ingest'
|
|
const BRANCH = 'feature/phase-3-shared-extraction'
|
|
let _args = args
|
|
if (typeof _args === 'string') {
|
|
try { _args = JSON.parse(_args) } catch (e) { _args = null }
|
|
}
|
|
const BASE = (_args && _args.base) || 'main'
|
|
|
|
const CONSTRAINTS = `
|
|
PINNED BEHAVIORAL CONSTRAINTS (docs/refactor-evaluation.md Phase 3 — violating any is a build failure):
|
|
1. THE MOVE SET IS EXACTLY FOUR MODULES, in this order of dependency risk:
|
|
lambdas/shared/ses_auth.py, lambdas/shared/web_ui_auth.py,
|
|
lambdas/shared/email_parsing.py, lambdas/shared/emf.py. Nothing else
|
|
moves into shared/. lambdas/shared/ is the handbook location
|
|
(cdk-project-layout.md); modules land FLAT in every bundle so bare-name
|
|
imports keep working.
|
|
2. ses_auth: FIRST verify the two current copies are still byte-identical
|
|
(sha256 both against the ${BASE} versions — any drift since the audit is
|
|
a blocker, not something to silently reconcile). shared/ses_auth.py is
|
|
the EXACT bytes of that single copy; both originals are git rm'd. The
|
|
email-processor handlers keep 'from ses_auth import
|
|
authenticate_inbound_email' UNCHANGED — flat landing in /asset-output
|
|
means ZERO handler diff for this move, which is what keeps fail-closed
|
|
auth byte-identical through the change.
|
|
3. web_ui_auth: extract exactly the byte-identical block — _get_auth_token /
|
|
_header / is_authenticated + the four cache globals — from BOTH web_ui
|
|
handlers. The per-stack INFRA-74 comments STAY in each handler (their
|
|
wording has drifted deliberately; they are stack-specific — do NOT unify
|
|
or move them into the shared module). Fail-closed semantics (unset ARN,
|
|
Secrets Manager exception -> deny) must be unchanged; the token-cache
|
|
globals move with the functions that read them.
|
|
4. email_parsing.py: parse_raw_email as the SUPERSET version returning cc
|
|
unconditionally. WO's output is bit-identical to today; PO simply
|
|
ignores cc — do NOT "clean up" PO to consume it, and do NOT preserve two
|
|
variants.
|
|
5. emf.py: a generic emitter parameterized by namespace / dimension-sets /
|
|
properties. Every call site's emitted EMF envelope must be EXACTLY what
|
|
it emits today — the dimension-set list
|
|
[["ParseMethod"],["ParseMethod","TemplateId"]] is load-bearing for the
|
|
alarms and metric filters; making one-sided dimension fixes impossible
|
|
is the point of this move. Emission ORDERING is untouchable: PO emits
|
|
ai_fallback BEFORE the Bedrock call, WO after its gate with mutually-
|
|
exclusive ai_fallback/ai_fallback_rejected — these two deliberate
|
|
per-pipeline differences are pinned by tests; converting a call site
|
|
must not move it. The deliberate-double-count comments survive.
|
|
6. DERIVED-FIELDS EXCEPTION: if recon finds _emit_derived_agreement_metric
|
|
lives INSIDE derived_fields.py, do NOT convert it — derived_fields.py
|
|
and the shadow DerivedFieldAgreement telemetry are UNTOUCHABLE while the
|
|
bake runs (this outranks the emf consolidation). Leave it as a third
|
|
copy with a code comment pointing at shared/emf.py and report it in
|
|
notes/blockers. Only convert it if it lives outside derived_fields.py.
|
|
7. UNTOUCHABLE FILES (git diff ${BASE}...HEAD must be empty for each):
|
|
both template_parser.py (990 vs 508 lines, genuinely divergent — stays
|
|
per-pipeline), derived_fields.py, both validate_ai_fallback gate
|
|
modules, extract_with_claude's Bedrock invocation/prompt logic. Handler
|
|
diffs are LIMITED to: deleting moved code, import changes, and
|
|
emitter-call swaps per the binding spec. No opportunistic refactors.
|
|
8. Bundling (cdk): append 'cp shared/*.py /asset-output/' to BOTH
|
|
email-processor bundling commands (Phase 2 made the bundling cwd the
|
|
../lambdas asset root, so the path resolves as written).
|
|
BASE-AWARENESS — READ THE ACTUAL cdk FILES ON THE BASE, DO NOT ASSUME:
|
|
this phase is stacked on the Phase 7 branch, which ALREADY removed the
|
|
pip install step from both email-processor commands (they are now
|
|
CP-ONLY: no pip line, no manylinux pin, because nothing third-party is
|
|
installed). Phase 3 moves only pure first-party modules (no new deps),
|
|
so cp-only STAYS cp-only — you simply add another 'cp shared/*.py
|
|
/asset-output/' line. DO NOT re-introduce a pip install step and DO NOT
|
|
re-add the manylinux pin: there is nothing to pin when nothing installs,
|
|
and adding a pip step back would be the regression here. (The PR #34
|
|
lesson — never drop the manylinux pin while a pip install runs — still
|
|
holds ONLY if recon finds a pip step actually present on the base; on
|
|
the cp-only Phase 7 base there is none.) PRESERVE the Phase 2 exclude
|
|
lists exactly. Both web_ui functions gain the SAME staging mechanism
|
|
(widened-root bundled from_asset) so web_ui_auth.py ships beside their
|
|
handler — the spec agent pins the exact form. site_extractor's
|
|
from_asset is UNTOUCHED (Phase 6 territory).
|
|
9. tests/test_bundle_consistency.py is updated IN THE SAME CHANGE without
|
|
losing teeth: the PO_EXPECTED_TOP_LEVEL_MODULES exact-set pin gains the
|
|
shared modules that now ship; the AST sibling-import check must resolve
|
|
imports whose source file now lives under shared/; the new shared cp
|
|
line gets its own revert/mutation detection (a commented-out
|
|
'cp shared/*.py' must fail the test).
|
|
10. Test plumbing in the same PR: update _SIBLING_MODULES resolution and
|
|
_po_parser_support.py (~line 71 — verify the current line) so tests
|
|
load ses_auth/email_parsing/emf from shared/; retire or repoint the
|
|
fixture-hygiene test that polices the two ses_auth copies for
|
|
byte-identity (obsolete once there is one copy — do not leave it
|
|
failing); drop the ses_auth fixture params from test_ses_auth
|
|
(parameterizing over two identical copies is dead weight — halves the
|
|
run). The sys.modules save/restore dance SURVIVES for template_parser
|
|
(still a duplicated bare name) — do not delete it. Preserve the
|
|
load-bearing moto-before-handler import ordering.
|
|
11. STALE-SHADOW HAZARD: after the move, no stale ses_auth.py / .pyc /
|
|
__pycache__ copy may remain anywhere it could shadow the shared copy —
|
|
in the repo (git rm, don't empty), in staged assets, or on any test
|
|
sys.path. Verify explicitly.
|
|
12. cdk diff on BOTH stacks: the only resource deltas allowed are Code/
|
|
S3Key (asset) changes on the two email processors and the two web_ui
|
|
functions (+ CDK metadata). No logical-ID changes, no alarm, IAM,
|
|
table, env, runtime, or handler-property deltas of any kind.
|
|
13. AWS access is READ-ONLY (get-function, downloading deployed zips via
|
|
presigned URLs). NEVER cdk deploy, never invoke, never mutate.
|
|
`
|
|
|
|
const PREAMBLE = `
|
|
You are one of several agents building refactor Phase 3 in the git repo at
|
|
${REPO} on branch ${BRANCH} (already checked out — do NOT switch branches,
|
|
do NOT create branches, do NOT commit, NEVER push, do NOT run cdk deploy or
|
|
touch AWS resources beyond read-only calls).
|
|
Authoritative spec: docs/refactor-evaluation.md, section "Phase 3".
|
|
Work ONLY in the files you are told you own; other agents are concurrently
|
|
editing other files in this same working tree.
|
|
${CONSTRAINTS}
|
|
Your final message is consumed by an orchestrator script, not a human —
|
|
return only the structured data requested.
|
|
`
|
|
|
|
// ------------------------------------------------------------------ schemas
|
|
|
|
const RECON = {
|
|
type: 'object',
|
|
required: ['summary', 'facts'],
|
|
properties: {
|
|
summary: { type: 'string' },
|
|
facts: { type: 'array', items: { type: 'string' } },
|
|
blockers: { type: 'array', items: { type: 'string' } },
|
|
},
|
|
}
|
|
|
|
const SPEC = {
|
|
type: 'object',
|
|
required: ['sharedModules', 'handlerEdits', 'webUiStaging', 'bundlingEdits', 'emfDesign', 'testPlumbing', 'parityRules', 'notes'],
|
|
properties: {
|
|
sharedModules: { type: 'string', description: 'per shared module: exact provenance (which copy is the source, sha256), full contents decision (verbatim move vs superset vs parameterized), and the public surface each importer uses' },
|
|
handlerEdits: { type: 'string', description: 'per handler (po/wo email_processor, po/wo web_ui): the exact deletions, import lines, and emitter-call swaps — file:line, nothing else may change' },
|
|
webUiStaging: { type: 'string', description: 'the complete new from_asset blocks for both web_ui functions: asset path, command or cp form, exclude list — exact code' },
|
|
bundlingEdits: { type: 'string', description: 'the two email-processor bundling command strings with cp shared/*.py appended, verbatim, plus the expected top-level module set of each resulting bundle' },
|
|
emfDesign: { type: 'string', description: 'shared/emf.py signature + per-call-site mapping proving each emitted envelope (namespace, dimension-set list, properties) is byte-equivalent to today; the derived-agreement emitter decision per constraint 6' },
|
|
testPlumbing: { type: 'string', description: 'every test/support file change: loader resolution, _SIBLING_MODULES, fixture-hygiene retirement, test_ses_auth de-parameterization, test_bundle_consistency edits with their mutation-detection shapes' },
|
|
parityRules: { type: 'string', description: 'how Verify judges staged bundles vs the deployed-zip baseline: expected first-party set per function (= deployed set + the new shared modules), ses_auth byte-identity requirement, expected removals (none beyond moved-file provenance), web_ui bundle expectations' },
|
|
notes: { type: 'string' },
|
|
},
|
|
}
|
|
|
|
const IMPL = {
|
|
type: 'object',
|
|
required: ['filesChanged', 'summary', 'checksRun'],
|
|
properties: {
|
|
filesChanged: { type: 'array', items: { type: 'string' } },
|
|
summary: { type: 'string' },
|
|
checksRun: { type: 'string' },
|
|
blockers: { type: 'array', items: { type: 'string' } },
|
|
},
|
|
}
|
|
|
|
const CHECKS = {
|
|
type: 'object',
|
|
required: ['passed', 'details'],
|
|
properties: {
|
|
passed: { type: 'boolean' },
|
|
details: { type: 'string' },
|
|
scopeViolations: { type: 'array', items: { type: 'string' } },
|
|
},
|
|
}
|
|
|
|
const PARITY = {
|
|
type: 'object',
|
|
required: ['passed', 'poVerdict', 'woVerdict', 'webUiVerdict', 'details'],
|
|
properties: {
|
|
passed: { type: 'boolean' },
|
|
poVerdict: { type: 'string', description: 'PO email-processor staged vs deployed: module-set delta exactly as spec, ses_auth bytes identical — full evidence' },
|
|
woVerdict: { type: 'string', description: 'same for WO email-processor' },
|
|
webUiVerdict: { type: 'string', description: 'both web_ui staged bundles: own *.py + web_ui_auth.py present, nothing stray, hashes deterministic' },
|
|
details: { type: 'string' },
|
|
},
|
|
}
|
|
|
|
const FINDINGS = {
|
|
type: 'object',
|
|
required: ['findings'],
|
|
properties: {
|
|
findings: {
|
|
type: 'array',
|
|
items: {
|
|
type: 'object',
|
|
required: ['title', 'severity', 'confirmed', 'evidence', 'fix'],
|
|
properties: {
|
|
title: { type: 'string' },
|
|
severity: { enum: ['critical', 'high', 'medium', 'low'] },
|
|
confirmed: { type: 'boolean' },
|
|
evidence: { type: 'string' },
|
|
fix: { type: 'string' },
|
|
},
|
|
},
|
|
},
|
|
},
|
|
}
|
|
|
|
// ------------------------------------------------------------------- setup
|
|
|
|
phase('Setup')
|
|
const setup = await agent(`
|
|
In ${REPO}:
|
|
1. SEQUENCING GATE — Phases 0, 1 AND 2 must all be on ${BASE} (Phase 3
|
|
edits the same stack files as Phase 2 and depends on its widened
|
|
../lambdas asset roots to make shared/ reachable). git fetch origin,
|
|
then pick the base ref: origin/${BASE} if that remote ref exists,
|
|
otherwise the local branch ${BASE} (a stacked local-only base is
|
|
expected and fine). Verify on the base ref:
|
|
(a) Phase 0: tests/test_bundle_consistency.py exists
|
|
(git show <baseref>:tests/test_bundle_consistency.py | head -3);
|
|
(b) Phase 1: validate_ai_fallback exists in the PO pipeline
|
|
(git grep validate_ai_fallback <baseref> -- lambdas/po);
|
|
(c) Phase 2: BOTH cdk/po_stack.py and cdk/wo_stack.py on the base ref
|
|
contain Code.from_asset("../lambdas") for the email processors
|
|
(git show <baseref>:cdk/po_stack.py | grep -n '\\.\\./lambdas', same
|
|
for wo_stack.py).
|
|
If any is missing, STOP with a blocker naming the unmet phase and do
|
|
nothing else.
|
|
2. Verify clean working tree (untracked .coverage / .claude/ / the local
|
|
44 MB lambdas/po/email_processor/package/ dir are fine; any OTHER dirt =
|
|
blocker, never stash or discard).
|
|
3. git checkout ${BASE}; then git pull --ff-only ONLY if the branch has an
|
|
upstream (a local-only base skips the pull — not a blocker); then
|
|
git checkout -b ${BRANCH}
|
|
4. gh pr list --state open --json number,title,headRefName (overlap check).
|
|
Return facts: HEAD sha, per-phase gate evidence, open PRs, blockers.
|
|
`, { label: 'setup:branch', model: 'haiku', schema: RECON })
|
|
|
|
if (!setup || (setup.blockers && setup.blockers.length)) {
|
|
return { aborted: 'setup blockers', blockers: setup ? setup.blockers : ['setup agent died'], facts: setup ? setup.facts : [] }
|
|
}
|
|
log(`Branch ${BRANCH} ready off ${BASE}. ${setup.summary}`)
|
|
|
|
// ------------------------------------------------------------------- recon
|
|
|
|
phase('Recon')
|
|
const recon = await parallel([
|
|
() => agent(`${PREAMBLE}
|
|
Read-only recon of the FOUR duplicated surfaces being extracted:
|
|
1. ses_auth.py both copies — sha256 of each (MUST match; drift = blocker),
|
|
line count, the exact import line each email-processor handler uses,
|
|
any other importer (git grep 'import ses_auth\\|from ses_auth').
|
|
2. web_ui auth block — in BOTH web_ui handlers quote with file:line the
|
|
exact boundaries of the byte-identical block (_get_auth_token, _header,
|
|
is_authenticated, the four cache globals), diff the two blocks to prove
|
|
byte-identity, quote each INFRA-74 comment verbatim (they differ —
|
|
that is expected), and note everything else in each handler that CALLS
|
|
the block.
|
|
3. parse_raw_email both copies — where each lives (own sibling file vs
|
|
inside handler.py), the exact cc delta between them, every call site.
|
|
4. The THREE EMF emitters — quote each verbatim with file:line (PO
|
|
ParseMethod emitter, WO ParseMethod emitter,
|
|
_emit_derived_agreement_metric), namespace + dimension-set list +
|
|
properties of each, and CRITICALLY: which FILE _emit_derived_agreement_metric
|
|
lives in (constraint 6 hinges on whether it is inside derived_fields.py).
|
|
Also pin current line numbers of PO's pre-Bedrock ai_fallback emit and
|
|
WO's post-gate emit.
|
|
25-35 precise facts.`,
|
|
{ label: 'recon:duplicated-modules', model: 'sonnet', phase: 'Recon', schema: RECON }),
|
|
|
|
() => agent(`${PREAMBLE}
|
|
Read-only recon of the post-Phase-2 CDK bundling state: for ALL FIVE
|
|
Code.from_asset sites in cdk/po_stack.py and cdk/wo_stack.py quote verbatim
|
|
with current file:line — asset path, full bundling command (or plain form),
|
|
exclude list. For the two web_ui functions additionally record: runtime,
|
|
architecture, handler property, memory/timeout, whether any requirements.txt
|
|
exists for them, and every function property that must NOT change when
|
|
bundling is added. Confirm lambdas/shared/ does not exist yet and list
|
|
anything at lambdas/ top level. 15-25 facts.`,
|
|
{ label: 'recon:cdk-bundling', model: 'haiku', phase: 'Recon', schema: RECON }),
|
|
|
|
() => agent(`${PREAMBLE}
|
|
Read-only recon of the test plumbing this phase must rewire:
|
|
- tests/conftest.py importlib loader: how it resolves module paths, the
|
|
sys.modules save/restore, _SIBLING_MODULES (exact current contents).
|
|
- _po_parser_support.py: the independent importlib reimplementation, the
|
|
load-bearing moto-before-handler import order (~lines 26-33), and what
|
|
is at line ~71 (the doc cites it — quote the current code).
|
|
- _wo_parser_support.py bare sys.path 'import handler' strategy.
|
|
- test_ses_auth.py at root: the fixture parameterization over both copies
|
|
(quote it), total line count, the fixture-hygiene test that polices
|
|
byte-identity between the two copies (name + assertion).
|
|
- test_parse_raw_email.py: how it loads both handlers' parse_raw_email.
|
|
- tests/test_bundle_consistency.py: every assertion that will fail red
|
|
against the Phase 3 shapes (shared cp line, PO_EXPECTED_TOP_LEVEL_MODULES,
|
|
AST sibling-import resolution for moved modules).
|
|
20-30 facts.`,
|
|
{ label: 'recon:test-plumbing', model: 'sonnet', phase: 'Recon', schema: RECON }),
|
|
|
|
() => agent(`${PREAMBLE}
|
|
Read-only AWS recon (region us-east-1, READ-ONLY): for po-email-processor,
|
|
workorder-email-processor, AND both web_ui functions (find their exact
|
|
function names via the stacks/aws lambda list-functions) run aws lambda
|
|
get-function; for the two email processors download each Code.Location
|
|
presigned zip to a scratch dir and produce the COMPLETE file list
|
|
(unzip -l) separated into (a) first-party top-level .py, (b) dependency
|
|
dirs, (c) anything else; record every function's CodeSha256 and the
|
|
sha256 of ses_auth.py inside each deployed zip (the byte-identity baseline
|
|
the moved copy is judged against). For the web_ui functions the current
|
|
zip file list is the baseline their new bundled asset must cover.
|
|
Return the categorized lists as facts.`,
|
|
{ label: 'recon:deployed-baseline', model: 'sonnet', phase: 'Recon', schema: RECON }),
|
|
])
|
|
|
|
const reconOk = recon.filter(Boolean)
|
|
const pack = reconOk.map(r => `## ${r.summary}\n${r.facts.join('\n')}`).join('\n\n')
|
|
const reconBlockers = reconOk.flatMap(r => r.blockers || [])
|
|
.filter(b => b && !/^\s*(none|n\/a)\b/i.test(b))
|
|
log(`Recon complete: ${reconOk.length}/4 mappers, ${reconBlockers.length} blockers`)
|
|
if (reconBlockers.length) {
|
|
return { aborted: 'recon blockers (likely copy drift — resolve before extracting)', blockers: reconBlockers, reconPack: pack }
|
|
}
|
|
|
|
// -------------------------------------------------------------------- spec
|
|
|
|
phase('Spec')
|
|
const spec = await agent(`${PREAMBLE}
|
|
You are the SPEC agent — the single authority that pins every contested
|
|
decision BEFORE parallel implementation (parallel leaves cannot see each
|
|
other's choices). Using the recon pack below plus your own reads of the
|
|
actual files, produce the binding implementation spec:
|
|
- sharedModules: per constraint 1's four modules — provenance, contents
|
|
decision, public surface. ses_auth is a verbatim byte-move; web_ui_auth
|
|
is the exact block; email_parsing is the WO superset (cc unconditional);
|
|
emf is the parameterized emitter.
|
|
- handlerEdits: for each of the four handlers, the exact minimal edit list
|
|
(constraint 7 — deletions, imports, emitter-call swaps ONLY). State
|
|
explicitly that the two email-processor ses_auth import lines are
|
|
UNCHANGED.
|
|
- webUiStaging: complete replacement from_asset blocks for both web_ui
|
|
functions — widened root, cp command staging the function's own *.py
|
|
plus web_ui_auth.py (pin whether to cp shared/*.py or just
|
|
shared/web_ui_auth.py — pick ONE rule, state why), Phase-2-style
|
|
excludes. Mind bundling cwd semantics: the container mounts the asset
|
|
root (../lambdas) as the working dir. If a no-Docker staging form is
|
|
viable and simpler, you may pin that instead — but ONE mechanism for
|
|
both, exact code.
|
|
- bundlingEdits: both email-processor command strings verbatim with
|
|
'cp shared/*.py /asset-output/' appended, plus the exact expected
|
|
top-level module set of each resulting bundle (this feeds
|
|
PO_EXPECTED_TOP_LEVEL_MODULES and the parity gate).
|
|
- emfDesign: the shared emitter signature and a per-call-site table
|
|
proving envelope byte-equivalence; resolve the derived-agreement emitter
|
|
per constraint 6 based on where recon found it.
|
|
- testPlumbing: every file, every edit — loader resolution for shared/,
|
|
_SIBLING_MODULES, _po_parser_support.py, _wo_parser_support.py if it
|
|
needs the shared path, fixture-hygiene retirement, test_ses_auth
|
|
de-parameterization, test_parse_raw_email (single implementation now —
|
|
what does it exercise?), test_bundle_consistency edits including the new
|
|
mutation shapes (commented-out shared cp must fail).
|
|
- parityRules: exactly how Verify judges staged bundles vs recon's
|
|
deployed baseline — per-function expected first-party set (deployed set
|
|
minus nothing, plus the shared modules per your cp rule), ses_auth
|
|
sha256 must equal the deployed zips' copy, web_ui bundles must cover
|
|
their deployed file list plus web_ui_auth.py, nothing else may appear
|
|
or vanish.
|
|
Recon pack:\n${pack}`,
|
|
{ label: 'spec:pin-extraction', phase: 'Spec', schema: SPEC })
|
|
|
|
if (!spec) return { aborted: 'spec agent died — rerun workflow', reconBlockers }
|
|
const specBlock = `BINDING SPEC (from the spec agent — implement EXACTLY this):\n${JSON.stringify(spec, null, 2)}`
|
|
log('Spec pinned: shared modules, handler edits, web_ui staging, bundling strings, EMF design, test plumbing')
|
|
|
|
// --------------------------------------------------------------- implement
|
|
|
|
phase('Implement')
|
|
const impl = await parallel([
|
|
() => agent(`${PREAMBLE}
|
|
YOU OWN: everything under lambdas/ EXCEPT the tests/ directories (another
|
|
agent owns all test and support files). Do not touch cdk/ or README.
|
|
Task: execute the four moves per spec.sharedModules + spec.handlerEdits +
|
|
spec.emfDesign, in the spec's order:
|
|
1. git mv (or create+git rm preserving exact bytes) ses_auth.py ->
|
|
lambdas/shared/ses_auth.py; delete both originals; prove sha256
|
|
equality in your summary. Handler import lines untouched.
|
|
2. Create lambdas/shared/web_ui_auth.py from the byte-identical block;
|
|
replace the block in BOTH web_ui handlers with the import; keep each
|
|
INFRA-74 comment in place.
|
|
3. Create lambdas/shared/email_parsing.py (superset); rewire both
|
|
email-processor call sites.
|
|
4. Create lambdas/shared/emf.py; swap the emitter call sites per
|
|
spec.emfDesign — honoring constraint 6 (derived_fields.py stays
|
|
untouched if the third emitter lives there) and constraint 5 (emission
|
|
ordering does not move).
|
|
Delete every now-empty moved-out file with git rm (constraint 11).
|
|
Run before returning: ruff check lambdas && ruff format lambdas --check,
|
|
plus an import smoke: python3 -c with sys.path prepended for
|
|
lambdas/shared + each function dir, importing every touched module (tests
|
|
are NOT yours to run — the plumbing agent lands them in parallel).
|
|
${specBlock}`,
|
|
{ label: 'impl:shared-lambdas', model: 'opus', phase: 'Implement', schema: IMPL }),
|
|
|
|
() => agent(`${PREAMBLE}
|
|
YOU OWN: cdk/po_stack.py and cdk/wo_stack.py ONLY (and only the from_asset
|
|
regions — alarms, IAM, tables, env are all off-limits).
|
|
Task: apply spec.bundlingEdits (append the shared cp to both
|
|
email-processor commands — the base is the Phase 7 CP-ONLY bundling, so
|
|
just add another 'cp shared/*.py /asset-output/' line; DO NOT re-add a pip
|
|
install step or a manylinux pin, there is nothing third-party to install
|
|
(constraint 8); preserve the Phase 2 excludes verbatim) and
|
|
spec.webUiStaging (both web_ui functions). Touch nothing else;
|
|
site_extractor's from_asset stays byte-identical.
|
|
Run before returning: ruff check cdk && cd cdk &&
|
|
npx cdk synth po-ingest -q -o /tmp/phase3-synth &&
|
|
npx cdk synth workorder-ingest -q -o /tmp/phase3-synth (artifact-id
|
|
selectors, NOT stack_name). Docker bundling runs — confirm each staged
|
|
email-processor asset contains the shared modules and each staged web_ui
|
|
asset contains web_ui_auth.py; note asset hashes in your summary. If the
|
|
lambdas/ moves have not landed yet the synth will fail on missing files —
|
|
poll by re-running up to ~10 min before reporting a blocker.
|
|
${specBlock}`,
|
|
{ label: 'impl:cdk-stacks', model: 'sonnet', phase: 'Implement', schema: IMPL }),
|
|
|
|
() => agent(`${PREAMBLE}
|
|
YOU OWN: all test and support files (tests/ at repo root including
|
|
tests/test_bundle_consistency.py and tests/conftest.py,
|
|
lambdas/po/email_processor/tests/, lambdas/wo/email_processor/tests/)
|
|
and README.md ONLY.
|
|
Task A: apply spec.testPlumbing in full — loader/_SIBLING_MODULES
|
|
resolution for shared/, _po_parser_support.py edit (preserving the
|
|
moto-before-handler ordering comment), fixture-hygiene retirement,
|
|
test_ses_auth de-parameterization, test_parse_raw_email update,
|
|
test_bundle_consistency updates WITHOUT losing teeth (constraint 9 — the
|
|
new shared cp pin must reject a commented-out or narrowed variant; keep
|
|
the existing mutation tests green).
|
|
Task B: README — document lambdas/shared/ in the repo-layout section
|
|
(which modules live there, the flat-landing import rule, the bundling cp
|
|
that ships them), update the bundling/deploy-guards paragraphs for the
|
|
shared cp + web_ui staging, and note that ses_auth is now single-sourced
|
|
(one hardening fix lands once). Match existing README style.
|
|
Run before returning: pytest -q --no-cov at repo root (must be green
|
|
against the OTHER agents' edits — they land in parallel; poll by
|
|
re-running up to ~10 min before reporting a blocker) and ruff check on
|
|
every file you touched.
|
|
${specBlock}`,
|
|
{ label: 'impl:test-plumbing-readme', model: 'opus', phase: 'Implement', schema: IMPL }),
|
|
])
|
|
|
|
const implOk = impl.filter(Boolean)
|
|
const implBlockers = implOk.flatMap(r => r.blockers || [])
|
|
log(`Implement complete: ${implOk.length}/3 agents, blockers: ${implBlockers.length}`)
|
|
|
|
// ---------------------------------------------------- verify + fix loop
|
|
|
|
const EXPECTED_SCOPE = [
|
|
'lambdas/shared/',
|
|
'lambdas/po/email_processor/',
|
|
'lambdas/wo/email_processor/',
|
|
'lambdas/po/web_ui/',
|
|
'lambdas/wo/web_ui/',
|
|
'cdk/po_stack.py',
|
|
'cdk/wo_stack.py',
|
|
'tests/',
|
|
'README.md',
|
|
]
|
|
|
|
const mechanicalPrompt = `${PREAMBLE}
|
|
Independent re-verification — trust nothing self-reported. Run ALL gates,
|
|
quoting failures verbatim:
|
|
1. pytest -q --no-cov (repo root, all three roots green)
|
|
2. ruff check . && ruff format --check .
|
|
3. cd cdk && npx cdk synth po-ingest -q && npx cdk synth workorder-ingest -q
|
|
4. cd cdk && npx cdk diff po-ingest ; npx cdk diff workorder-ingest —
|
|
constraint 12: only the four functions' Code/S3Key (+ metadata) deltas.
|
|
Any alarm/IAM/env/runtime/handler-prop/logical-ID delta = FAIL. Paste
|
|
the diff summaries.
|
|
5. UNTOUCHABLES (each must output NOTHING):
|
|
git diff ${BASE}...HEAD -- lambdas/po/email_processor/derived_fields.py
|
|
git diff ${BASE}...HEAD -- lambdas/po/email_processor/template_parser.py
|
|
git diff ${BASE}...HEAD -- lambdas/wo/email_processor/template_parser.py
|
|
plus both validate_ai_fallback gate modules (resolve their filenames
|
|
first) and lambdas/po/site_extractor/.
|
|
6. BYTE-IDENTITY: sha256 of lambdas/shared/ses_auth.py equals sha256 of
|
|
git show ${BASE}:lambdas/po/email_processor/ses_auth.py (and the wo
|
|
copy). Both originals gone from the tree (git ls-files check) and no
|
|
stray ses_auth.py/__pycache__ anywhere under lambdas/ outside shared/
|
|
(constraint 11).
|
|
7. ORDERING PINS: grep line numbers proving PO's ai_fallback emit still
|
|
precedes the Bedrock invoke and WO's emits are untouched relative to
|
|
${BASE} (the wo handler diff must contain ONLY the spec's edit classes).
|
|
8. INFRA-74: both web_ui handlers still contain their own INFRA-74
|
|
comment verbatim per ${BASE}.
|
|
9. git status --porcelain scope check: every modified/added/deleted path
|
|
under ${EXPECTED_SCOPE.join(', ')} (untracked .coverage/.claude/
|
|
package/ tolerated).
|
|
passed=true only if all green. YOU MAY NOT edit files.`
|
|
|
|
const parityPrompt = `${PREAMBLE}
|
|
You are the BUNDLE-PARITY verifier — the load-bearing gate of this phase.
|
|
Everything is local synth + read-only AWS.
|
|
1. cd cdk && npx cdk synth po-ingest -q -o /tmp/phase3-parity &&
|
|
npx cdk synth workorder-ingest -q -o /tmp/phase3-parity. Locate all
|
|
four staged assets (two email processors, two web_ui).
|
|
2. Re-download both email-processor deployed zips fresh (aws lambda
|
|
get-function Code.Location, us-east-1) — do not trust a recon cache —
|
|
and fetch both web_ui functions' deployed file lists.
|
|
3. Judge per spec.parityRules: per email processor, staged first-party
|
|
top-level .py set == deployed set PLUS exactly the shared modules the
|
|
spec's cp rule ships (list both sets; nothing else may appear or
|
|
vanish); sha256 of staged ses_auth.py == sha256 of the ses_auth.py
|
|
inside each deployed zip (fail-closed auth byte-identical through the
|
|
move); dependency packages compared by name, version drift noted not
|
|
failed. Per web_ui function: staged bundle covers the deployed file
|
|
list plus web_ui_auth.py, nothing stray (no tests/, no other
|
|
pipeline's sources, no .eml, no package/).
|
|
4. DETERMINISM: synth po-ingest twice into fresh -o dirs — identical
|
|
asset hashes, including the NEW web_ui assets. Then drop a throwaway
|
|
__pycache__/junk.pyc under lambdas/shared/ (delete it afterwards),
|
|
re-synth, and confirm the excludes keep every hash unchanged.
|
|
5. IMPORT-RESOLUTION sanity: inside each staged email-processor asset
|
|
dir run python3 -c "import handler" with that dir alone on sys.path
|
|
(env-var stubs as needed) — proves the flat landing satisfies every
|
|
import including 'from ses_auth import ...' with the moved copy.
|
|
passed=true only if every check holds.`
|
|
|
|
const lenses = [
|
|
{ key: 'auth-integrity', prompt: `${PREAMBLE}
|
|
ADVERSARIAL REVIEW — auth-integrity lens. Auth code moved; try to prove
|
|
the move WEAKENED it. (1) ses_auth: byte-compare the shared copy against
|
|
${BASE}'s copies yourself; then attack import resolution — in the BUNDLE
|
|
and in TESTS, which ses_auth wins if anything shadows (stale .pyc, a
|
|
same-named module on sys.path, the tests loader resolving the old path
|
|
silently to a stub)? Could a test now pass against a MOCK of ses_auth
|
|
where it previously exercised the real module? (2) web_ui_auth: diff the
|
|
extracted block against both originals — any dropped line, changed
|
|
global, or reordered check? Is the fail-closed path (unset ARN, Secrets
|
|
exception, wrong token -> 401 before any table access) provably
|
|
unchanged? Do the four cache globals still behave per-function (module
|
|
now shared — could cross-importer state ever leak)? (3) The gate call
|
|
sites: could any handler path now reach S3-fetch/Bedrock/save before
|
|
authenticate_inbound_email or is_authenticated, where it could not
|
|
before? confirmed=true only with a concrete exploit sketch or
|
|
file:line proof.` },
|
|
{ key: 'telemetry-emf', prompt: `${PREAMBLE}
|
|
ADVERSARIAL REVIEW — EMF/telemetry lens. The alarms and metric filters
|
|
consume exact EMF shapes; a silent envelope change breaks paging without
|
|
failing any test. Read shared/emf.py + every converted call site
|
|
(git diff ${BASE}) and the synthesized templates' metric filters/alarms.
|
|
Verify per call site: namespace exact, dimension-set list EXACT
|
|
([["ParseMethod"],["ParseMethod","TemplateId"]] where applicable, order
|
|
included), property keys/types identical, timestamp/CloudWatchMetrics
|
|
envelope structure identical — construct a sample emission per call site
|
|
and diff it against ${BASE}'s hand-built _aws dict output. Then the
|
|
ordering pins: PO ai_fallback still pre-Bedrock, WO still post-gate
|
|
mutually-exclusive, the deliberate double-count comment intact. Finally
|
|
constraint 6: where does _emit_derived_agreement_metric live and was the
|
|
decision honored (derived_fields.py diff empty)? confirmed=true only
|
|
with evidence.` },
|
|
{ key: 'loader-integrity', prompt: `${PREAMBLE}
|
|
ADVERSARIAL REVIEW — test/loader-integrity lens. The three loading idioms
|
|
were rewired; attack them. (1) Does test_ses_auth now exercise the REAL
|
|
lambdas/shared/ses_auth.py (trace the loader path), and did
|
|
de-parameterization silently drop any assertion that only ran under one
|
|
param? (2) Fixture-hygiene: the byte-identity police is retired — does
|
|
anything still guard against a future stray ses_auth.py copy reappearing
|
|
in a pipeline dir (should the bundle-consistency or a hygiene test)? If
|
|
nothing does, that is a finding. (3) sys.modules save/restore: prove it
|
|
still isolates template_parser between pipelines (run two cross-pipeline
|
|
tests back to back); prove moto-before-handler ordering survived in
|
|
_po_parser_support.py. (4) test_bundle_consistency: hand-mutate command
|
|
strings on scratch copies — a commented-out 'cp shared/*.py', a shared
|
|
glob narrowed to one file, and a PO_EXPECTED_TOP_LEVEL_MODULES missing a
|
|
shared module must each FAIL. (5) Run pytest twice in one session and in
|
|
file-shuffled order (-p no:randomly not installed? then two explicit
|
|
orderings) to smoke out import-order coupling introduced by the shared
|
|
path. confirmed=true only with file:line or reproduced-failure
|
|
evidence.` },
|
|
]
|
|
|
|
let round = 0
|
|
let checks = null
|
|
let parity = null
|
|
let confirmed = []
|
|
while (round < 3) {
|
|
phase('Verify')
|
|
const results = await parallel([
|
|
() => agent(mechanicalPrompt, { label: `verify:mechanical-r${round}`, model: 'sonnet', phase: 'Verify', schema: CHECKS }),
|
|
() => agent(parityPrompt, { label: `verify:parity-r${round}`, model: 'opus', phase: 'Verify', schema: PARITY }),
|
|
...lenses.map(l => () =>
|
|
agent(l.prompt, { label: `verify:${l.key}-r${round}`, phase: 'Verify', schema: FINDINGS })),
|
|
])
|
|
checks = results[0]
|
|
parity = results[1]
|
|
confirmed = results.slice(2).filter(Boolean)
|
|
.flatMap(r => r.findings || [])
|
|
.filter(f => f.confirmed && f.severity !== 'low')
|
|
const green = checks && checks.passed && parity && parity.passed
|
|
log(`Verify round ${round}: mechanical ${checks && checks.passed ? 'GREEN' : 'RED'}, parity ${parity && parity.passed ? 'GREEN' : 'RED'}, confirmed findings: ${confirmed.length}`)
|
|
if (green && confirmed.length === 0) break
|
|
|
|
round += 1
|
|
if (round >= 3) break
|
|
phase('Fix')
|
|
await agent(`${PREAMBLE}
|
|
You are the fix agent — you may edit files under: ${EXPECTED_SCOPE.join(', ')}.
|
|
Fix EVERY item below minimally; the binding spec and 13 pinned constraints
|
|
still hold (a finding that conflicts with a constraint is reported, not
|
|
"fixed" — the constraint wins, esp. constraint 6's derived_fields
|
|
untouchability and constraint 5's emission ordering). Re-run the specific
|
|
failing gate/test per fix.
|
|
MECHANICAL:\n${checks ? checks.details : '(agent died — rerun all gates)'}
|
|
PARITY:\n${parity ? parity.details : '(agent died — rerun all)'}
|
|
CONFIRMED FINDINGS:\n${JSON.stringify(confirmed, null, 2)}
|
|
${specBlock}`,
|
|
{ label: `fix:round-${round}`, model: 'opus', phase: 'Fix', schema: IMPL })
|
|
}
|
|
|
|
const verifyClean = checks && checks.passed && parity && parity.passed && confirmed.length === 0
|
|
if (!verifyClean) {
|
|
return {
|
|
status: 'NEEDS ATTENTION — verify not clean after 3 rounds; branch left uncommitted',
|
|
branch: BRANCH,
|
|
mechanical: checks,
|
|
parity,
|
|
unresolvedFindings: confirmed,
|
|
implBlockers,
|
|
reconBlockers,
|
|
spec,
|
|
}
|
|
}
|
|
|
|
// ----------------------------------------------------------------- package
|
|
|
|
phase('Package')
|
|
const commit = await agent(`${PREAMBLE.replace('do NOT commit, ', '')}
|
|
YOU are the commit agent:
|
|
1. Read ~/Documents/repositories/seahaven/engineering-handbook/commit-messages.md
|
|
and follow it exactly.
|
|
2. git add only paths under: ${EXPECTED_SCOPE.join(', ')} and
|
|
.claude/workflows/phase-3-shared-extraction.js. NOT .coverage, NOT
|
|
package/. Verify the staged set with git status — the deletions of the
|
|
moved originals MUST be staged too.
|
|
3. ONE commit; write the message to /tmp/phase3-commit-msg.txt and use
|
|
git commit -F /tmp/phase3-commit-msg.txt (backticks in -m get eaten by
|
|
zsh). Suggested subject:
|
|
"feat: extract lambdas/shared/ — single-source ses_auth, web_ui auth, email parsing, EMF emitter (refactor phase 3)"
|
|
Body: the four moves with the byte-identity evidence one-liner for
|
|
ses_auth, the flat-landing import rule, the shared cp bundling change +
|
|
web_ui staging, the derived-agreement emitter decision (constraint 6),
|
|
and the test-plumbing summary. NO AI attribution / Co-Authored-By
|
|
lines.
|
|
4. Do NOT push. Return commit sha + shortstat in summary.`,
|
|
{ label: 'package:commit', model: 'sonnet', phase: 'Package', schema: IMPL })
|
|
|
|
return {
|
|
status: 'BUILT — committed locally, NOT pushed',
|
|
branch: BRANCH,
|
|
base: BASE,
|
|
commit: commit ? commit.summary : 'commit agent died — commit manually',
|
|
spec: { sharedModules: spec.sharedModules, webUiStaging: spec.webUiStaging, emfDesign: spec.emfDesign, derivedAgreementNote: spec.notes },
|
|
parityEvidence: parity ? { po: parity.poVerdict, wo: parity.woVerdict, webUi: parity.webUiVerdict } : null,
|
|
implementation: implOk.map(r => r.summary),
|
|
filesChanged: implOk.flatMap(r => r.filesChanged),
|
|
verifyRounds: round + 1,
|
|
blockers: implBlockers.concat(reconBlockers),
|
|
outstandingGates: [
|
|
'/sh-security-review (MANDATORY before push — auth code moved: ses_auth + the web_ui auth gate are exactly the authentication surface; run on the committed diff, and re-run after any post-review fix to this code)',
|
|
'cross-family cross_review.py NOT mandatory (no IAM change; handler event/return contracts unchanged — internal module moves only). Opt-in if judgment says so',
|
|
'push + PR + gh pr checks green',
|
|
'deploy-then-merge: deploy from branch, smoke green, one real PO + WO email each, watch BOTH sender-auth-rejected alarms through live mail (the auth move must not change accept/reject behavior), verify web_ui login still works, THEN merge — and note the post-merge CI redeploy being a no-op (unchanged asset hashes) is itself a verification signal',
|
|
],
|
|
}
|