procurement-ingest/.claude/workflows/phase-3-shared-extraction.js
Adam Moussa 30112cc680
Some checks are pending
Deploy / deploy (push) Waiting to run
feat: extract lambdas/shared/ — single-source ses_auth, web_ui auth, email parsing, EMF emitter (refactor phase 3) (#111)
Four modules move into the handbook-mandated lambdas/shared/ location,
collapsing duplicated logic that had to be kept in sync by hand across
the PO and WO pipelines:

- ses_auth.py: the PO and WO copies were verified sha256-identical
  against the feature/phase-7-ops-recovery baseline before the move
  (no drift since the last audit). shared/ses_auth.py is the exact
  bytes of that one copy; both originals are git rm'd (the PO copy
  via rename, the WO copy as a straight delete). Bundling lands the
  module flat in /asset-output for both email processors, so the
  handlers keep `from ses_auth import authenticate_inbound_email`
  unchanged — zero handler diff for this move, which is what keeps
  fail-closed auth byte-identical through the change.

- web_ui_auth.py: extracts the byte-identical _get_auth_token /
  _header / is_authenticated block plus the four token-cache globals
  out of both web_ui handlers. The per-stack INFRA-74 comments stay
  in each handler as-is (deliberately drifted wording, stack-specific)
  rather than being unified into the shared module. Fail-closed
  semantics (unset ARN or Secrets Manager exception -> deny) are
  unchanged.

- email_parsing.py: parse_raw_email ships as the superset version that
  returns cc unconditionally. WO's output is bit-identical to before;
  PO simply ignores the cc field rather than being "cleaned up" to
  consume it. No second variant is kept.

- emf.py: a generic emitter parameterized by namespace, dimension
  sets, and properties. Every call site's emitted EMF envelope is
  unchanged, including the load-bearing
  [["ParseMethod"],["ParseMethod","TemplateId"]] dimension-set shape
  the alarms and metric filters depend on. Emission ordering is
  untouched: PO still emits ai_fallback before the Bedrock call, WO
  still emits its mutually-exclusive ai_fallback/ai_fallback_rejected
  after its gate. The deliberate-double-count comments survive.
  _emit_derived_agreement_metric was found living inside
  derived_fields.py, so per the DERIVED-FIELDS exception it is left
  as a third, unconverted copy (derived_fields.py and the shadow
  DerivedFieldAgreement telemetry stay untouchable while that bake
  runs) — a comment there points at shared/emf.py for the eventual
  follow-up.

Bundling: both email-processor cdk bundling commands gain a trailing
`cp shared/*.py /asset-output/` (they were already cp-only post-Phase
7, so no pip step or manylinux pin is reintroduced). Both web_ui
functions gain the same widened-root staging so web_ui_auth.py ships
beside their handler; site_extractor's from_asset is untouched.

Tests: PO_EXPECTED_TOP_LEVEL_MODULES gains the shared modules that now
ship, the AST sibling-import check resolves imports whose source now
lives under shared/, and the new shared cp line has its own
revert/mutation detection. _SIBLING_MODULES resolution and
_po_parser_support.py now load ses_auth/email_parsing/emf from
shared/; the two-copy ses_auth byte-identity fixture-hygiene test is
retired as obsolete now that there is one copy, and the ses_auth
fixture parameterization over two identical copies is dropped. The
sys.modules save/restore dance for template_parser (still duplicated
per-pipeline) is left in place.
2026-07-20 13:38:23 -04:00

679 lines
38 KiB
JavaScript

export const meta = {
name: 'phase-3-shared-extraction',
description: 'Phase 3 of the procurement-ingest refactor (docs/refactor-evaluation.md): extract lambdas/shared/ — ses_auth.py (byte-identical move, zero handler diff), web_ui_auth.py (byte-identical auth block from both web_ui handlers), email_parsing.py (parse_raw_email superset, cc unconditional), emf.py (parameterized emitter preserving every envelope byte-for-byte). Bundling gains cp shared/*.py in both email-processor commands + the same staging mechanism for both web_ui functions. Requires Phases 0-2 on the base branch. Auth code moves, so push is gated on /sh-security-review in the main loop. Committed locally, never pushed.',
phases: [
{ title: 'Setup', detail: 'verify Phases 0+1+2 on base, branch feature/phase-3-shared-extraction', model: 'haiku' },
{ title: 'Recon', detail: '4 mappers: the four duplicated modules, post-Phase-2 bundling sites, test-loader plumbing, deployed-zip baseline (read-only AWS)' },
{ title: 'Spec', detail: 'serial fable spec: pin shared-module contents, handler edit lists, web_ui staging, bundling strings, EMF design, test plumbing' },
{ title: 'Implement', detail: 'opus: shared/ + four handlers; sonnet: both CDK stacks; opus: all test plumbing + README — disjoint files', model: 'opus' },
{ title: 'Verify', detail: 'mechanical gates + bundle-parity verifier + 3 fable lenses (auth integrity, EMF/telemetry, loader integrity)' },
{ title: 'Fix', detail: 'opus fixer, full re-verify, max 3 rounds', model: 'opus' },
{ title: 'Package', detail: 'single commit via -F (no push)', model: 'sonnet' },
],
}
// ---------------------------------------------------------------- constants
const REPO = '/Users/adammoussa/Documents/repositories/seahaven/procurement-ingest'
const BRANCH = 'feature/phase-3-shared-extraction'
let _args = args
if (typeof _args === 'string') {
try { _args = JSON.parse(_args) } catch (e) { _args = null }
}
const BASE = (_args && _args.base) || 'main'
const CONSTRAINTS = `
PINNED BEHAVIORAL CONSTRAINTS (docs/refactor-evaluation.md Phase 3 — violating any is a build failure):
1. THE MOVE SET IS EXACTLY FOUR MODULES, in this order of dependency risk:
lambdas/shared/ses_auth.py, lambdas/shared/web_ui_auth.py,
lambdas/shared/email_parsing.py, lambdas/shared/emf.py. Nothing else
moves into shared/. lambdas/shared/ is the handbook location
(cdk-project-layout.md); modules land FLAT in every bundle so bare-name
imports keep working.
2. ses_auth: FIRST verify the two current copies are still byte-identical
(sha256 both against the ${BASE} versions — any drift since the audit is
a blocker, not something to silently reconcile). shared/ses_auth.py is
the EXACT bytes of that single copy; both originals are git rm'd. The
email-processor handlers keep 'from ses_auth import
authenticate_inbound_email' UNCHANGED — flat landing in /asset-output
means ZERO handler diff for this move, which is what keeps fail-closed
auth byte-identical through the change.
3. web_ui_auth: extract exactly the byte-identical block — _get_auth_token /
_header / is_authenticated + the four cache globals — from BOTH web_ui
handlers. The per-stack INFRA-74 comments STAY in each handler (their
wording has drifted deliberately; they are stack-specific — do NOT unify
or move them into the shared module). Fail-closed semantics (unset ARN,
Secrets Manager exception -> deny) must be unchanged; the token-cache
globals move with the functions that read them.
4. email_parsing.py: parse_raw_email as the SUPERSET version returning cc
unconditionally. WO's output is bit-identical to today; PO simply
ignores cc — do NOT "clean up" PO to consume it, and do NOT preserve two
variants.
5. emf.py: a generic emitter parameterized by namespace / dimension-sets /
properties. Every call site's emitted EMF envelope must be EXACTLY what
it emits today — the dimension-set list
[["ParseMethod"],["ParseMethod","TemplateId"]] is load-bearing for the
alarms and metric filters; making one-sided dimension fixes impossible
is the point of this move. Emission ORDERING is untouchable: PO emits
ai_fallback BEFORE the Bedrock call, WO after its gate with mutually-
exclusive ai_fallback/ai_fallback_rejected — these two deliberate
per-pipeline differences are pinned by tests; converting a call site
must not move it. The deliberate-double-count comments survive.
6. DERIVED-FIELDS EXCEPTION: if recon finds _emit_derived_agreement_metric
lives INSIDE derived_fields.py, do NOT convert it — derived_fields.py
and the shadow DerivedFieldAgreement telemetry are UNTOUCHABLE while the
bake runs (this outranks the emf consolidation). Leave it as a third
copy with a code comment pointing at shared/emf.py and report it in
notes/blockers. Only convert it if it lives outside derived_fields.py.
7. UNTOUCHABLE FILES (git diff ${BASE}...HEAD must be empty for each):
both template_parser.py (990 vs 508 lines, genuinely divergent — stays
per-pipeline), derived_fields.py, both validate_ai_fallback gate
modules, extract_with_claude's Bedrock invocation/prompt logic. Handler
diffs are LIMITED to: deleting moved code, import changes, and
emitter-call swaps per the binding spec. No opportunistic refactors.
8. Bundling (cdk): append 'cp shared/*.py /asset-output/' to BOTH
email-processor bundling commands (Phase 2 made the bundling cwd the
../lambdas asset root, so the path resolves as written).
BASE-AWARENESS — READ THE ACTUAL cdk FILES ON THE BASE, DO NOT ASSUME:
this phase is stacked on the Phase 7 branch, which ALREADY removed the
pip install step from both email-processor commands (they are now
CP-ONLY: no pip line, no manylinux pin, because nothing third-party is
installed). Phase 3 moves only pure first-party modules (no new deps),
so cp-only STAYS cp-only — you simply add another 'cp shared/*.py
/asset-output/' line. DO NOT re-introduce a pip install step and DO NOT
re-add the manylinux pin: there is nothing to pin when nothing installs,
and adding a pip step back would be the regression here. (The PR #34
lesson — never drop the manylinux pin while a pip install runs — still
holds ONLY if recon finds a pip step actually present on the base; on
the cp-only Phase 7 base there is none.) PRESERVE the Phase 2 exclude
lists exactly. Both web_ui functions gain the SAME staging mechanism
(widened-root bundled from_asset) so web_ui_auth.py ships beside their
handler — the spec agent pins the exact form. site_extractor's
from_asset is UNTOUCHED (Phase 6 territory).
9. tests/test_bundle_consistency.py is updated IN THE SAME CHANGE without
losing teeth: the PO_EXPECTED_TOP_LEVEL_MODULES exact-set pin gains the
shared modules that now ship; the AST sibling-import check must resolve
imports whose source file now lives under shared/; the new shared cp
line gets its own revert/mutation detection (a commented-out
'cp shared/*.py' must fail the test).
10. Test plumbing in the same PR: update _SIBLING_MODULES resolution and
_po_parser_support.py (~line 71 — verify the current line) so tests
load ses_auth/email_parsing/emf from shared/; retire or repoint the
fixture-hygiene test that polices the two ses_auth copies for
byte-identity (obsolete once there is one copy — do not leave it
failing); drop the ses_auth fixture params from test_ses_auth
(parameterizing over two identical copies is dead weight — halves the
run). The sys.modules save/restore dance SURVIVES for template_parser
(still a duplicated bare name) — do not delete it. Preserve the
load-bearing moto-before-handler import ordering.
11. STALE-SHADOW HAZARD: after the move, no stale ses_auth.py / .pyc /
__pycache__ copy may remain anywhere it could shadow the shared copy —
in the repo (git rm, don't empty), in staged assets, or on any test
sys.path. Verify explicitly.
12. cdk diff on BOTH stacks: the only resource deltas allowed are Code/
S3Key (asset) changes on the two email processors and the two web_ui
functions (+ CDK metadata). No logical-ID changes, no alarm, IAM,
table, env, runtime, or handler-property deltas of any kind.
13. AWS access is READ-ONLY (get-function, downloading deployed zips via
presigned URLs). NEVER cdk deploy, never invoke, never mutate.
`
const PREAMBLE = `
You are one of several agents building refactor Phase 3 in the git repo at
${REPO} on branch ${BRANCH} (already checked out — do NOT switch branches,
do NOT create branches, do NOT commit, NEVER push, do NOT run cdk deploy or
touch AWS resources beyond read-only calls).
Authoritative spec: docs/refactor-evaluation.md, section "Phase 3".
Work ONLY in the files you are told you own; other agents are concurrently
editing other files in this same working tree.
${CONSTRAINTS}
Your final message is consumed by an orchestrator script, not a human —
return only the structured data requested.
`
// ------------------------------------------------------------------ schemas
const RECON = {
type: 'object',
required: ['summary', 'facts'],
properties: {
summary: { type: 'string' },
facts: { type: 'array', items: { type: 'string' } },
blockers: { type: 'array', items: { type: 'string' } },
},
}
const SPEC = {
type: 'object',
required: ['sharedModules', 'handlerEdits', 'webUiStaging', 'bundlingEdits', 'emfDesign', 'testPlumbing', 'parityRules', 'notes'],
properties: {
sharedModules: { type: 'string', description: 'per shared module: exact provenance (which copy is the source, sha256), full contents decision (verbatim move vs superset vs parameterized), and the public surface each importer uses' },
handlerEdits: { type: 'string', description: 'per handler (po/wo email_processor, po/wo web_ui): the exact deletions, import lines, and emitter-call swaps — file:line, nothing else may change' },
webUiStaging: { type: 'string', description: 'the complete new from_asset blocks for both web_ui functions: asset path, command or cp form, exclude list — exact code' },
bundlingEdits: { type: 'string', description: 'the two email-processor bundling command strings with cp shared/*.py appended, verbatim, plus the expected top-level module set of each resulting bundle' },
emfDesign: { type: 'string', description: 'shared/emf.py signature + per-call-site mapping proving each emitted envelope (namespace, dimension-set list, properties) is byte-equivalent to today; the derived-agreement emitter decision per constraint 6' },
testPlumbing: { type: 'string', description: 'every test/support file change: loader resolution, _SIBLING_MODULES, fixture-hygiene retirement, test_ses_auth de-parameterization, test_bundle_consistency edits with their mutation-detection shapes' },
parityRules: { type: 'string', description: 'how Verify judges staged bundles vs the deployed-zip baseline: expected first-party set per function (= deployed set + the new shared modules), ses_auth byte-identity requirement, expected removals (none beyond moved-file provenance), web_ui bundle expectations' },
notes: { type: 'string' },
},
}
const IMPL = {
type: 'object',
required: ['filesChanged', 'summary', 'checksRun'],
properties: {
filesChanged: { type: 'array', items: { type: 'string' } },
summary: { type: 'string' },
checksRun: { type: 'string' },
blockers: { type: 'array', items: { type: 'string' } },
},
}
const CHECKS = {
type: 'object',
required: ['passed', 'details'],
properties: {
passed: { type: 'boolean' },
details: { type: 'string' },
scopeViolations: { type: 'array', items: { type: 'string' } },
},
}
const PARITY = {
type: 'object',
required: ['passed', 'poVerdict', 'woVerdict', 'webUiVerdict', 'details'],
properties: {
passed: { type: 'boolean' },
poVerdict: { type: 'string', description: 'PO email-processor staged vs deployed: module-set delta exactly as spec, ses_auth bytes identical — full evidence' },
woVerdict: { type: 'string', description: 'same for WO email-processor' },
webUiVerdict: { type: 'string', description: 'both web_ui staged bundles: own *.py + web_ui_auth.py present, nothing stray, hashes deterministic' },
details: { type: 'string' },
},
}
const FINDINGS = {
type: 'object',
required: ['findings'],
properties: {
findings: {
type: 'array',
items: {
type: 'object',
required: ['title', 'severity', 'confirmed', 'evidence', 'fix'],
properties: {
title: { type: 'string' },
severity: { enum: ['critical', 'high', 'medium', 'low'] },
confirmed: { type: 'boolean' },
evidence: { type: 'string' },
fix: { type: 'string' },
},
},
},
},
}
// ------------------------------------------------------------------- setup
phase('Setup')
const setup = await agent(`
In ${REPO}:
1. SEQUENCING GATE — Phases 0, 1 AND 2 must all be on ${BASE} (Phase 3
edits the same stack files as Phase 2 and depends on its widened
../lambdas asset roots to make shared/ reachable). git fetch origin,
then pick the base ref: origin/${BASE} if that remote ref exists,
otherwise the local branch ${BASE} (a stacked local-only base is
expected and fine). Verify on the base ref:
(a) Phase 0: tests/test_bundle_consistency.py exists
(git show <baseref>:tests/test_bundle_consistency.py | head -3);
(b) Phase 1: validate_ai_fallback exists in the PO pipeline
(git grep validate_ai_fallback <baseref> -- lambdas/po);
(c) Phase 2: BOTH cdk/po_stack.py and cdk/wo_stack.py on the base ref
contain Code.from_asset("../lambdas") for the email processors
(git show <baseref>:cdk/po_stack.py | grep -n '\\.\\./lambdas', same
for wo_stack.py).
If any is missing, STOP with a blocker naming the unmet phase and do
nothing else.
2. Verify clean working tree (untracked .coverage / .claude/ / the local
44 MB lambdas/po/email_processor/package/ dir are fine; any OTHER dirt =
blocker, never stash or discard).
3. git checkout ${BASE}; then git pull --ff-only ONLY if the branch has an
upstream (a local-only base skips the pull — not a blocker); then
git checkout -b ${BRANCH}
4. gh pr list --state open --json number,title,headRefName (overlap check).
Return facts: HEAD sha, per-phase gate evidence, open PRs, blockers.
`, { label: 'setup:branch', model: 'haiku', schema: RECON })
if (!setup || (setup.blockers && setup.blockers.length)) {
return { aborted: 'setup blockers', blockers: setup ? setup.blockers : ['setup agent died'], facts: setup ? setup.facts : [] }
}
log(`Branch ${BRANCH} ready off ${BASE}. ${setup.summary}`)
// ------------------------------------------------------------------- recon
phase('Recon')
const recon = await parallel([
() => agent(`${PREAMBLE}
Read-only recon of the FOUR duplicated surfaces being extracted:
1. ses_auth.py both copies — sha256 of each (MUST match; drift = blocker),
line count, the exact import line each email-processor handler uses,
any other importer (git grep 'import ses_auth\\|from ses_auth').
2. web_ui auth block — in BOTH web_ui handlers quote with file:line the
exact boundaries of the byte-identical block (_get_auth_token, _header,
is_authenticated, the four cache globals), diff the two blocks to prove
byte-identity, quote each INFRA-74 comment verbatim (they differ —
that is expected), and note everything else in each handler that CALLS
the block.
3. parse_raw_email both copies — where each lives (own sibling file vs
inside handler.py), the exact cc delta between them, every call site.
4. The THREE EMF emitters — quote each verbatim with file:line (PO
ParseMethod emitter, WO ParseMethod emitter,
_emit_derived_agreement_metric), namespace + dimension-set list +
properties of each, and CRITICALLY: which FILE _emit_derived_agreement_metric
lives in (constraint 6 hinges on whether it is inside derived_fields.py).
Also pin current line numbers of PO's pre-Bedrock ai_fallback emit and
WO's post-gate emit.
25-35 precise facts.`,
{ label: 'recon:duplicated-modules', model: 'sonnet', phase: 'Recon', schema: RECON }),
() => agent(`${PREAMBLE}
Read-only recon of the post-Phase-2 CDK bundling state: for ALL FIVE
Code.from_asset sites in cdk/po_stack.py and cdk/wo_stack.py quote verbatim
with current file:line — asset path, full bundling command (or plain form),
exclude list. For the two web_ui functions additionally record: runtime,
architecture, handler property, memory/timeout, whether any requirements.txt
exists for them, and every function property that must NOT change when
bundling is added. Confirm lambdas/shared/ does not exist yet and list
anything at lambdas/ top level. 15-25 facts.`,
{ label: 'recon:cdk-bundling', model: 'haiku', phase: 'Recon', schema: RECON }),
() => agent(`${PREAMBLE}
Read-only recon of the test plumbing this phase must rewire:
- tests/conftest.py importlib loader: how it resolves module paths, the
sys.modules save/restore, _SIBLING_MODULES (exact current contents).
- _po_parser_support.py: the independent importlib reimplementation, the
load-bearing moto-before-handler import order (~lines 26-33), and what
is at line ~71 (the doc cites it — quote the current code).
- _wo_parser_support.py bare sys.path 'import handler' strategy.
- test_ses_auth.py at root: the fixture parameterization over both copies
(quote it), total line count, the fixture-hygiene test that polices
byte-identity between the two copies (name + assertion).
- test_parse_raw_email.py: how it loads both handlers' parse_raw_email.
- tests/test_bundle_consistency.py: every assertion that will fail red
against the Phase 3 shapes (shared cp line, PO_EXPECTED_TOP_LEVEL_MODULES,
AST sibling-import resolution for moved modules).
20-30 facts.`,
{ label: 'recon:test-plumbing', model: 'sonnet', phase: 'Recon', schema: RECON }),
() => agent(`${PREAMBLE}
Read-only AWS recon (region us-east-1, READ-ONLY): for po-email-processor,
workorder-email-processor, AND both web_ui functions (find their exact
function names via the stacks/aws lambda list-functions) run aws lambda
get-function; for the two email processors download each Code.Location
presigned zip to a scratch dir and produce the COMPLETE file list
(unzip -l) separated into (a) first-party top-level .py, (b) dependency
dirs, (c) anything else; record every function's CodeSha256 and the
sha256 of ses_auth.py inside each deployed zip (the byte-identity baseline
the moved copy is judged against). For the web_ui functions the current
zip file list is the baseline their new bundled asset must cover.
Return the categorized lists as facts.`,
{ label: 'recon:deployed-baseline', model: 'sonnet', phase: 'Recon', schema: RECON }),
])
const reconOk = recon.filter(Boolean)
const pack = reconOk.map(r => `## ${r.summary}\n${r.facts.join('\n')}`).join('\n\n')
const reconBlockers = reconOk.flatMap(r => r.blockers || [])
.filter(b => b && !/^\s*(none|n\/a)\b/i.test(b))
log(`Recon complete: ${reconOk.length}/4 mappers, ${reconBlockers.length} blockers`)
if (reconBlockers.length) {
return { aborted: 'recon blockers (likely copy drift — resolve before extracting)', blockers: reconBlockers, reconPack: pack }
}
// -------------------------------------------------------------------- spec
phase('Spec')
const spec = await agent(`${PREAMBLE}
You are the SPEC agent — the single authority that pins every contested
decision BEFORE parallel implementation (parallel leaves cannot see each
other's choices). Using the recon pack below plus your own reads of the
actual files, produce the binding implementation spec:
- sharedModules: per constraint 1's four modules — provenance, contents
decision, public surface. ses_auth is a verbatim byte-move; web_ui_auth
is the exact block; email_parsing is the WO superset (cc unconditional);
emf is the parameterized emitter.
- handlerEdits: for each of the four handlers, the exact minimal edit list
(constraint 7 — deletions, imports, emitter-call swaps ONLY). State
explicitly that the two email-processor ses_auth import lines are
UNCHANGED.
- webUiStaging: complete replacement from_asset blocks for both web_ui
functions — widened root, cp command staging the function's own *.py
plus web_ui_auth.py (pin whether to cp shared/*.py or just
shared/web_ui_auth.py — pick ONE rule, state why), Phase-2-style
excludes. Mind bundling cwd semantics: the container mounts the asset
root (../lambdas) as the working dir. If a no-Docker staging form is
viable and simpler, you may pin that instead — but ONE mechanism for
both, exact code.
- bundlingEdits: both email-processor command strings verbatim with
'cp shared/*.py /asset-output/' appended, plus the exact expected
top-level module set of each resulting bundle (this feeds
PO_EXPECTED_TOP_LEVEL_MODULES and the parity gate).
- emfDesign: the shared emitter signature and a per-call-site table
proving envelope byte-equivalence; resolve the derived-agreement emitter
per constraint 6 based on where recon found it.
- testPlumbing: every file, every edit — loader resolution for shared/,
_SIBLING_MODULES, _po_parser_support.py, _wo_parser_support.py if it
needs the shared path, fixture-hygiene retirement, test_ses_auth
de-parameterization, test_parse_raw_email (single implementation now —
what does it exercise?), test_bundle_consistency edits including the new
mutation shapes (commented-out shared cp must fail).
- parityRules: exactly how Verify judges staged bundles vs recon's
deployed baseline — per-function expected first-party set (deployed set
minus nothing, plus the shared modules per your cp rule), ses_auth
sha256 must equal the deployed zips' copy, web_ui bundles must cover
their deployed file list plus web_ui_auth.py, nothing else may appear
or vanish.
Recon pack:\n${pack}`,
{ label: 'spec:pin-extraction', phase: 'Spec', schema: SPEC })
if (!spec) return { aborted: 'spec agent died — rerun workflow', reconBlockers }
const specBlock = `BINDING SPEC (from the spec agent — implement EXACTLY this):\n${JSON.stringify(spec, null, 2)}`
log('Spec pinned: shared modules, handler edits, web_ui staging, bundling strings, EMF design, test plumbing')
// --------------------------------------------------------------- implement
phase('Implement')
const impl = await parallel([
() => agent(`${PREAMBLE}
YOU OWN: everything under lambdas/ EXCEPT the tests/ directories (another
agent owns all test and support files). Do not touch cdk/ or README.
Task: execute the four moves per spec.sharedModules + spec.handlerEdits +
spec.emfDesign, in the spec's order:
1. git mv (or create+git rm preserving exact bytes) ses_auth.py ->
lambdas/shared/ses_auth.py; delete both originals; prove sha256
equality in your summary. Handler import lines untouched.
2. Create lambdas/shared/web_ui_auth.py from the byte-identical block;
replace the block in BOTH web_ui handlers with the import; keep each
INFRA-74 comment in place.
3. Create lambdas/shared/email_parsing.py (superset); rewire both
email-processor call sites.
4. Create lambdas/shared/emf.py; swap the emitter call sites per
spec.emfDesign — honoring constraint 6 (derived_fields.py stays
untouched if the third emitter lives there) and constraint 5 (emission
ordering does not move).
Delete every now-empty moved-out file with git rm (constraint 11).
Run before returning: ruff check lambdas && ruff format lambdas --check,
plus an import smoke: python3 -c with sys.path prepended for
lambdas/shared + each function dir, importing every touched module (tests
are NOT yours to run — the plumbing agent lands them in parallel).
${specBlock}`,
{ label: 'impl:shared-lambdas', model: 'opus', phase: 'Implement', schema: IMPL }),
() => agent(`${PREAMBLE}
YOU OWN: cdk/po_stack.py and cdk/wo_stack.py ONLY (and only the from_asset
regions — alarms, IAM, tables, env are all off-limits).
Task: apply spec.bundlingEdits (append the shared cp to both
email-processor commands — the base is the Phase 7 CP-ONLY bundling, so
just add another 'cp shared/*.py /asset-output/' line; DO NOT re-add a pip
install step or a manylinux pin, there is nothing third-party to install
(constraint 8); preserve the Phase 2 excludes verbatim) and
spec.webUiStaging (both web_ui functions). Touch nothing else;
site_extractor's from_asset stays byte-identical.
Run before returning: ruff check cdk && cd cdk &&
npx cdk synth po-ingest -q -o /tmp/phase3-synth &&
npx cdk synth workorder-ingest -q -o /tmp/phase3-synth (artifact-id
selectors, NOT stack_name). Docker bundling runs — confirm each staged
email-processor asset contains the shared modules and each staged web_ui
asset contains web_ui_auth.py; note asset hashes in your summary. If the
lambdas/ moves have not landed yet the synth will fail on missing files —
poll by re-running up to ~10 min before reporting a blocker.
${specBlock}`,
{ label: 'impl:cdk-stacks', model: 'sonnet', phase: 'Implement', schema: IMPL }),
() => agent(`${PREAMBLE}
YOU OWN: all test and support files (tests/ at repo root including
tests/test_bundle_consistency.py and tests/conftest.py,
lambdas/po/email_processor/tests/, lambdas/wo/email_processor/tests/)
and README.md ONLY.
Task A: apply spec.testPlumbing in full — loader/_SIBLING_MODULES
resolution for shared/, _po_parser_support.py edit (preserving the
moto-before-handler ordering comment), fixture-hygiene retirement,
test_ses_auth de-parameterization, test_parse_raw_email update,
test_bundle_consistency updates WITHOUT losing teeth (constraint 9 — the
new shared cp pin must reject a commented-out or narrowed variant; keep
the existing mutation tests green).
Task B: README — document lambdas/shared/ in the repo-layout section
(which modules live there, the flat-landing import rule, the bundling cp
that ships them), update the bundling/deploy-guards paragraphs for the
shared cp + web_ui staging, and note that ses_auth is now single-sourced
(one hardening fix lands once). Match existing README style.
Run before returning: pytest -q --no-cov at repo root (must be green
against the OTHER agents' edits — they land in parallel; poll by
re-running up to ~10 min before reporting a blocker) and ruff check on
every file you touched.
${specBlock}`,
{ label: 'impl:test-plumbing-readme', model: 'opus', phase: 'Implement', schema: IMPL }),
])
const implOk = impl.filter(Boolean)
const implBlockers = implOk.flatMap(r => r.blockers || [])
log(`Implement complete: ${implOk.length}/3 agents, blockers: ${implBlockers.length}`)
// ---------------------------------------------------- verify + fix loop
const EXPECTED_SCOPE = [
'lambdas/shared/',
'lambdas/po/email_processor/',
'lambdas/wo/email_processor/',
'lambdas/po/web_ui/',
'lambdas/wo/web_ui/',
'cdk/po_stack.py',
'cdk/wo_stack.py',
'tests/',
'README.md',
]
const mechanicalPrompt = `${PREAMBLE}
Independent re-verification — trust nothing self-reported. Run ALL gates,
quoting failures verbatim:
1. pytest -q --no-cov (repo root, all three roots green)
2. ruff check . && ruff format --check .
3. cd cdk && npx cdk synth po-ingest -q && npx cdk synth workorder-ingest -q
4. cd cdk && npx cdk diff po-ingest ; npx cdk diff workorder-ingest —
constraint 12: only the four functions' Code/S3Key (+ metadata) deltas.
Any alarm/IAM/env/runtime/handler-prop/logical-ID delta = FAIL. Paste
the diff summaries.
5. UNTOUCHABLES (each must output NOTHING):
git diff ${BASE}...HEAD -- lambdas/po/email_processor/derived_fields.py
git diff ${BASE}...HEAD -- lambdas/po/email_processor/template_parser.py
git diff ${BASE}...HEAD -- lambdas/wo/email_processor/template_parser.py
plus both validate_ai_fallback gate modules (resolve their filenames
first) and lambdas/po/site_extractor/.
6. BYTE-IDENTITY: sha256 of lambdas/shared/ses_auth.py equals sha256 of
git show ${BASE}:lambdas/po/email_processor/ses_auth.py (and the wo
copy). Both originals gone from the tree (git ls-files check) and no
stray ses_auth.py/__pycache__ anywhere under lambdas/ outside shared/
(constraint 11).
7. ORDERING PINS: grep line numbers proving PO's ai_fallback emit still
precedes the Bedrock invoke and WO's emits are untouched relative to
${BASE} (the wo handler diff must contain ONLY the spec's edit classes).
8. INFRA-74: both web_ui handlers still contain their own INFRA-74
comment verbatim per ${BASE}.
9. git status --porcelain scope check: every modified/added/deleted path
under ${EXPECTED_SCOPE.join(', ')} (untracked .coverage/.claude/
package/ tolerated).
passed=true only if all green. YOU MAY NOT edit files.`
const parityPrompt = `${PREAMBLE}
You are the BUNDLE-PARITY verifier — the load-bearing gate of this phase.
Everything is local synth + read-only AWS.
1. cd cdk && npx cdk synth po-ingest -q -o /tmp/phase3-parity &&
npx cdk synth workorder-ingest -q -o /tmp/phase3-parity. Locate all
four staged assets (two email processors, two web_ui).
2. Re-download both email-processor deployed zips fresh (aws lambda
get-function Code.Location, us-east-1) — do not trust a recon cache —
and fetch both web_ui functions' deployed file lists.
3. Judge per spec.parityRules: per email processor, staged first-party
top-level .py set == deployed set PLUS exactly the shared modules the
spec's cp rule ships (list both sets; nothing else may appear or
vanish); sha256 of staged ses_auth.py == sha256 of the ses_auth.py
inside each deployed zip (fail-closed auth byte-identical through the
move); dependency packages compared by name, version drift noted not
failed. Per web_ui function: staged bundle covers the deployed file
list plus web_ui_auth.py, nothing stray (no tests/, no other
pipeline's sources, no .eml, no package/).
4. DETERMINISM: synth po-ingest twice into fresh -o dirs — identical
asset hashes, including the NEW web_ui assets. Then drop a throwaway
__pycache__/junk.pyc under lambdas/shared/ (delete it afterwards),
re-synth, and confirm the excludes keep every hash unchanged.
5. IMPORT-RESOLUTION sanity: inside each staged email-processor asset
dir run python3 -c "import handler" with that dir alone on sys.path
(env-var stubs as needed) — proves the flat landing satisfies every
import including 'from ses_auth import ...' with the moved copy.
passed=true only if every check holds.`
const lenses = [
{ key: 'auth-integrity', prompt: `${PREAMBLE}
ADVERSARIAL REVIEW — auth-integrity lens. Auth code moved; try to prove
the move WEAKENED it. (1) ses_auth: byte-compare the shared copy against
${BASE}'s copies yourself; then attack import resolution — in the BUNDLE
and in TESTS, which ses_auth wins if anything shadows (stale .pyc, a
same-named module on sys.path, the tests loader resolving the old path
silently to a stub)? Could a test now pass against a MOCK of ses_auth
where it previously exercised the real module? (2) web_ui_auth: diff the
extracted block against both originals — any dropped line, changed
global, or reordered check? Is the fail-closed path (unset ARN, Secrets
exception, wrong token -> 401 before any table access) provably
unchanged? Do the four cache globals still behave per-function (module
now shared — could cross-importer state ever leak)? (3) The gate call
sites: could any handler path now reach S3-fetch/Bedrock/save before
authenticate_inbound_email or is_authenticated, where it could not
before? confirmed=true only with a concrete exploit sketch or
file:line proof.` },
{ key: 'telemetry-emf', prompt: `${PREAMBLE}
ADVERSARIAL REVIEW — EMF/telemetry lens. The alarms and metric filters
consume exact EMF shapes; a silent envelope change breaks paging without
failing any test. Read shared/emf.py + every converted call site
(git diff ${BASE}) and the synthesized templates' metric filters/alarms.
Verify per call site: namespace exact, dimension-set list EXACT
([["ParseMethod"],["ParseMethod","TemplateId"]] where applicable, order
included), property keys/types identical, timestamp/CloudWatchMetrics
envelope structure identical — construct a sample emission per call site
and diff it against ${BASE}'s hand-built _aws dict output. Then the
ordering pins: PO ai_fallback still pre-Bedrock, WO still post-gate
mutually-exclusive, the deliberate double-count comment intact. Finally
constraint 6: where does _emit_derived_agreement_metric live and was the
decision honored (derived_fields.py diff empty)? confirmed=true only
with evidence.` },
{ key: 'loader-integrity', prompt: `${PREAMBLE}
ADVERSARIAL REVIEW — test/loader-integrity lens. The three loading idioms
were rewired; attack them. (1) Does test_ses_auth now exercise the REAL
lambdas/shared/ses_auth.py (trace the loader path), and did
de-parameterization silently drop any assertion that only ran under one
param? (2) Fixture-hygiene: the byte-identity police is retired — does
anything still guard against a future stray ses_auth.py copy reappearing
in a pipeline dir (should the bundle-consistency or a hygiene test)? If
nothing does, that is a finding. (3) sys.modules save/restore: prove it
still isolates template_parser between pipelines (run two cross-pipeline
tests back to back); prove moto-before-handler ordering survived in
_po_parser_support.py. (4) test_bundle_consistency: hand-mutate command
strings on scratch copies — a commented-out 'cp shared/*.py', a shared
glob narrowed to one file, and a PO_EXPECTED_TOP_LEVEL_MODULES missing a
shared module must each FAIL. (5) Run pytest twice in one session and in
file-shuffled order (-p no:randomly not installed? then two explicit
orderings) to smoke out import-order coupling introduced by the shared
path. confirmed=true only with file:line or reproduced-failure
evidence.` },
]
let round = 0
let checks = null
let parity = null
let confirmed = []
while (round < 3) {
phase('Verify')
const results = await parallel([
() => agent(mechanicalPrompt, { label: `verify:mechanical-r${round}`, model: 'sonnet', phase: 'Verify', schema: CHECKS }),
() => agent(parityPrompt, { label: `verify:parity-r${round}`, model: 'opus', phase: 'Verify', schema: PARITY }),
...lenses.map(l => () =>
agent(l.prompt, { label: `verify:${l.key}-r${round}`, phase: 'Verify', schema: FINDINGS })),
])
checks = results[0]
parity = results[1]
confirmed = results.slice(2).filter(Boolean)
.flatMap(r => r.findings || [])
.filter(f => f.confirmed && f.severity !== 'low')
const green = checks && checks.passed && parity && parity.passed
log(`Verify round ${round}: mechanical ${checks && checks.passed ? 'GREEN' : 'RED'}, parity ${parity && parity.passed ? 'GREEN' : 'RED'}, confirmed findings: ${confirmed.length}`)
if (green && confirmed.length === 0) break
round += 1
if (round >= 3) break
phase('Fix')
await agent(`${PREAMBLE}
You are the fix agent — you may edit files under: ${EXPECTED_SCOPE.join(', ')}.
Fix EVERY item below minimally; the binding spec and 13 pinned constraints
still hold (a finding that conflicts with a constraint is reported, not
"fixed" — the constraint wins, esp. constraint 6's derived_fields
untouchability and constraint 5's emission ordering). Re-run the specific
failing gate/test per fix.
MECHANICAL:\n${checks ? checks.details : '(agent died — rerun all gates)'}
PARITY:\n${parity ? parity.details : '(agent died — rerun all)'}
CONFIRMED FINDINGS:\n${JSON.stringify(confirmed, null, 2)}
${specBlock}`,
{ label: `fix:round-${round}`, model: 'opus', phase: 'Fix', schema: IMPL })
}
const verifyClean = checks && checks.passed && parity && parity.passed && confirmed.length === 0
if (!verifyClean) {
return {
status: 'NEEDS ATTENTION — verify not clean after 3 rounds; branch left uncommitted',
branch: BRANCH,
mechanical: checks,
parity,
unresolvedFindings: confirmed,
implBlockers,
reconBlockers,
spec,
}
}
// ----------------------------------------------------------------- package
phase('Package')
const commit = await agent(`${PREAMBLE.replace('do NOT commit, ', '')}
YOU are the commit agent:
1. Read ~/Documents/repositories/seahaven/engineering-handbook/commit-messages.md
and follow it exactly.
2. git add only paths under: ${EXPECTED_SCOPE.join(', ')} and
.claude/workflows/phase-3-shared-extraction.js. NOT .coverage, NOT
package/. Verify the staged set with git status — the deletions of the
moved originals MUST be staged too.
3. ONE commit; write the message to /tmp/phase3-commit-msg.txt and use
git commit -F /tmp/phase3-commit-msg.txt (backticks in -m get eaten by
zsh). Suggested subject:
"feat: extract lambdas/shared/ — single-source ses_auth, web_ui auth, email parsing, EMF emitter (refactor phase 3)"
Body: the four moves with the byte-identity evidence one-liner for
ses_auth, the flat-landing import rule, the shared cp bundling change +
web_ui staging, the derived-agreement emitter decision (constraint 6),
and the test-plumbing summary. NO AI attribution / Co-Authored-By
lines.
4. Do NOT push. Return commit sha + shortstat in summary.`,
{ label: 'package:commit', model: 'sonnet', phase: 'Package', schema: IMPL })
return {
status: 'BUILT — committed locally, NOT pushed',
branch: BRANCH,
base: BASE,
commit: commit ? commit.summary : 'commit agent died — commit manually',
spec: { sharedModules: spec.sharedModules, webUiStaging: spec.webUiStaging, emfDesign: spec.emfDesign, derivedAgreementNote: spec.notes },
parityEvidence: parity ? { po: parity.poVerdict, wo: parity.woVerdict, webUi: parity.webUiVerdict } : null,
implementation: implOk.map(r => r.summary),
filesChanged: implOk.flatMap(r => r.filesChanged),
verifyRounds: round + 1,
blockers: implBlockers.concat(reconBlockers),
outstandingGates: [
'/sh-security-review (MANDATORY before push — auth code moved: ses_auth + the web_ui auth gate are exactly the authentication surface; run on the committed diff, and re-run after any post-review fix to this code)',
'cross-family cross_review.py NOT mandatory (no IAM change; handler event/return contracts unchanged — internal module moves only). Opt-in if judgment says so',
'push + PR + gh pr checks green',
'deploy-then-merge: deploy from branch, smoke green, one real PO + WO email each, watch BOTH sender-auth-rejected alarms through live mail (the auth move must not change accept/reject behavior), verify web_ui login still works, THEN merge — and note the post-merge CI redeploy being a no-op (unchanged asset hashes) is itself a verification signal',
],
}