export const meta = { name: 'phase-2-bundling-root', description: 'Phase 2 of the procurement-ingest refactor (docs/refactor-evaluation.md): widen both email-processor asset roots to ../lambdas (making lambdas/shared/ reachable for Phase 3) with scoped cp globs, add exclude lists (from_asset ignores .gitignore — the stale 44 MB package/ dir would poison asset hashes), and add __pycache__ excludes to the three plain from_asset calls. ZERO code change under lambdas/ — the workflow proves bundle parity by diffing the locally-synthed staged asset against the currently-deployed zip. Committed locally, never pushed.', phases: [ { title: 'Setup', detail: 'verify Phases 0+1 merged into base, branch feature/phase-2-bundling-root', model: 'haiku' }, { title: 'Recon', detail: '3 mappers: all five from_asset sites, deployed zip file lists (read-only AWS), AST-test shapes' }, { title: 'Spec', detail: 'serial opus spec: exact new bundling commands, exclude lists, and the parity-comparison rules', model: 'opus' }, { title: 'Implement', detail: 'sonnet per stack file + sonnet for AST-test/README updates — disjoint files', model: 'sonnet' }, { title: 'Verify', detail: 'mechanical gates + staged-vs-deployed parity + determinism + 2 fable lenses' }, { title: 'Fix', detail: 'opus fixer, full re-verify, max 3 rounds', model: 'opus' }, { title: 'Package', detail: 'single commit via -F (no push)', model: 'sonnet' }, ], } // ---------------------------------------------------------------- constants const REPO = '/Users/adammoussa/Documents/repositories/seahaven/procurement-ingest' const BRANCH = 'feature/phase-2-bundling-root' const BASE = (args && args.base) || 'main' const CONSTRAINTS = ` PINNED BEHAVIORAL CONSTRAINTS (docs/refactor-evaluation.md Phase 2 — violating any is a build failure): 1. ZERO diff under lambdas/: git diff ${BASE}...HEAD -- lambdas/ must be EMPTY. This phase moves NO code — only cdk/, tests/test_bundle_consistency.py, README.md (and docs/ if needed) may change. 2. Both email processors: Code.from_asset("../lambdas") with the bundling command rewritten for the new cwd (bundling cwd IS the asset root): pip install ... -r po/email_processor/requirements.txt -t /asset-output && cp po/email_processor/*.py /asset-output/ (resp. wo/email_processor). The -r path change is REQUIRED. PRESERVE the pip "--platform manylinux2014_aarch64 --only-binary=:all:" pin exactly — its removal shipped x86 wheels into the ARM64 function and caused a 100% outage (PR #34); it is non-negotiable. 3. Both widened from_asset calls get exclude=['**/__pycache__/**', '**/tests/**', '**/package/**']. from_asset does NOT honor .gitignore; without the package/ exclude the stale untracked 44 MB lambdas/po/email_processor/package/ dir diverges LOCAL vs CI asset hashes (CI never has it) and forces spurious redeploys. 4. WO prod-zip shrinkage is DELIBERATE cleanup, not an accident to fix: the current 'cp -r .' ships tests/ (real scrubbed .eml fixtures), __pycache__, and requirements.txt in the production zip. The acceptance criterion for WO is "runtime-imported module set unchanged + smoke", NOT byte-identical zip. Byte-identical first-party file set applies to PO ONLY. Document the shrinkage explicitly (list every file class that drops out). 5. The three plain from_asset calls (po web_ui, po site_extractor, wo web_ui — locate them, line numbers have shifted since the doc) each gain exclude=['**/__pycache__/**'] so local __pycache__ stops making their asset hashes nondeterministic. Nothing else about them changes. 6. tests/test_bundle_consistency.py must be updated IN THE SAME CHANGE — it deliberately change-detects both bundling commands and will fail red on the new shapes. Update it to recognize the scoped glob 'cp po/email_processor/*.py' (resp. wo) as the new unconditionally-safe shape, WITHOUT loosening it: the allowlist-revert detection, the detection-logic mutation test, and the PO_EXPECTED_TOP_LEVEL_MODULES exact-set pin must all keep their teeth. A bare-substring match that any commented-out glob satisfies is a regression. 7. cdk diff on BOTH stacks must show ONLY the two functions' Code/asset property changes (+ CDK metadata). No logical-ID changes, no alarm, IAM, table, env, or function-config deltas of any kind. 8. Do NOT delete lambdas/po/email_processor/package/ (untracked, local-only; deletion is Adam's call — with the exclude in place it is hash-neutral). Do NOT touch requirements.txt contents (Phase 7 scope). 9. AWS access is READ-ONLY (get-function, downloading the deployed zip via its presigned URL). NEVER cdk deploy, never invoke, never mutate. ` const PREAMBLE = ` You are one of several agents building refactor Phase 2 in the git repo at ${REPO} on branch ${BRANCH} (already checked out — do NOT switch branches, do NOT create branches, do NOT commit, NEVER push). Authoritative spec: docs/refactor-evaluation.md, section "Phase 2". Work ONLY in the files you are told you own; other agents are concurrently editing other files in this same working tree. ${CONSTRAINTS} Your final message is consumed by an orchestrator script, not a human — return only the structured data requested. ` // ------------------------------------------------------------------ schemas const RECON = { type: 'object', required: ['summary', 'facts'], properties: { summary: { type: 'string' }, facts: { type: 'array', items: { type: 'string' } }, blockers: { type: 'array', items: { type: 'string' } }, }, } const SPEC = { type: 'object', required: ['poBundling', 'woBundling', 'plainAssetEdits', 'astTestChanges', 'parityRules', 'notes'], properties: { poBundling: { type: 'string', description: 'the complete new po_stack.py from_asset block: asset path, full command string, exclude list — exact code' }, woBundling: { type: 'string', description: 'same for wo_stack.py' }, plainAssetEdits: { type: 'string', description: 'the three plain from_asset calls: current file:line + exact exclude addition each' }, astTestChanges: { type: 'string', description: 'exactly how test_bundle_consistency.py recognizes the new scoped-glob shapes without losing revert detection; which assertions/regexes change and to what' }, parityRules: { type: 'string', description: 'the staged-asset vs deployed-zip comparison rules: strict first-party .py set equality for PO; WO expected-shrinkage list + runtime-module retention; how dependency version drift is tolerated' }, notes: { type: 'string' }, }, } const IMPL = { type: 'object', required: ['filesChanged', 'summary', 'checksRun'], properties: { filesChanged: { type: 'array', items: { type: 'string' } }, summary: { type: 'string' }, checksRun: { type: 'string' }, blockers: { type: 'array', items: { type: 'string' } }, }, } const CHECKS = { type: 'object', required: ['passed', 'details'], properties: { passed: { type: 'boolean' }, details: { type: 'string' }, scopeViolations: { type: 'array', items: { type: 'string' } }, }, } const PARITY = { type: 'object', required: ['passed', 'poVerdict', 'woVerdict', 'details'], properties: { passed: { type: 'boolean' }, poVerdict: { type: 'string', description: 'PO staged vs deployed: identical first-party set? full evidence' }, woVerdict: { type: 'string', description: 'WO: runtime modules retained + exact shrinkage list' }, determinism: { type: 'string', description: 'repeat-synth + injected-__pycache__ hash results' }, details: { type: 'string' }, }, } const FINDINGS = { type: 'object', required: ['findings'], properties: { findings: { type: 'array', items: { type: 'object', required: ['title', 'severity', 'confirmed', 'evidence', 'fix'], properties: { title: { type: 'string' }, severity: { enum: ['critical', 'high', 'medium', 'low'] }, confirmed: { type: 'boolean' }, evidence: { type: 'string' }, fix: { type: 'string' }, }, }, }, }, } // ------------------------------------------------------------------- setup phase('Setup') const setup = await agent(` In ${REPO}: 1. SEQUENCING GATE — Phases 0 AND 1 must be merged into ${BASE} (Phase 1 also edits cdk/po_stack.py; building Phase 2 against a pre-Phase-1 stack file guarantees a conflict). git fetch origin, then verify on origin/${BASE}: (a) the healthcheck branch exists in both handlers and tests/test_bundle_consistency.py exists (Phase 0); (b) validate_ai_fallback exists in the PO pipeline and po_stack.py contains the ai-fallback-rejected alarm (Phase 1). If either is missing, STOP with a blocker naming the unmerged phase and do nothing else. 2. Verify clean tree (untracked .coverage/.claude/ fine; other dirt = blocker, never stash or discard). 3. git checkout ${BASE} && git pull --ff-only && git checkout -b ${BRANCH} 4. gh pr list --state open --json number,title,headRefName Return: HEAD sha, gate evidence, open PRs, blockers. `, { label: 'setup:branch', model: 'haiku', schema: RECON }) if (!setup || (setup.blockers && setup.blockers.length)) { return { aborted: 'setup blockers', blockers: setup ? setup.blockers : ['setup agent died'], facts: setup ? setup.facts : [] } } log(`Branch ${BRANCH} ready off ${BASE}. ${setup.summary}`) // ------------------------------------------------------------------- recon phase('Recon') const recon = await parallel([ () => agent(`${PREAMBLE} Read-only recon of ALL FIVE Code.from_asset sites in cdk/po_stack.py and cdk/wo_stack.py: for each, quote verbatim with current file:line — the asset path, full bundling command (if bundled) or plain form, and any existing exclude. Also: both email_processor requirements.txt paths + contents (the pip -r path must be rewritten); the exact set of top-level .py files in lambdas/po/email_processor and lambdas/wo/email_processor (non-recursive); every subdirectory of each (tests/, package/, __pycache__ presence); and whether lambdas/ contains anything else a widened ../lambdas asset root would stage (shared/ existing yet? stray files at lambdas/ top level?). 15-25 precise facts.`, { label: 'recon:asset-sites', model: 'sonnet', phase: 'Recon', schema: RECON }), () => agent(`${PREAMBLE} Read-only AWS recon (region us-east-1, READ-ONLY): for po-email-processor and workorder-email-processor run aws lambda get-function, download each Code.Location presigned zip to a scratch dir, and produce the COMPLETE file list of each deployed zip (unzip -l), separating (a) first-party top-level .py files, (b) pip-installed dependency dirs (name + version from .dist-info), (c) everything else (tests/, fixtures, __pycache__, requirements.txt — expected in the WO zip today). Also record each function's CodeSha256. This is the parity baseline the new bundles are judged against. Return the categorized lists as facts.`, { label: 'recon:deployed-zips', model: 'sonnet', phase: 'Recon', schema: RECON }), () => agent(`${PREAMBLE} Read-only recon of tests/test_bundle_consistency.py: quote the exact regexes/assertions that will change — the PO executed-glob pin, the WO recursive-copy pin, _bundling_ships_all's two unconditionally-safe shapes, _extract_bundling_command's exactly-one-command demand (both stacks still have exactly one bundled function after Phase 2 — confirm), the PO_EXPECTED_TOP_LEVEL_MODULES pin, and the mutation test. State which assertions fail red against the Phase 2 command shapes (expected: PO glob pin and WO cp -r pin both fail). 8-15 facts.`, { label: 'recon:ast-test', model: 'haiku', phase: 'Recon', schema: RECON }), ]) const reconOk = recon.filter(Boolean) const pack = reconOk.map(r => `## ${r.summary}\n${r.facts.join('\n')}`).join('\n\n') const reconBlockers = reconOk.flatMap(r => r.blockers || []) log(`Recon complete: ${reconOk.length}/3 mappers, ${reconBlockers.length} blockers`) // -------------------------------------------------------------------- spec phase('Spec') const spec = await agent(`${PREAMBLE} You are the SPEC agent — pin every exact string before parallel implementation. Using the recon pack and your own reads, produce: - poBundling / woBundling: the complete replacement from_asset blocks as exact code — asset path "../lambdas", full bash -c command (pip line with rewritten -r path and the preserved manylinux2014_aarch64 pin, then the scoped non-recursive glob cp), exclude list per constraint 3. Mind the bundling cwd semantics: the container mounts the ASSET ROOT (../lambdas) as the working dir, so all paths are relative to lambdas/. - plainAssetEdits: the three plain from_asset calls with exact exclude additions. - astTestChanges: precise edits to test_bundle_consistency.py — the new scoped-glob regex (must match 'cp po/email_processor/*.py /asset-output/' as executed, still comment-strip-proof, still reject narrowed or commented-out variants), what replaces the WO cp -r pin, and confirmation the mutation test + exact-set pin survive unchanged in spirit. - parityRules: exactly how Verify judges the new bundles against the deployed-zip baseline from recon: PO = first-party top-level .py set must be IDENTICAL; dependency packages compared by NAME (version drift from the floor-pinned boto3 is tolerated and noted, not failed); nothing new may appear. WO = every first-party module in the CURRENT deployed zip that the handler imports (cross-check the AST sibling list) must be retained; expected drop list = tests/ + fixtures + __pycache__ + requirements.txt and NOTHING else may silently vanish; nothing new may appear. Recon pack:\n${pack}`, { label: 'spec:pin-strings', model: 'opus', phase: 'Spec', schema: SPEC }) if (!spec) return { aborted: 'spec agent died — rerun workflow', reconBlockers } const specBlock = `BINDING SPEC (implement EXACTLY this):\n${JSON.stringify(spec, null, 2)}` log('Spec pinned: bundling commands, excludes, AST-test edits, parity rules') // --------------------------------------------------------------- implement phase('Implement') const impl = await parallel([ () => agent(`${PREAMBLE} YOU OWN: cdk/po_stack.py ONLY. Apply spec.poBundling (email processor from_asset: root, command, exclude) and the po web_ui + site_extractor exclude additions from spec.plainAssetEdits. Touch NOTHING else in the file (alarms, IAM, tables are all off-limits). Preserve/adapt the bundling warning comment so it stays true. Run before returning: ruff check cdk && cd cdk && npx cdk synth po-ingest -q (this runs Docker bundling — confirm the staged asset in cdk.out contains exactly the expected files and note the asset hash in your summary). ${specBlock}`, { label: 'impl:po-stack', model: 'sonnet', phase: 'Implement', schema: IMPL }), () => agent(`${PREAMBLE} YOU OWN: cdk/wo_stack.py ONLY. Apply spec.woBundling and the wo web_ui exclude from spec.plainAssetEdits. Touch nothing else. Run before returning: ruff check cdk && cd cdk && npx cdk synth workorder-ingest -q (artifact-id selector, NOT stack_name) — confirm staged asset contents + hash in your summary. ${specBlock}`, { label: 'impl:wo-stack', model: 'sonnet', phase: 'Implement', schema: IMPL }), () => agent(`${PREAMBLE} YOU OWN: tests/test_bundle_consistency.py and README.md ONLY. Task A: apply spec.astTestChanges. The test must PASS against the new stack files and still FAIL against: a reverted filename allowlist missing a sibling, a commented-out glob, and a narrowed glob (add/adjust the mutation tests to cover the new shapes — e.g. 'cp po/email_processor/handler.py' alone must not satisfy the scoped-glob pin). Task B: README — update the Deploy-Pipeline Guards bundling paragraph and the CI/CD + repo-layout sections for the widened ../lambdas asset root; document the deliberate WO prod-zip shrinkage (what dropped and why that is cleanup, per constraint 4) and the exclude rationale (from_asset ignores .gitignore; package/ hash poisoning). Run before returning: pytest tests/test_bundle_consistency.py -q --no-cov (must be green against the OTHER agents' stack edits — if they have not landed yet, poll by re-running up to ~10 min before reporting a blocker), ruff check tests. ${specBlock}`, { label: 'impl:ast-test-readme', model: 'sonnet', phase: 'Implement', schema: IMPL }), ]) const implOk = impl.filter(Boolean) const implBlockers = implOk.flatMap(r => r.blockers || []) log(`Implement complete: ${implOk.length}/3 agents, blockers: ${implBlockers.length}`) // ---------------------------------------------------- verify + fix loop const EXPECTED_SCOPE = [ 'cdk/po_stack.py', 'cdk/wo_stack.py', 'tests/test_bundle_consistency.py', 'README.md', ] const mechanicalPrompt = `${PREAMBLE} Independent re-verification — trust nothing self-reported. Run ALL gates, quoting failures verbatim: 1. pytest -q --no-cov (repo root) 2. ruff check . && ruff format --check . 3. cd cdk && npx cdk synth po-ingest -q && npx cdk synth workorder-ingest -q 4. cd cdk && npx cdk diff po-ingest ; npx cdk diff workorder-ingest — the ONLY resource deltas allowed are the two email-processor functions' Code/S3Key (asset hash) changes + CDK metadata. Any alarm/IAM/env/config/ logical-ID delta = FAIL (constraint 7). Paste the diff summaries. 5. ZERO-CODE-MOVE invariant: git diff ${BASE}...HEAD -- lambdas/ must output NOTHING. 6. git status scope: every change under ${EXPECTED_SCOPE.join(', ')} only (untracked .coverage/.claude/ tolerated). passed=true only if all green. YOU MAY NOT edit files.` const parityPrompt = `${PREAMBLE} You are the ARTIFACT-PARITY verifier — the load-bearing gate of this phase. Everything is local synth + read-only AWS. 1. cd cdk && npx cdk synth po-ingest -q -o /tmp/phase2-synth && npx cdk synth workorder-ingest -q -o /tmp/phase2-synth. Locate each email processor's staged bundled asset dir in /tmp/phase2-synth (the asset.* dir containing handler.py). 2. Re-download both deployed zips (aws lambda get-function Code.Location, region us-east-1) fresh — do not trust a recon cache. 3. Judge per the spec parityRules: PO first-party top-level .py set staged vs deployed IDENTICAL (list both sets); dependency packages by name with version drift noted; NOTHING new. WO: every AST-derived handler sibling retained; the drop list is EXACTLY tests//fixtures/__pycache__/ requirements.txt-class files — enumerate every dropped path class and every added path; anything outside the expected classes = FAIL. 4. DETERMINISM: run the po-ingest synth twice into two fresh -o dirs — asset hashes must be identical. Then create a throwaway __pycache__/junk.pyc under lambdas/po/web_ui/ AND a dummy file under lambdas/po/email_processor/package/ (create the dir if absent, remember to DELETE everything you created afterwards), re-synth, and confirm NO asset hash changed (proves the excludes + the package/ hash-poisoning fix actually work). Report before/after hashes. 5. Sanity: the staged PO asset must NOT contain web_ui/site_extractor sources, tests/, package/, or any .eml — the widened root stages more context but the cp glob + excludes must keep the OUTPUT clean. passed=true only if every check holds.` const lenses = [ { key: 'bundling-semantics', prompt: `${PREAMBLE} ADVERSARIAL REVIEW — bundling-semantics lens. Read the full cdk diff (git diff ${BASE}) plus aws-cdk-lib's documented from_asset/BundlingOptions semantics and try to REFUTE correctness: (1) is the bundling container cwd really the widened asset root, making 'pip install -r po/email_processor/requirements.txt' and 'cp po/email_processor/*.py' resolve correctly — or does an image workdir default break it? (2) do the exclude patterns ('**/__pycache__/**' etc.) apply to ASSET STAGING (hash input) and not merely the docker mount — i.e. does the package/ exclude actually stop local-vs-CI hash divergence? (3) does exclude affect what the bundling container can SEE, and could excluding tests/ break the pip install or cp step? (4) non-recursive scoped glob: could bash glob expansion inside the container (sh vs bash, nullglob) change behavior vs the Phase 0 top-level glob? (5) does widening the asset root change the ASSET HASH INPUTS such that unrelated wo/ file edits now redeploy the PO function (and vice versa) — if so, state the blast radius plainly so it is documented, not discovered. confirmed=true only with concrete evidence (CDK docs/source or a reproduced synth).` }, { key: 'guard-integrity', prompt: `${PREAMBLE} ADVERSARIAL REVIEW — guard-integrity lens. The Phase 0 guards must come through Phase 2 with their teeth intact. (1) Attack the updated test_bundle_consistency.py: does the new scoped-glob regex accept a commented-out glob, a narrowed single-file cp, a glob for the WRONG pipeline dir (cp wo/email_processor/*.py in po_stack), or a cp missing /asset-output? Does the WO assertion still reject a narrowed recursive copy? Run the test against hand-mutated command strings to prove each rejection (scratch copies, not repo edits). (2) Does PO_EXPECTED_TOP_LEVEL_MODULES still bound what ships (the glob is now scoped — is the pin still checking the right directory)? (3) The smoke script + healthcheck are untouched by this diff — confirm zero diff to scripts/ and lambdas/. (4) README claims about bundling must match the new reality — no stale Phase 0 text contradicting the widened root. confirmed=true only with file:line evidence.` }, ] let round = 0 let checks = null let parity = null let confirmed = [] while (round < 3) { phase('Verify') const results = await parallel([ () => agent(mechanicalPrompt, { label: `verify:mechanical-r${round}`, model: 'sonnet', phase: 'Verify', schema: CHECKS }), () => agent(parityPrompt, { label: `verify:parity-r${round}`, model: 'opus', phase: 'Verify', schema: PARITY }), ...lenses.map(l => () => agent(l.prompt, { label: `verify:${l.key}-r${round}`, phase: 'Verify', schema: FINDINGS })), ]) checks = results[0] parity = results[1] confirmed = results.slice(2).filter(Boolean) .flatMap(r => r.findings || []) .filter(f => f.confirmed && f.severity !== 'low') const green = checks && checks.passed && parity && parity.passed log(`Verify round ${round}: mechanical ${checks && checks.passed ? 'GREEN' : 'RED'}, parity ${parity && parity.passed ? 'GREEN' : 'RED'}, confirmed findings: ${confirmed.length}`) if (green && confirmed.length === 0) break round += 1 if (round >= 3) break phase('Fix') await agent(`${PREAMBLE} You are the fix agent — you may edit files under: ${EXPECTED_SCOPE.join(', ')}. Fix EVERY item minimally; the binding spec and 9 pinned constraints hold (a finding that conflicts with a constraint is reported, not "fixed"). Re-run the specific failing gate per fix. MECHANICAL:\n${checks ? checks.details : '(agent died — rerun all)'} PARITY:\n${parity ? parity.details : '(agent died — rerun all)'} CONFIRMED FINDINGS:\n${JSON.stringify(confirmed, null, 2)} ${specBlock}`, { label: `fix:round-${round}`, model: 'opus', phase: 'Fix', schema: IMPL }) } const verifyClean = checks && checks.passed && parity && parity.passed && confirmed.length === 0 if (!verifyClean) { return { status: 'NEEDS ATTENTION — verify not clean after 3 rounds; branch left uncommitted', branch: BRANCH, mechanical: checks, parity, unresolvedFindings: confirmed, implBlockers, reconBlockers, } } // ----------------------------------------------------------------- package phase('Package') const commit = await agent(`${PREAMBLE.replace('do NOT commit, ', '')} YOU are the commit agent: 1. Read ~/Documents/repositories/seahaven/engineering-handbook/commit-messages.md. 2. git add only: ${EXPECTED_SCOPE.join(', ')} and .claude/workflows/phase-2-bundling-root.js. NOT .coverage. Verify staged set with git status. 3. ONE commit via git commit -F /tmp/phase2-commit-msg.txt (write the message file first; backticks in -m get eaten by zsh). Suggested subject: "feat: widen email-processor asset roots to lambdas/ with scoped globs + excludes (refactor phase 2)" Body: why (shared/ reachability for Phase 3), the WO shrinkage list, the package/ hash-poisoning exclude, parity evidence one-liner. NO AI attribution / Co-Authored-By lines. 4. Do NOT push. Return commit sha + shortstat.`, { label: 'package:commit', model: 'sonnet', phase: 'Package', schema: IMPL }) return { status: 'BUILT — committed locally, NOT pushed', branch: BRANCH, base: BASE, commit: commit ? commit.summary : 'commit agent died — commit manually', parityEvidence: parity ? { po: parity.poVerdict, wo: parity.woVerdict, determinism: parity.determinism } : null, implementation: implOk.map(r => r.summary), filesChanged: implOk.flatMap(r => r.filesChanged), verifyRounds: round + 1, blockers: implBlockers.concat(reconBlockers), outstandingGates: [ 'push + PR + gh pr checks green (no /sh-security-review required — no untrusted-input/auth/IAM surface; pre-push scanners still run; cross-family review not required — no IAM or handler-signature change)', 'deploy-then-merge: deploy from branch, then the LIVE gate — aws lambda get-function zip file-list diff (PO first-party set identical; WO shrinkage exactly as documented), smoke green, one real PO + WO email each with ParseMethod=template, alarms green, THEN merge', 'optional cleanup for Adam (hash-neutral once excludes are deployed): delete the untracked 44 MB lambdas/po/email_processor/package/ dir (doc Phase 7 item, endorsed to do with/after Phase 2)', ], }