diff --git a/docs/provisioning/P3-LIVE-FLIP-PLAN.md b/docs/provisioning/P3-LIVE-FLIP-PLAN.md index 781c078..9f5c68e 100644 --- a/docs/provisioning/P3-LIVE-FLIP-PLAN.md +++ b/docs/provisioning/P3-LIVE-FLIP-PLAN.md @@ -40,10 +40,11 @@ The box never gains a write token. | `/sh-plan-review` on THIS plan | adversarial plan audit before build | me → GPT-4.1 | | `/sh-security-review` on the apply/verify CI surface | auth + untrusted-input + CI trust boundary | me | | GPT-4.1 cross-family review on the apply/verify CI + any permission change | mandatory for the trust-boundary / permissions surface | orchestrator | -| `GH_TOKEN`→`GITHUB_TOKEN` resolved | the GitHub transport reads `GITHUB_TOKEN`; box has `GH_TOKEN` | me (folded in here) | +| ~~`GH_TOKEN`→`GITHUB_TOKEN` resolved~~ ✅ DONE 2026-06-22 | transport reads `GITHUB_TOKEN`; box now has a `GITHUB_TOKEN` alias of `GH_TOKEN` in `~/secrev.env` | me | The flip does NOT proceed until `/sh-security-review` AND the GPT-4.1 cross-review on -the CI surface both pass. +the CI surface both pass — **and these gates are re-run against the ACTUAL enabled +workflow (Phase 3), not only the inert version** (see Phase 3). > **Plan-review disposition (GPT-4.1 cross-family, 2026-06-22 — REQUEST CHANGES).** Findings > folded into §4 and Phases 1/1b: expanded denylist vectors, runner-trust, concrete @@ -55,6 +56,21 @@ the CI surface both pass. > them rather than rebuilding. Open QUESTIONs to answer when building: how human reviewers are > notified of new draft PRs, and how "every deployable repo has CI" is enforced for targets. +> **Plan-review disposition — ROUND 2 (GPT-4.1 cross-family, 2026-06-22 eve — REQUEST CHANGES).** +> Re-review after the durable-intake work merged (#32) and the coordinator went live. Six BLOCK +> items folded into the phases below: (B1) deploy-before-merge made a CONCRETE CI-enforced gate, +> not prose (Phase 1 + Phase 3); (B2) rollback added for a *prematurely-flipped privileged job that +> RUNS* — not only an accidental merge (Phase 1b); (B3) rollback for ALL privileged surfaces is a +> tested script, exercised, not one-time manual (Phase 1b); (B4) Phase 3 is explicitly gated on +> Phase 2 being fully provisioned + verified (Phase 3 precondition); (B5) `/sh-security-review` + +> GPT-4.1 cross-review are RE-RUN on the actual *enabled* workflow before the flip (Phase 3); (B6) +> docs/memory updated INCREMENTALLY at each privileged step, not deferred to Phase 6. FIX/NIT also +> folded: periodic review of gate-weakening patterns, a check-name-discovery test across targets, +> a memory update when denylist vectors change, snapshot retention in Phase 0, and the draft-PR +> notification QUESTION assigned in Phase 4. (Findings-to-verify, not gospel: the reviewer +> under-read §3 — cross-review IS already a hard gate; B5 is the genuine delta = re-run on the +> *enabled* file.) + ## 4. The CI trust boundary (design B4 — what the workflow must enforce) > **Already implemented in the merged P3-live CI hardening (PR #17) — Phase 1 VERIFIES, does not rebuild:** @@ -102,9 +118,11 @@ the CI surface both pass. ## 5. Phases ### Phase 0 — Plan review & pre-reqs 🤖/🧑 -- [ ] Run `/sh-plan-review` on this doc; fold BLOCK/FIX items in. +- [x] Run `/sh-plan-review` on this doc; fold BLOCK/FIX items in. (Round 1 + Round 2 done; this doc is the result.) - [ ] Confirm a clean revert point (git tag main; Hyper-V snapshot of sh-secrev). -- [ ] Resolve `GH_TOKEN`→`GITHUB_TOKEN` (transport accepts both / box env updated). + **Snapshot retention:** keep the pre-P3 snapshot until P3 has run clean for one full + cycle (Phase 4 DoD), then prune — recorded here so it is not an open-ended snapshot. +- [x] Resolve `GH_TOKEN`→`GITHUB_TOKEN` (box `~/secrev.env` now has a `GITHUB_TOKEN` alias). - **Rollback:** none (no state changed). ### Phase 1 — Author/verify the split-CI apply/verify workflow 🤖 (review-gated) @@ -121,22 +139,60 @@ the CI surface both pass. anti-replay; document and test it. - [ ] **Gate-weakening detector** (§4.5): CI step that fails on a diff adding `noqa`/`type: ignore`/skip/xfail/excludes/`--no-verify` or editing the gate config. + The detector's pattern list is **reviewed/expanded each time a new bypass vector is + found** (FIX) — record the list in code with a comment pointing here, and update it + + memory when a vector is added (same discipline as the denylist below). - [ ] **PR-metadata sanitization** (§4.6) and **ledger anti-tamper** (§4.7) implemented + tested. - [ ] `agent_team/ci_fetcher.py` (read-only Checks-API fetcher; fails closed) + `ci_gate.py` (pure-code green). Add a mechanism for the gate to **discover the correct - required check names per repo/branch** (avoid hardcoded check-name drift across repos). -- [ ] **Deploy-before-merge enforcement:** a documented/CI gate ensuring the privileged flip - is exercised on the box before the workflow change is merged (Sea Haven deploy-then-merge). -- [ ] **Concrete "no write token on the box" audit** (a test/script, not just a claim). + required check names per repo/branch** (avoid hardcoded check-name drift across repos), + **with a test that exercises discovery against every intended target repo/branch** (FIX). +- [ ] **Memory/doc update when denylist vectors change** (FIX): adding a denylist vector (here + or later) updates `project_r720_agent_team` memory + the Confluence host page in the SAME + change — the denied set is operational/security-critical, not tribal knowledge. +- [ ] **Deploy-before-merge enforcement — CONCRETE, CI-enforced (B1). REQUIRED Phase-1 + deliverable; the flip does not proceed without it (no manual fallback).** "Deploy" of this + change = the privileged path is *proven on the live box before the workflow PR merges*. + Build, in this phase: + - (a) the box exercises a **dispatch dry-run** of the apply/verify workflow (privileged steps + still `if:${{ false }}`) that emits a signed "exercised-on-box" artifact/status. **Dry-run + faithfulness (NIT):** it runs the SAME workflow file and the SAME untrusted build/verify + + pure-code-gate jobs as the live path — ONLY the privileged `if:` differs — so the dry-run + is a faithful proof of the path, not a separate mock. + - (b) a **required status check** on the workflow-file PR consumes that proof, so the PR is + un-mergeable until the box has run it. + - (c) branch-protection set so the required checks **cannot be bypassed by admins** ("do not + allow bypassing the above settings" / include-administrators) — **no `--admin` merge of the + privileged flip**. Applied in Phase 2; **no ordering window** because the flip (Phase 3) is + gated on Phase 2 completion (the B4 precondition), so admin-bypass is already disabled before + any flip is possible. The Phase-1 required-status-check (a/b) and the Phase-2 branch-protection + (c) together are the gate; the flip cannot happen until BOTH are in place. + This is the gate; until it is built+green, the flip is blocked. (Owner: 🤖 build; verified in + the Phase-1 `/sh-security-review` + cross-review hard stop. The check's implementation is itself + a Phase-1 build task — that it is not yet physically built is expected for a pre-build plan; what + matters is it is non-optional and flip-blocking, enforced at the Phase-1 hard stop.) +- [ ] **Concrete "no write token on the box" audit (B-QUESTION → a real check):** a + test/script asserting the box env + coordinator config hold no `pull-requests:write` / + contents-write token (grep the live env names + assert the App token is only an Actions + secret), runnable on the box and in CI. Not a prose claim. - [ ] `/sh-security-review` + GPT-4.1 cross-review on this surface. **Hard stop until both pass.** + (NOTE: these are RE-RUN on the *enabled* workflow in Phase 3 — see B5 there.) - **Rollback:** workflow file stays inert (`if: ${{ false }}` not yet flipped); delete the file. ### Phase 1b — Recovery for an accidentally-merged/applied privileged change 🤖/🧑 -- [ ] Document + **exercise once** a rollback for the case where a privileged change (the - apply/verify workflow, the `agent-apply` environment, the GitHub App perms, or - branch-protection) is merged or applied in error: revert the SHA, rotate the GitHub App - token, restore branch-protection/environment from a recorded baseline, and confirm no - draft-PR apply ran in the window. (The plan previously only covered reverting inert files.) +- [ ] **Rollback as a TESTED SCRIPT covering ALL privileged surfaces (B3)** — not a one-time + manual exercise. One scripted, re-runnable rollback that, per surface, restores from a + recorded baseline: (1) the apply/verify **workflow** (revert the SHA → inert), (2) the + **`agent-apply` environment** (required-reviewer + protection rules), (3) the **GitHub App + permissions/installation** (rotate token, reduce/uninstall), (4) **branch protection**. + The script asserts the post-restore state matches the baseline. Exercised in Phase 3 + rollback AND re-runnable on demand. +- [ ] **Premature-flip-that-RAN rollback (B2).** Distinct from an accidental *merge*: cover the + case where `if:${{ false }}` is flipped early (or the environment gate is misconfigured) and + the privileged job **actually runs** — incident steps: rotate the GitHub App token + immediately, close/revert any draft PR (or branch) it opened, confirm via the Checks/PR + audit trail exactly what ran in the window, restore the environment + branch protection from + baseline, and file the incident. This is the "it executed" path, not just "it merged." - [ ] Add light **monitoring on draft-PR creation rate** (runaway-volume alarm) and an **orphaned/stale draft-PR cleanup** step. @@ -149,14 +205,27 @@ the CI surface both pass. - **Rollback:** uninstall the App; delete the environment + secrets. ### Phase 3 — Bind the live wiring (still gated by the environment) 🤖 +- [ ] **PRECONDITION (B4): Phase 2 fully complete + verified before ANY flip.** Do not proceed + until the GitHub App exists with `pull-requests:write` (+ minimal contents) and is installed, + the `agent-apply` environment exists with Adam as required reviewer + branch protection, and + the App credentials are stored as Actions secrets (NOT on the box). Verify each before the + next step; the flip is blocked otherwise. - [ ] In the workflow: uncomment `permissions: pull-requests: write` and `environment: agent-apply`; flip the two `if: ${{ false }}` → enabled. +- [ ] **RE-RUN BOTH GATES ON THE ENABLED WORKFLOW (B5).** `/sh-security-review` + the GPT-4.1 + cross-family review are run again against the *actual enabled* `agent-team-apply-verify.yml` + (permissions live, `if:` true) and the bound `gated_build_verify_wiring` — NOT only the inert + Phase-1 version. **Hard stop until both pass on the enabled file.** (Permissions changed → + the mandatory cross-family review is independently required here against the real diff.) - [ ] Bind `agent_team.coordinator.gated_build_verify_wiring(...)` (real diff builder + read-only CI-result fetcher) so a leaf calls it only **after** the gate clears. - [ ] Set the box-side apply env vars the live path reads (read-only CI-result token + - dispatch target). Confirm **no** write token lands on the box. -- **Rollback:** re-set `if: ${{ false }}`, re-comment `environment:`, set - `build_verify_wiring=None`; restart the coordinator. (Exercise this rollback once.) + dispatch target). Confirm **no** write token lands on the box (run the Phase-1 no-write-token audit). +- [ ] **Incremental docs (B6):** update `OPERATOR-RUNBOOK.md` + memory NOW that the flip is live + (do not wait for Phase 6) — what the apply path can/can't do, the denylist, the rollback. +- **Rollback:** run the Phase-1b tested rollback script (re-set `if: ${{ false }}`, re-comment + `environment:`, set `build_verify_wiring=None`, restart the coordinator). **Exercise it once + here** to prove it works before relying on it. ### Phase 4 — Smoke test to a first draft PR 🧑/🤖 - [ ] Drive one trivial, in-scope task end-to-end → confirm: untrusted job builds/tests with @@ -164,16 +233,27 @@ the CI surface both pass. results, privileged job opens a **draft PR** with required checks attached, **nothing merged**. - [ ] Flip the **Tier-3 fixer** off `--dry-run` only after the smoke test passes; verify a dependency-CVE bump produces a draft PR. +- [ ] **Draft-PR reviewer notification (QUESTION resolved/assigned):** decide + wire how a new + draft PR pings the human reviewer — default = the existing Slack ALARM path posts a + `#agent-team` notice with the PR link (Adam owns this decision; recorded so it is not left open). - **Rollback:** close the draft PR; Phase-3 rollback. ### Phase 5 — Enable the cross-plane loop 🤖 - [ ] Allow confirmed Plane-1 checker findings (`intake-checker`) to flow into pipeline tasks that end in draft-PR fixes (start conservative: highest-severity, one at a time). +- [ ] **Conservative-rollout controls (QUESTION resolved):** enforce "one at a time, highest + severity" concretely — a cap on concurrently-open cross-plane draft PRs (start = 1) read by + the intake-checker leaf, the severity floor in config, and a `#agent-team` log line per + promotion so the rate is observable. Not just prose. - **Rollback:** revert intake-checker wiring to report-only. -### Phase 6 — Docs & memory 🤖 -- [ ] Update `OPERATOR-RUNBOOK.md`, the Confluence host page, and memory: P3 is LIVE, - what the apply path can/can't do, the denylist, the merge gate, the rollback. +### Phase 6 — Final docs & memory consolidation 🤖 +> **B6: docs/memory are updated INCREMENTALLY at each privileged step above** (Phase 3 runbook+memory +> on flip; denylist/gate-weakening changes update memory in the same change). Phase 6 is the FINAL +> reconciliation pass, not the first time docs are touched. +- [ ] Reconcile `OPERATOR-RUNBOOK.md`, the Confluence host page (20054017) + runbook (22609921), + and `project_r720_agent_team` memory: P3 is LIVE, what the apply path can/can't do, the + denylist, the merge gate, the rollback — confirming the incremental updates are complete + consistent. ## 6. What changes (and what does NOT)