This repository has been archived on 2026-08-04. You can view files and clone it, but cannot push or open issues or pull requests.
orchestrator/docs/provisioning/P3-LIVE-FLIP-PLAN.md

357 lines
27 KiB
Markdown
Raw Normal View History

# P3-LIVE-FLIP PLAN — agent-team build → verify → draft-PR
Formal phased plan to take the agent-team Plane-2 pipeline from **clarify+plan only**
to **producing reviewable draft PRs**, while keeping the always-on R720 box
read-only and the apply path zero-AWS.
**Status as of 2026-06-23: PARTIALLY LIVE.** The split-CI apply/verify workflow
and its provisioning (the scoped GitHub App, the `agent-apply` environment + Actions
secrets, and the dispatched runs) are **LIVE since 2026-06-22** — i.e. the
workflow-authoring + provisioning phases below (Phases 1/1b/2) are **DONE**. The
remaining work is the **box-side build → dispatch → verify integration** on
`feat/agent-team-p3-box-integration` (BUILD → DISPATCH → VERIFY wiring;
operator-initiated dispatch; run-name correlation for run_id capture; async
CI-watch resume-on-complete; fail-safe serve default) — see `docs/P3-PHASE0-DESIGN.md`
for the recorded design decisions. The two remaining **human gates** are: (C1) re-run
`/sh-security-review` + the GPT-4.1 cross-family review against the *enabled* workflow
+ the bound box-side wiring (Phase 3 / B5), and (D) deploy → smoke-test → merge
(Phase 4). This plan was the input to `/sh-plan-review` before the build began; that
loop is complete.
> **Prerequisite reading:** `docs/r720-agent-team-design.md` §3.3.2 (CI-as-verifier
> trust boundary, "B4"), `PROVISIONING-RUNBOOK.md` (the P3-live-flip section),
> and memory `project_r720_agent_team` (locked decisions).
---
## 1. Objective & current state
**Live infrastructure (since 2026-06-22):** the split-CI apply/verify workflow
(`agent-team/ci/agent-team-apply-verify.yml`) is authored, enabled, and **provisioned**
— the scoped **GitHub App** (`pull-requests:write` + minimal contents), the
`agent-apply` **GitHub Actions Environment** (Adam as required reviewer + branch
protection), and the App credentials as **Actions secrets** all exist, and dispatched
runs have executed against it. Org CI can build/test/security-review an untrusted diff
in a sandbox, a pure-code gate confirms green from authenticated Checks-API results,
and the privileged job opens a **draft PR** — all without the box ever holding a write
token.
**Not yet wired (the remaining build, on `feat/agent-team-p3-box-integration`):** the
**box-side integration** that makes a task flow BUILD → DISPATCH (trigger that live CI,
capture the run_id) → VERIFY (read its conclusion) automatically. As-built, `serve`
binds the fail-safe gated P3 wiring (degrading to the INERT path if the dispatch target
/ CI-read token is unset — Phase-0 Decision 5), dispatch is **operator-initiated**
(branch push + `gh workflow run` on operator-host credentials; the box holds no standing
write token), run_id capture is **run-name / correlation-tag** based, and the CI wait is
**async resume-on-complete** via a `tick()`-driven CI-watcher (not a blocking poll). The
Tier-3 fixer remains `--dry-run` only until the Phase-4 smoke test passes.
**After the box-side integration + the remaining human gates:** the pipeline emits a
diff end-to-end into a reviewable **draft PR** for human review. The box never gains a
write token.
## 2. Locked decisions (carried in — do not relitigate here)
- **D-OIDC:** apply path uses a **GitHub App `pull-requests:write` token**, **ZERO AWS, no OIDC**.
- **D2:** box stays **read-only / no standing write token**; **org CI does the applying**.
- **B4:** the LLM-proposed diff is **untrusted code**; the CI trust boundary (below) is mandatory.
- Output is **draft PRs only** — nothing auto-merges; human approval is the merge gate.
- (Separate, not part of this plan: `aws-posture` resident access via step-ca + IAM Roles
Anywhere — that IAM is already cross-review-approved and is its own sub-task.)
## 3. Hard gates (must clear before the flip — these block everything)
| Gate | Why | Owner | Status |
|---|---|---|---|
| `/sh-plan-review` on THIS plan | adversarial plan audit before build | me → GPT-4.1 | ✅ DONE (Round 1 + 2, 2026-06-22) |
| `/sh-security-review` on the apply/verify CI surface | auth + untrusted-input + CI trust boundary | me | ⏳ RE-RUN on the *enabled* workflow + bound box-side wiring (C1 / Phase 3 / B5) |
| GPT-4.1 cross-family review on the apply/verify CI + any permission change | mandatory for the trust-boundary / permissions surface | orchestrator | ⏳ RE-RUN on the *enabled* workflow + bound box-side wiring (C1 / Phase 3 / B5) |
| ~~`GH_TOKEN`→`GITHUB_TOKEN` resolved~~ ✅ DONE 2026-06-22 | transport reads `GITHUB_TOKEN`; box now has a `GITHUB_TOKEN` alias of `GH_TOKEN` in `~/secrev.env` | me | ✅ DONE |
**The apply/verify CI workflow + its provisioning are already LIVE (2026-06-22).** The
two `/sh-security-review` + GPT-4.1 cross-review gates were satisfied against the
authored workflow during build; per B5 they are **RE-RUN against the ACTUAL enabled
workflow AND the bound box-side build→dispatch→verify wiring** before the box-side
integration is deployed/merged (Phase 3). The box-side integration does NOT deploy/merge
until that re-run passes — **hard stop** (see Phase 3).
> **Plan-review disposition (GPT-4.1 cross-family, 2026-06-22 — REQUEST CHANGES).** Findings
> folded into §4 and Phases 1/1b: expanded denylist vectors, runner-trust, concrete
> diff-transport + threat model, gate-weakening detection, PR-metadata sanitization, ledger
> anti-tamper, recovery for a merged-privileged change, required-check-name discovery,
> deploy-before-merge enforcement, no-write-token audit, draft-PR rate monitoring + stale-PR
> cleanup. Several items the reviewer marked BLOCK are **already implemented in PR #17's CI**
> (canonicalization, egress, SHA-pin, empty-hash fail-closed, Checks-API) — Phase 1 verifies
> them rather than rebuilding. Open QUESTIONs to answer when building: how human reviewers are
> notified of new draft PRs, and how "every deployable repo has CI" is enforced for targets.
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
> **Plan-review disposition — ROUND 2 (GPT-4.1 cross-family, 2026-06-22 eve — REQUEST CHANGES).**
> Re-review after the durable-intake work merged (#32) and the coordinator went live. Six BLOCK
> items folded into the phases below: (B1) deploy-before-merge made a CONCRETE CI-enforced gate,
> not prose (Phase 1 + Phase 3); (B2) rollback added for a *prematurely-flipped privileged job that
> RUNS* — not only an accidental merge (Phase 1b); (B3) rollback for ALL privileged surfaces is a
> tested script, exercised, not one-time manual (Phase 1b); (B4) Phase 3 is explicitly gated on
> Phase 2 being fully provisioned + verified (Phase 3 precondition); (B5) `/sh-security-review` +
> GPT-4.1 cross-review are RE-RUN on the actual *enabled* workflow before the flip (Phase 3); (B6)
> docs/memory updated INCREMENTALLY at each privileged step, not deferred to Phase 6. FIX/NIT also
> folded: periodic review of gate-weakening patterns, a check-name-discovery test across targets,
> a memory update when denylist vectors change, snapshot retention in Phase 0, and the draft-PR
> notification QUESTION assigned in Phase 4. (Findings-to-verify, not gospel: the reviewer
> under-read §3 — cross-review IS already a hard gate; B5 is the genuine delta = re-run on the
> *enabled* file.)
## 4. The CI trust boundary (design B4 — what the workflow must enforce)
> **Already implemented in the merged P3-live CI hardening (PR #17) — Phase 1 VERIFIES, does not rebuild:**
> denylist path **canonicalization + symlink/rename/traversal resistance**, **egress
> restriction** on the untrusted job, **SHA-pinned** actions, **empty/missing-hash
> fail-closed**, and **authenticated Checks-API** result consumption. The GPT-4.1
> plan-review (2026-06-22) flagged these as "missing" because the *plan* under-restated
> them; confirm each against the actual `agent-team/ci/agent-team-apply-verify.yml` +
> `ci_fetcher.py`/`ci_gate.py` rather than re-authoring.
1. **Split CI.** An **untrusted build/test job**: `contents: read` only, **no secrets / no
OIDC / no write token**, **egress-restricted** (verify/audit the restriction, don't just
assert it). A **separate privileged job** that **never checks out the patch code** (no
`pull_request_target` + head checkout) opens the draft PR. **Privileged jobs MUST run only
on GitHub-hosted runners — assert no self-hosted/user-provided runner can pick them up.**
2. **Denylist** (reject/escalate, never auto-build). Beyond `.github/workflows/**`,
IAM/permission IaC, branch-protection / `CODEOWNERS` / Dependabot config, and
out-of-scope files, the denylist MUST also cover these RCE/priv-esc vectors:
`.gitmodules` / submodule changes, **git hooks** (`.git/hooks`, `core.hooksPath`,
`.husky/**`), **`.gitattributes`** (filter/clean-smudge process), lockfiles +
package-manager **postinstall/preinstall** hooks, and **generated/build-artifact** files
(codegen output is not reviewable as source). Path matching is **canonicalized** (PR #17)
so symlinks/renames/traversal can't slip a denied path past.
3. **Diff-transport integrity (concretely specified, threat-modeled).** The box records the
diff **content hash** in its ledger; CI **verifies the hash before apply**. The diff
reaches CI as a **content-addressed signed artifact** (HMAC/keyed digest the box and the
privileged job share via an Actions secret) **OR** a short-lived, **branch-only** token
scoped to a single ref — the box never holds a write token. Threat model the path:
tamper-in-transit (defeated by hash+signature verify), replay of an old diff (defeated by
per-task nonce + the `status='open'`/one-shot ledger state), and a hostile artifact name.
4. **Pure-code green gate.** Pass/fail is owned by a **pure-code gate** reading
**authenticated Checks-API results** (run id + diff hash). The **LLM verifier may propose
fixes but can never declare a build green**.
5. **Gate-weakening detection.** A diff that **lowers a gate** — adds `# noqa`, `# type: ignore`,
test `skip`/`xfail`, coverage/lint **excludes**, `--no-verify`, or edits the gate config
itself — is flagged and escalated (a build can't make itself pass by disabling the checks).
6. **PR-metadata sanitization.** The draft-PR **title / body / comments** are sanitized so a
hostile diff or LLM output can't exfiltrate secrets/env or inject content into the PR text.
7. **Ledger anti-tamper.** The diff-hash ledger entry is integrity-protected (the existing
atomic-write + integrity-check substrate; verify the hash row can't be silently rewritten
between record and apply).
8. **Merge gate.** Draft PR + required checks + `/sh-security-review` + Claude Code App
review + **human approval**.
## 5. Phases
### Phase 0 — Plan review & pre-reqs 🤖/🧑 — ✅ DONE
> **DONE.** Plan review complete; pre-reqs resolved. (Task label: **C0**.)
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
- [x] Run `/sh-plan-review` on this doc; fold BLOCK/FIX items in. (Round 1 + Round 2 done; this doc is the result.)
- [x] Confirm a clean revert point (git tag main; Hyper-V snapshot of sh-secrev).
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
**Snapshot retention:** keep the pre-P3 snapshot until P3 has run clean for one full
cycle (Phase 4 DoD), then prune — recorded here so it is not an open-ended snapshot.
- [x] Resolve `GH_TOKEN`→`GITHUB_TOKEN` (box `~/secrev.env` now has a `GITHUB_TOKEN` alias).
- **Rollback:** none (no state changed).
### Phase 1 — Author/verify the split-CI apply/verify workflow 🤖 (review-gated) — ✅ DONE (2026-06-22)
> **DONE.** The split-CI apply/verify workflow is authored, enabled, and provisioned.
> The deliverables below were built and shipped; they are retained for the record and
> for the Phase-3 re-run gates to verify against. (Task label: this is part of **C2**.)
- [x] Reconcile `agent-team/ci/agent-team-apply-verify.yml` with §4. **First confirm** the
PR-#17 controls are present (canonicalized denylist, egress restriction, SHA-pins,
empty-hash fail-closed, Checks-API consumption); only then add the new §4 items.
- [x] **Add denylist vectors** (§4.2): submodules/`.gitmodules`, git hooks/`core.hooksPath`/`.husky`,
`.gitattributes` filters, lockfile postinstall/preinstall, generated/build artifacts.
Add a test suite proving canonicalization resists symlink/rename/traversal.
- [x] **Runner-trust assertion** (§4.1): test that privileged jobs cannot run on a
self-hosted/user-provided runner.
- [x] **Concretize + threat-model the diff transport** (§4.3): pick content-addressed signed
artifact (shared HMAC secret) or short-lived branch-only token; add per-task nonce
anti-replay; document and test it.
- [x] **Gate-weakening detector** (§4.5): CI step that fails on a diff adding
`noqa`/`type: ignore`/skip/xfail/excludes/`--no-verify` or editing the gate config.
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
The detector's pattern list is **reviewed/expanded each time a new bypass vector is
found** (FIX) — record the list in code with a comment pointing here, and update it +
memory when a vector is added (same discipline as the denylist below).
- [x] **PR-metadata sanitization** (§4.6) and **ledger anti-tamper** (§4.7) implemented + tested.
- [x] `agent_team/ci_fetcher.py` (read-only Checks-API fetcher; fails closed) +
`ci_gate.py` (pure-code green). Add a mechanism for the gate to **discover the correct
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
required check names per repo/branch** (avoid hardcoded check-name drift across repos),
**with a test that exercises discovery against every intended target repo/branch** (FIX).
- [x] **Memory/doc update when denylist vectors change** (FIX): adding a denylist vector (here
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
or later) updates `project_r720_agent_team` memory + the Confluence host page in the SAME
change — the denied set is operational/security-critical, not tribal knowledge.
- [x] **Deploy-before-merge enforcement — CONCRETE, CI-enforced (B1). REQUIRED Phase-1
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
deliverable; the flip does not proceed without it (no manual fallback).** "Deploy" of this
change = the privileged path is *proven on the live box before the workflow PR merges*.
Build, in this phase:
- (a) the box exercises a **dispatch dry-run** of the apply/verify workflow (privileged steps
still `if:${{ false }}`) that emits a signed "exercised-on-box" artifact/status. **Dry-run
faithfulness (NIT):** it runs the SAME workflow file and the SAME untrusted build/verify +
pure-code-gate jobs as the live path — ONLY the privileged `if:` differs — so the dry-run
is a faithful proof of the path, not a separate mock.
- (b) a **required status check** on the workflow-file PR consumes that proof, so the PR is
un-mergeable until the box has run it.
- (c) branch-protection set so the required checks **cannot be bypassed by admins** ("do not
allow bypassing the above settings" / include-administrators) — **no `--admin` merge of the
privileged flip**. Applied in Phase 2; **no ordering window** because the flip (Phase 3) is
gated on Phase 2 completion (the B4 precondition), so admin-bypass is already disabled before
any flip is possible. The Phase-1 required-status-check (a/b) and the Phase-2 branch-protection
(c) together are the gate; the flip cannot happen until BOTH are in place.
This is the gate; until it is built+green, the flip is blocked. (Owner: 🤖 build; verified in
the Phase-1 `/sh-security-review` + cross-review hard stop. The check's implementation is itself
a Phase-1 build task — that it is not yet physically built is expected for a pre-build plan; what
matters is it is non-optional and flip-blocking, enforced at the Phase-1 hard stop.)
- [x] **Concrete "no write token on the box" audit (B-QUESTION → a real check):** a
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
test/script asserting the box env + coordinator config hold no `pull-requests:write` /
contents-write token (grep the live env names + assert the App token is only an Actions
secret), runnable on the box and in CI. Not a prose claim.
- [x] `/sh-security-review` + GPT-4.1 cross-review on this surface (against the authored
workflow during build). **Hard stop until both pass.** (These are RE-RUN on the *enabled*
workflow + the bound box-side wiring in Phase 3 — the C1 human gate, see B5 there. That
re-run is the genuine outstanding gate; the in-build pass is satisfied.)
- **Rollback:** the apply/verify workflow is now LIVE; rollback is no longer "delete an inert
file" — use the Phase-1b tested rollback script (revert the workflow SHA → inert, restore
the environment + branch protection from baseline). See Phase 1b / Phase 3 rollback.
### Phase 1b — Recovery for an accidentally-merged/applied privileged change 🤖/🧑 — ✅ DONE (2026-06-22)
> **DONE.** The privileged surfaces are LIVE, so their recovery tooling shipped with them.
> The tested rollback script + the runaway/stale-PR monitoring below are built and in place;
> the Phase-3 rollback exercises the script against the live state. (Task label: part of **C2**.)
- [x] **Rollback as a TESTED SCRIPT covering ALL privileged surfaces (B3)** — not a one-time
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
manual exercise. One scripted, re-runnable rollback that, per surface, restores from a
recorded baseline: (1) the apply/verify **workflow** (revert the SHA → inert), (2) the
**`agent-apply` environment** (required-reviewer + protection rules), (3) the **GitHub App
permissions/installation** (rotate token, reduce/uninstall), (4) **branch protection**.
The script asserts the post-restore state matches the baseline. Exercised in Phase 3
rollback AND re-runnable on demand.
- [x] **Premature-flip-that-RAN rollback (B2).** Distinct from an accidental *merge*: cover the
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
case where `if:${{ false }}` is flipped early (or the environment gate is misconfigured) and
the privileged job **actually runs** — incident steps: rotate the GitHub App token
immediately, close/revert any draft PR (or branch) it opened, confirm via the Checks/PR
audit trail exactly what ran in the window, restore the environment + branch protection from
baseline, and file the incident. This is the "it executed" path, not just "it merged."
- [x] Add light **monitoring on draft-PR creation rate** (runaway-volume alarm) and an
**orphaned/stale draft-PR cleanup** step.
### Phase 2 — Provision the GitHub App + environment 🧑 OPERATOR (browser/admin) — ✅ DONE (2026-06-22)
> **DONE.** Provisioning is LIVE: the scoped GitHub App, the `agent-apply` environment, and
> the App-credential Actions secrets all exist, and dispatched runs have executed against
> them. (Task label: **C0** complete.) No write token landed on the box — the App token lives
> only as an Actions secret; this is enforced by `scripts/assert_no_write_token.py`.
- [x] Create a dedicated **GitHub App** with **`pull-requests:write`** (+ minimal contents to
open a branch/PR); install on the org. Token lives in **CI**, never on the box.
- [x] Create the **`agent-apply` GitHub Actions Environment** with a **required reviewer**
(Adam) + branch-protection so the privileged job cannot run unreviewed.
- [x] Store the App credentials as repo/org **Actions secrets** (not on the box).
- **Rollback:** uninstall the App; delete the environment + secrets (via the Phase-1b script).
### Phase 3 — Bind the box-side build→dispatch→verify wiring 🤖 — IN PROGRESS (the remaining build)
> **Workflow flip already DONE (2026-06-22):** `permissions: pull-requests: write` and
> `environment: agent-apply` are uncommented and the two `if: ${{ false }}` are enabled — the
> workflow is LIVE. The PRECONDITION (B4) below is **satisfied** (Phase 2 provisioning is
> complete + verified). The remaining Phase-3 work is the **box-side integration** built on
> `feat/agent-team-p3-box-integration` (BUILD → DISPATCH → VERIFY; operator-initiated dispatch;
> run-name correlation; async CI-watch; fail-safe serve default — see `docs/P3-PHASE0-DESIGN.md`)
> plus the C1 re-run gates and the box env wiring. **C1 (the re-run gates) and D (deploy/smoke/
> merge, Phase 4) remain the outstanding HUMAN gates.**
- [x] **PRECONDITION (B4): Phase 2 fully complete + verified before ANY flip.** ✅ SATISFIED —
the GitHub App exists with `pull-requests:write` (+ minimal contents) and is installed, the
`agent-apply` environment exists with Adam as required reviewer + branch protection, and the
App credentials are stored as Actions secrets (NOT on the box).
- [x] In the workflow: uncomment `permissions: pull-requests: write` and
`environment: agent-apply`; flip the two `if: ${{ false }}` → enabled. ✅ DONE 2026-06-22.
- [ ] **C1 — RE-RUN BOTH GATES ON THE ENABLED WORKFLOW + BOUND WIRING (B5). 🧑 OUTSTANDING HUMAN
GATE.** `/sh-security-review` + the GPT-4.1 cross-family review are run again against the
*actual enabled* `agent-team-apply-verify.yml` (permissions live, `if:` true) **and** the
bound box-side `gated_build_verify_wiring` (BUILD → DISPATCH → VERIFY) — NOT only the inert
Phase-1 version. **Hard stop until both pass.** (Permissions are live + the dispatch/verify
wiring is new → the mandatory cross-family review is independently required here against the
real diff.)
- [ ] Bind `agent_team.coordinator.gated_build_verify_wiring(...)` (real diff builder + dispatch
with run-name correlation run_id capture + read-only CI-result fetcher) as the **fail-safe
`serve` default** (Decision 5: degrade to the INERT P3 path if the dispatch target / CI-read
token is unset, never crash-loop). The leaf path is BUILD → DISPATCH (trigger the live CI,
suspend) → [CI-watcher resumes on terminal conclusion] → VERIFY (pure-code gate). *(Built on
`feat/agent-team-p3-box-integration`.)*
- [ ] Set the box-side apply env vars the live path reads (read-only CI-result token + dispatch
target: `AGENT_TEAM_REPO_OWNER`/`_NAME`/`_BASE_BRANCH`/`_CI_READ_TOKEN`). Confirm **no** write
token lands on the box (run the no-write-token audit, `scripts/assert_no_write_token.py`).
- [ ] **Incremental docs (B6):** update `OPERATOR-RUNBOOK.md` + memory as the box-side flip lands
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
(do not wait for Phase 6) — what the apply path can/can't do, the denylist, the rollback.
*(The P3 box env wiring is already documented in `docs/provisioning/OPERATOR-RUNBOOK.md`.)*
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
- **Rollback:** run the Phase-1b tested rollback script (re-set `if: ${{ false }}`, re-comment
`environment:`, set `build_verify_wiring=None`, restart the coordinator). **Exercise it once
here** to prove it works before relying on it.
### Phase 4 — Deploy, smoke test to a first draft PR, merge 🧑/🤖 — D (OUTSTANDING HUMAN GATE)
> **D — the remaining HUMAN gate.** After C1 passes, deploy the box-side integration to the live
> coordinator (`sh-secrev`, via `/sh-deploy-r720`), drive the smoke test below, then merge. Gated
> on Phase 3 (box-side wiring bound + C1 re-run gates green).
- [ ] Drive one trivial, in-scope task end-to-end → confirm: untrusted job builds/tests with
no secrets, denylist rejects an out-of-scope diff, pure-code gate gates on real Checks
results, privileged job opens a **draft PR** with required checks attached, **nothing merged**.
- [ ] Flip the **Tier-3 fixer** off `--dry-run` only after the smoke test passes; verify a
dependency-CVE bump produces a draft PR.
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
- [ ] **Draft-PR reviewer notification (QUESTION resolved/assigned):** decide + wire how a new
draft PR pings the human reviewer — default = the existing Slack ALARM path posts a
`#agent-team` notice with the PR link (Adam owns this decision; recorded so it is not left open).
- **Rollback:** close the draft PR; Phase-3 rollback.
### Phase 5 — Enable the cross-plane loop 🤖
- [ ] Allow confirmed Plane-1 checker findings (`intake-checker`) to flow into pipeline tasks
that end in draft-PR fixes (start conservative: highest-severity, one at a time).
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
- [ ] **Conservative-rollout controls (QUESTION resolved):** enforce "one at a time, highest
severity" concretely — a cap on concurrently-open cross-plane draft PRs (start = 1) read by
the intake-checker leaf, the severity floor in config, and a `#agent-team` log line per
promotion so the rate is observable. Not just prose.
- **Rollback:** revert intake-checker wiring to report-only.
docs(agent-team): fold round-2/3/4 plan-review findings into P3-live-flip plan (#33) * docs(agent-team): fold round-2 GPT-4.1 plan-review findings into P3-live-flip plan Round-2 cross-review (REQUEST CHANGES) folded: - B1 deploy-before-merge made a concrete CI-enforced gate (required status check fed by a box-exercised dispatch dry-run), not prose. - B2 added rollback for a prematurely-flipped privileged job that actually RAN (token rotate, revert opened PR/branch, audit the window) — distinct from an accidental merge. - B3 rollback is a tested, re-runnable script over ALL privileged surfaces (workflow, environment, App perms, branch protection), not a one-time manual run. - B4 Phase 3 explicitly gated on Phase 2 being fully provisioned + verified. - B5 /sh-security-review + GPT-4.1 cross-review RE-RUN on the actual enabled workflow before the flip, not only the inert version. - B6 docs/memory updated incrementally at each privileged step; Phase 6 is the final reconciliation pass. Plus FIX/NIT/QUESTION: gate-weakening pattern review, check-name discovery test, memory update on denylist change, snapshot retention in Phase 0, draft-PR notification (Phase 4) + conservative-rollout controls (Phase 5) made concrete. GH_TOKEN->GITHUB_TOKEN gate marked done (box alias added). * docs(agent-team): close B1 — deploy-before-merge is a committed hard gate, not a manual fallback Round-3 re-review resolved B2-B6 but flagged B1 still-open: the prior wording left a 'enforced by hand until the check exists' escape hatch. Reframe B1 as a REQUIRED Phase-1 build deliverable that blocks the flip (no manual fallback), with the required-status-check + admin-bypass-disabled branch protection, and a dry-run faithfulness note (same workflow file/jobs as live, only the privileged if: differs). * docs(agent-team): close B1 ordering — admin-bypass disabled before any flip via the B4 precondition Round-4 confirmatory re-review noted (c) admin-bypass-disable sits in Phase 2 while the gate is framed Phase-1. Clarify there is no ordering window: the flip (Phase 3) is gated on Phase 2 completion (B4), so branch protection incl. admin-bypass-disable is necessarily in place before any flip. The check's implementation being a Phase-1 build task (not yet physically built) is expected for a pre-build plan; it is non-optional and flip-blocking, enforced at the Phase-1 hard stop. Stopping the plan-review cycle here per the project's '3 cycles, residual is build-time' rule.
2026-06-22 17:28:37 -04:00
### Phase 6 — Final docs & memory consolidation 🤖
> **B6: docs/memory are updated INCREMENTALLY at each privileged step above** (Phase 3 runbook+memory
> on flip; denylist/gate-weakening changes update memory in the same change). Phase 6 is the FINAL
> reconciliation pass, not the first time docs are touched.
- [ ] Reconcile `OPERATOR-RUNBOOK.md`, the Confluence host page (20054017) + runbook (22609921),
and `project_r720_agent_team` memory: P3 is LIVE, what the apply path can/can't do, the
denylist, the merge gate, the rollback — confirming the incremental updates are complete + consistent.
## 6. What changes (and what does NOT)
**Changes:**
- The pipeline produces **draft PRs** for well-scoped work (dep bumps, small fixes, docs,
checker-surfaced issues). Your role shifts from author → reviewer/approver for that class.
- Tier-3 fixer goes live (dep-CVE bumps → draft PRs).
- Plane-1 findings can become draft-PR fixes (cross-plane loop).
**Does NOT change:**
- Box stays read-only; **CI applies, not the box**. No AWS / no OIDC in the apply path.
- **Denylisted, high-blast-radius changes** (workflows, IAM, branch protection, CODEOWNERS,
Dependabot) are **never auto-built** — always escalated.
- Human gates stay: the clarifier question-set AND the PR approval are both yours.
- **Drafts, not merges** — you remain the merge authority.
## 7. Risks & mitigations
| Risk | Mitigation |
|---|---|
| Untrusted diff exfiltrates secrets / escalates in CI | split CI: build job has no secrets/OIDC/write, egress-restricted; privileged job never checks out patch code |
| Malicious diff edits CI/IAM/branch-protection to self-escalate | denylist → reject/escalate, never auto-build |
| LLM "declares" a broken build green | pure-code gate reads authenticated Checks-API only; LLM can't set status |
| Diff tampered between box and CI | diff-hash recorded in ledger, verified before apply |
| Standing write capability on the always-on box | there is none — App token lives in CI; box holds only a read-only CI-result token |
| Runaway PR volume | start with Tier-3 only + one finding at a time; required-reviewer environment gates each |
## 8. Definition of done
- [x] `/sh-plan-review` passed; `/sh-security-review` + GPT-4.1 cross-review passed on the CI
surface during build. ⏳ **C1 outstanding:** both are RE-RUN against the *enabled* workflow +
the bound box-side wiring before the box-side integration deploys/merges (Phase 3 / B5).
- [ ] **D:** Phase-4 deploy + smoke test produced a draft PR; nothing auto-merged; rollback
exercised once.
- [x] No write token on the box (verified via `scripts/assert_no_write_token.py`); apply path is
zero-AWS. *(Re-confirm after the box-side env vars are set in Phase 3.)*
- [ ] Docs + Confluence + memory updated *(this plan + `OPERATOR-RUNBOOK.md` reflect the live
infra; Confluence + memory final reconciliation is Phase 6)*.
- [x] Snapshot retained until P3 runs clean for one cycle, then pruned.