docs(agent-team): formal phased plan for the P3-live flip (build->verify->draft-PR)
Phased plan to take Plane-2 from clarify+plan to producing reviewable draft PRs: locked decisions (GitHub App pull-requests:write, zero-AWS, read-only box), the mandatory gates (/sh-plan-review + /sh-security-review + GPT-4.1 cross-review on the CI surface), the B4 CI trust boundary (split CI, denylist, diff-hash, pure-code green gate), phases 0-6 with owners + exercised rollbacks, what changes vs what does not, risks, and definition of done. Input to /sh-plan-review before any build.
This commit is contained in:
parent
b6a12af477
commit
a4313d393f
1 changed files with 148 additions and 0 deletions
148
docs/provisioning/P3-LIVE-FLIP-PLAN.md
Normal file
148
docs/provisioning/P3-LIVE-FLIP-PLAN.md
Normal file
|
|
@ -0,0 +1,148 @@
|
|||
# P3-LIVE-FLIP PLAN — agent-team build → verify → draft-PR
|
||||
|
||||
Formal phased plan to take the agent-team Plane-2 pipeline from **clarify+plan only**
|
||||
to **producing reviewable draft PRs**, while keeping the always-on R720 box
|
||||
read-only and the apply path zero-AWS. Status as of 2026-06-22: **NOT STARTED**
|
||||
(P3 is built but inert). This plan is the input to `/sh-plan-review` before any build.
|
||||
|
||||
> **Prerequisite reading:** `docs/r720-agent-team-design.md` §3.3.2 (CI-as-verifier
|
||||
> trust boundary, "B4"), `PROVISIONING-RUNBOOK.md` (the P3-live-flip section),
|
||||
> and memory `project_r720_agent_team` (locked decisions).
|
||||
|
||||
---
|
||||
|
||||
## 1. Objective & current state
|
||||
|
||||
**Today (inert):** the pipeline runs `INTAKE → CLARIFIER → PLANNER → REVIEW`, but
|
||||
`serve` passes `build_verify_wiring=None`, the Tier-3 fixer is `--dry-run` only, and
|
||||
`agent-team/ci/agent-team-apply-verify.yml` has its privileged steps disabled with
|
||||
`if: ${{ false }}` and `pull-requests:write` / `environment:` commented out. So it
|
||||
clarifies + plans but writes no code and opens no PR.
|
||||
|
||||
**After P3:** the pipeline can emit a diff, have **org CI** build/test/security-review
|
||||
it in an untrusted sandbox, a pure-code gate confirm green from authenticated
|
||||
Checks-API results, and a scoped **GitHub App** open a **draft PR** for human review.
|
||||
The box never gains a write token.
|
||||
|
||||
## 2. Locked decisions (carried in — do not relitigate here)
|
||||
|
||||
- **D-OIDC:** apply path uses a **GitHub App `pull-requests:write` token**, **ZERO AWS, no OIDC**.
|
||||
- **D2:** box stays **read-only / no standing write token**; **org CI does the applying**.
|
||||
- **B4:** the LLM-proposed diff is **untrusted code**; the CI trust boundary (below) is mandatory.
|
||||
- Output is **draft PRs only** — nothing auto-merges; human approval is the merge gate.
|
||||
- (Separate, not part of this plan: `aws-posture` resident access via step-ca + IAM Roles
|
||||
Anywhere — that IAM is already cross-review-approved and is its own sub-task.)
|
||||
|
||||
## 3. Hard gates (must clear before the flip — these block everything)
|
||||
|
||||
| Gate | Why | Owner |
|
||||
|---|---|---|
|
||||
| `/sh-plan-review` on THIS plan | adversarial plan audit before build | me → GPT-4.1 |
|
||||
| `/sh-security-review` on the apply/verify CI surface | auth + untrusted-input + CI trust boundary | me |
|
||||
| GPT-4.1 cross-family review on the apply/verify CI + any permission change | mandatory for the trust-boundary / permissions surface | orchestrator |
|
||||
| `GH_TOKEN`→`GITHUB_TOKEN` resolved | the GitHub transport reads `GITHUB_TOKEN`; box has `GH_TOKEN` | me (folded in here) |
|
||||
|
||||
The flip does NOT proceed until `/sh-security-review` AND the GPT-4.1 cross-review on
|
||||
the CI surface both pass.
|
||||
|
||||
## 4. The CI trust boundary (design B4 — what the workflow must enforce)
|
||||
|
||||
1. **Split CI.** An **untrusted build/test job**: `contents: read` only, **no secrets / no
|
||||
OIDC / no write token**, egress-restricted. A **separate privileged job** that **never
|
||||
checks out the patch code** (no `pull_request_target` + head checkout) opens the draft PR.
|
||||
2. **Denylist.** A diff touching `.github/workflows/**`, IAM/permission IaC,
|
||||
branch-protection / `CODEOWNERS` / Dependabot config, or out-of-scope files is
|
||||
**rejected / escalated to a human — never auto-built**.
|
||||
3. **Diff-hash integrity.** The box records the diff hash in its ledger; CI verifies the
|
||||
hash before apply. The diff reaches CI as a signed artifact / short-lived branch-only
|
||||
token (the box has no write token).
|
||||
4. **Pure-code green gate.** Pass/fail is owned by a **pure-code gate** reading
|
||||
**authenticated Checks-API results** (run id + diff hash). The **LLM verifier may propose
|
||||
fixes but can never declare a build green**.
|
||||
5. **Merge gate.** Draft PR + required checks + `/sh-security-review` + Claude Code App
|
||||
review + **human approval**.
|
||||
|
||||
## 5. Phases
|
||||
|
||||
### Phase 0 — Plan review & pre-reqs 🤖/🧑
|
||||
- [ ] Run `/sh-plan-review` on this doc; fold BLOCK/FIX items in.
|
||||
- [ ] Confirm a clean revert point (git tag main; Hyper-V snapshot of sh-secrev).
|
||||
- [ ] Resolve `GH_TOKEN`→`GITHUB_TOKEN` (transport accepts both / box env updated).
|
||||
- **Rollback:** none (no state changed).
|
||||
|
||||
### Phase 1 — Author the split-CI apply/verify workflow 🤖 (review-gated)
|
||||
- [ ] Write `agent-team/ci/agent-team-apply-verify.yml` per §4: split jobs, denylist,
|
||||
diff-hash verification, pure-code gate reading Checks-API. **SHA-pin all actions.**
|
||||
- [ ] Implement/confirm `agent_team/ci_fetcher.py` (read-only Checks-API result fetcher;
|
||||
fails closed: missing token / 404 / auth fail → `None` → gate BLOCKs, task parks)
|
||||
and `agent_team/ci_gate.py` (pure-code green decision).
|
||||
- [ ] `/sh-security-review` + GPT-4.1 cross-review on this surface. **Hard stop until both pass.**
|
||||
- **Rollback:** workflow file stays inert (`if: ${{ false }}` not yet flipped); delete the file.
|
||||
|
||||
### Phase 2 — Provision the GitHub App + environment 🧑 OPERATOR (browser/admin)
|
||||
- [ ] Create a dedicated **GitHub App** with **`pull-requests:write`** (+ minimal contents to
|
||||
open a branch/PR); install on the org. Token lives in **CI**, never on the box.
|
||||
- [ ] Create the **`agent-apply` GitHub Actions Environment** with a **required reviewer**
|
||||
(Adam) + branch-protection so the privileged job cannot run unreviewed.
|
||||
- [ ] Store the App credentials as repo/org **Actions secrets** (not on the box).
|
||||
- **Rollback:** uninstall the App; delete the environment + secrets.
|
||||
|
||||
### Phase 3 — Bind the live wiring (still gated by the environment) 🤖
|
||||
- [ ] In the workflow: uncomment `permissions: pull-requests: write` and
|
||||
`environment: agent-apply`; flip the two `if: ${{ false }}` → enabled.
|
||||
- [ ] Bind `agent_team.coordinator.gated_build_verify_wiring(...)` (real diff builder +
|
||||
read-only CI-result fetcher) so a leaf calls it only **after** the gate clears.
|
||||
- [ ] Set the box-side apply env vars the live path reads (read-only CI-result token +
|
||||
dispatch target). Confirm **no** write token lands on the box.
|
||||
- **Rollback:** re-set `if: ${{ false }}`, re-comment `environment:`, set
|
||||
`build_verify_wiring=None`; restart the coordinator. (Exercise this rollback once.)
|
||||
|
||||
### Phase 4 — Smoke test to a first draft PR 🧑/🤖
|
||||
- [ ] Drive one trivial, in-scope task end-to-end → confirm: untrusted job builds/tests with
|
||||
no secrets, denylist rejects an out-of-scope diff, pure-code gate gates on real Checks
|
||||
results, privileged job opens a **draft PR** with required checks attached, **nothing merged**.
|
||||
- [ ] Flip the **Tier-3 fixer** off `--dry-run` only after the smoke test passes; verify a
|
||||
dependency-CVE bump produces a draft PR.
|
||||
- **Rollback:** close the draft PR; Phase-3 rollback.
|
||||
|
||||
### Phase 5 — Enable the cross-plane loop 🤖
|
||||
- [ ] Allow confirmed Plane-1 checker findings (`intake-checker`) to flow into pipeline tasks
|
||||
that end in draft-PR fixes (start conservative: highest-severity, one at a time).
|
||||
- **Rollback:** revert intake-checker wiring to report-only.
|
||||
|
||||
### Phase 6 — Docs & memory 🤖
|
||||
- [ ] Update `OPERATOR-RUNBOOK.md`, the Confluence host page, and memory: P3 is LIVE,
|
||||
what the apply path can/can't do, the denylist, the merge gate, the rollback.
|
||||
|
||||
## 6. What changes (and what does NOT)
|
||||
|
||||
**Changes:**
|
||||
- The pipeline produces **draft PRs** for well-scoped work (dep bumps, small fixes, docs,
|
||||
checker-surfaced issues). Your role shifts from author → reviewer/approver for that class.
|
||||
- Tier-3 fixer goes live (dep-CVE bumps → draft PRs).
|
||||
- Plane-1 findings can become draft-PR fixes (cross-plane loop).
|
||||
|
||||
**Does NOT change:**
|
||||
- Box stays read-only; **CI applies, not the box**. No AWS / no OIDC in the apply path.
|
||||
- **Denylisted, high-blast-radius changes** (workflows, IAM, branch protection, CODEOWNERS,
|
||||
Dependabot) are **never auto-built** — always escalated.
|
||||
- Human gates stay: the clarifier question-set AND the PR approval are both yours.
|
||||
- **Drafts, not merges** — you remain the merge authority.
|
||||
|
||||
## 7. Risks & mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| Untrusted diff exfiltrates secrets / escalates in CI | split CI: build job has no secrets/OIDC/write, egress-restricted; privileged job never checks out patch code |
|
||||
| Malicious diff edits CI/IAM/branch-protection to self-escalate | denylist → reject/escalate, never auto-build |
|
||||
| LLM "declares" a broken build green | pure-code gate reads authenticated Checks-API only; LLM can't set status |
|
||||
| Diff tampered between box and CI | diff-hash recorded in ledger, verified before apply |
|
||||
| Standing write capability on the always-on box | there is none — App token lives in CI; box holds only a read-only CI-result token |
|
||||
| Runaway PR volume | start with Tier-3 only + one finding at a time; required-reviewer environment gates each |
|
||||
|
||||
## 8. Definition of done
|
||||
- [ ] `/sh-plan-review`, `/sh-security-review`, and GPT-4.1 cross-review on the CI surface all passed.
|
||||
- [ ] Phase-4 smoke test produced a draft PR; nothing auto-merged; rollback exercised once.
|
||||
- [ ] No write token on the box (verified); apply path is zero-AWS.
|
||||
- [ ] Docs + Confluence + memory updated.
|
||||
- [ ] Snapshot retained until P3 runs clean for one cycle, then pruned.
|
||||
Reference in a new issue