490 lines
41 KiB
Markdown
490 lines
41 KiB
Markdown
# R720 Agent Team — Design (v2)
|
|
|
|
Status: **DESIGN LOCKED — ready to build. Not built yet.** **v5** folded the final plan-review refinements
|
|
(SQLite `BEGIN IMMEDIATE` for the compare-and-set §3.3.1; CI denylist defense-in-depth + authenticated-only
|
|
pass/fail gate + compromised-box honesty §3.3.2; contention reserve + parked-task aging §6.6; backup integrity
|
|
definition + post-restore reconciliation §6.7; tested rollbacks §7; per-role canaries §6.4; CLI audit/confirm
|
|
§3.3.1). The design went through 3 GPT-4.1 `sh-plan-review` cycles (v1, v3, v4); the architecture was stable
|
|
throughout and remaining grain is now build-time implementation detail captured in the P1/P3 exit gates. **No
|
|
further gate runs by decision; build may begin with Phase 0 / P1.** Earlier status: Drafted 2026-06-17. **v3** reframes around Adam's clarified north star: the R720
|
|
is not just scheduled checkers, it is a self-hosted, human-gated **agentic SDLC pipeline**
|
|
(intake -> clarify -> plan -> review -> build -> verify), fed from the Mac harness and tickets, with the
|
|
scheduled checkers as one task source. v2's roster work becomes "Plane 1"; the pipeline is "Plane 2" and the
|
|
centerpiece. **v4** folds in every v3 plan-review finding: B2 (§3.3.1) and B4 (§3.3.2) resolved with P1/P3 exit
|
|
gates; B1 billing realism + contention (§6.6); B3/B5/F5 provisioning + rollback + cross-review gate (§7); F1
|
|
state durability + backup (§6.7); F2 canary update process (§6.4); F3 service-account lifecycle + F1 backups
|
|
(§9); F4 runbook incident handling (Phase 6); Q3 escalation ladder (§5); Q1/Q2 in §3.3.1. Ready for a re-run of
|
|
`sh-plan-review`. No build until that gate passes and Adam approves.
|
|
|
|
Extends the `sh-secrev` pattern (see `security-review/DEPLOY-R720.md`) from a single security sweep into a small
|
|
roster of scheduled, unattended agents that check, plan, and (carefully) build, coordinated by a Claude `claude
|
|
-p` brain and using GPT / Gemini / DeepSeek where each is the better fit.
|
|
|
|
This doc is the plan, not a runbook. It deliberately reuses the security agent's proven substrate.
|
|
|
|
## 0. Locked decisions (this revision)
|
|
|
|
| # | Decision | Choice |
|
|
|---|---|---|
|
|
| D1 | Headless `claude -p` under Max | **Permitted for now** (Anthropic pushed the disallowing ToS change to a later, unannounced date). Build the auth path **swappable** so the cutover is a config flip. See `reference_claude_subscription_billing`. |
|
|
| D2 | Fixer write path | **Option B**: the always-on box stays read-only; it emits a patch + opens an issue, and a trusted org CI workflow (OIDC) applies the patch on a branch and opens the **draft** PR. No standing write token on the box. |
|
|
| D3 | Checker/planner output mode | **Report + ALARM-only to start.** Clean nights post nothing; confirmed criticals alarm Slack; everything else lands in a mode-600 report. No auto-Jira/Notion writes until signal quality is trusted. |
|
|
| D4 | Billing/auth resilience | **Build a billing-mode abstraction now** (subscription OAuth default, API-key / Bedrock fallback ready). |
|
|
| D5 | aws-posture | **Resident on the box via IAM Roles Anywhere**, with a new **step-ca** internal CA for automated short-lived leaf-cert rotation (no long-lived AWS key on the box). |
|
|
| D6 | Confluence write identity | **Dedicated `confluence-bot` Atlassian service account, edit scoped to the IT space only** (Confluence API tokens inherit the whole user's permissions, so a scoped service account is how we bound blast radius). Token in `~/secrev.env` (mode 600). Costs one Confluence seat. |
|
|
| D7 | Confluence agent modes + Mermaid | **Scheduled = read + recommend only** (gaps/staleness into the report, per D3, never auto-writes). **On-demand = SSH-invoked from Adam's Mac** for an actual write. Mermaid map edits go through `~/.claude/scripts/confluence_mermaid.py` (ADF-only, dry-run-default, macro-count + revert-diff guarded). |
|
|
| D8 | Two-plane architecture | **Plane 1** = scheduled checkers (v2 roster), which also act as a task source. **Plane 2** = the agentic SDLC task pipeline (the centerpiece). Shared substrate. |
|
|
| D9 | Pipeline foundation | **LangGraph (the open-source library, runs in-process, NOT SaaS) + a local SQLite checkpointer.** Durable, resumable graph; `interrupt()` for the human gate. Reuses the existing stack. |
|
|
| D10 | Human-in-the-loop transport | **Pluggable, all three adapters** (Slack Block Kit, GitHub/Jira ticket comments, Claude Code on the Mac); Adam picks the channel per task. A transport-agnostic notify/resume layer maps answers back to the task thread. |
|
|
| D11 | Build/verify execution | **Org CI is the primary sandbox.** Builders emit candidate diffs; the Option-B OIDC workflow builds/tests/security-reviews; the verifier agent reads CI results. Keeps the 4GB box light + read-only. Dedicated builder VM only if fast local loops prove necessary. |
|
|
| D12 | Observability (LangSmith) | **Deprecate LangSmith (the SaaS tracer).** Keep LangGraph (framework, local). Local JSONL (`telemetry.py`, already exists) is the default; self-hosted Phoenix optional later. No SaaS dependency. |
|
|
| D13 | Task intake | **Mac harness first** (SSH-invoke enqueues onto the box), **GitHub issues next** (phased). |
|
|
|
|
## 1. Goal and scope
|
|
|
|
Stand up a coordinated team of agents on the existing `sh-secrev` R720 VM that runs unattended on a schedule,
|
|
operates across every Sea-Haven-Industries org repo automatically, stays cost-bounded, and reports through one
|
|
alarm channel. Claude (subscription OAuth) coordinates and does deep reasoning; Gemini does broad scans; GPT does
|
|
adversarial cross-checks; DeepSeek does mechanical code edits.
|
|
|
|
In scope: read-mostly checkers, a low-blast-radius planner, a resident AWS posture check (D5), and a CI-applied
|
|
draft-PR fixer (D2). Out of scope: anything interactive or needing back-and-forth, and (per the secrev ethos) any
|
|
standing **write** credential on the always-on box.
|
|
|
|
This does **not** replace the security review agent; it sits beside it and reuses its plumbing.
|
|
|
|
## 2. What we reuse vs. what is new
|
|
|
|
Reused from `sh-secrev` as-is: the VM, the systemd-timer model, the OAuth billing path, clean-clone
|
|
auto-discovery into `~/repo-mirrors` (read-only PAT, scrubbed post-fetch; the team scans the **same mirrors**, it
|
|
does not re-clone), the budget primitives (per-call + total caps, fail-toward-over-reporting), ALARM-only Slack,
|
|
mode-600 reports, the anti-complacency canary + coverage-rotation idea, the `orchestrator/` rsync deploy, and the
|
|
non-Claude provider keys in `~/orchestrator/.env`.
|
|
|
|
New: a **coordinator** runner (`agent-team/` dir; entry CLI `run-team.py`, importable modules stay snake_case per
|
|
Python rules; the resource/dir name is kebab-case per handbook), per-role prompt/checklist modules each with its
|
|
own canary, a **billing-mode abstraction** (D4), the **fixer patch -> CI -> draft-PR** path (D2), and the
|
|
**step-ca + Roles Anywhere** setup for aws-posture (D5).
|
|
|
|
## 3. Architecture
|
|
|
|
```
|
|
systemd timer (shared with secrev — see §8)
|
|
│
|
|
▼
|
|
mirror step (reuse sh-secrev discovery) ──► ~/repo-mirrors (read-only)
|
|
│
|
|
▼
|
|
COORDINATOR (claude -p via billing-mode abstraction; read-only tools + Bash to call orchestrator/gh)
|
|
│ shared budget ledger + versioned rotation/coverage state
|
|
├──────────────┬───────────────┬────────────────┬──────────────┬───────────┐
|
|
▼ ▼ ▼ ▼ ▼ ▼
|
|
drift checker dep/CVE checker doc-drift checker aws-posture planner (each role
|
|
(Gemini scan (Claude + (Gemini large- (Sonnet via (Claude) has its own
|
|
+ Claude judge) GPT tiebreak) context) Roles Anywhere) canary)
|
|
│ │ │ │ │
|
|
└──────────────┴───────────────┴────────────────┴──────────────┘
|
|
│ structured JSON per agent
|
|
▼
|
|
coordinator: dedup + prioritize + route
|
|
│
|
|
┌──────────────────────────┼───────────────────────────┐
|
|
▼ ▼ ▼
|
|
Slack ALARM mode-600 report fix-spec queue (D2)
|
|
(confirmed crit) (everything else; │
|
|
no auto-Jira/Notion yet, D3) ▼
|
|
FIXER: DeepSeek edit + Claude
|
|
spec + GPT review ──► patch + issue
|
|
│
|
|
▼ org CI (OIDC) applies patch,
|
|
opens DRAFT PR, runs pre-push
|
|
hooks + CI gates + Claude Code App
|
|
```
|
|
|
|
Model assignment (matches how `orchestrator` already splits them): **Claude (subscription)** = coordinator, deep
|
|
checks, fix-spec authoring; **Gemini 2.5 Pro** = broad whole-repo scans; **GPT-4.1** = adversarial cross-check /
|
|
tiebreak / PR review; **DeepSeek** = mechanical patch writing.
|
|
|
|
### 3.1 Billing-mode abstraction (D4)
|
|
A single `claude_invoke(...)` seam selects the Claude auth/billing path from config: `subscription` (OAuth token,
|
|
default today), `api` (metered `ANTHROPIC_API_KEY`), or `bedrock` (cross-account Bedrock, already used by secrev
|
|
for the rare cross-family tiebreak). Switching modes is a config flip, not a code change. The box still pops any
|
|
stray `ANTHROPIC_API_KEY` in `subscription` mode so OAuth cannot be silently overridden.
|
|
|
|
### 3.2 Relationship to the LangSmith orchestrator (what stays vs what the R720 hosts)
|
|
|
|
Two orchestrators coexist after this plan; neither replaces the other. The split is **trigger + Claude billing**,
|
|
not capability.
|
|
|
|
- **LangSmith orchestrator (Mac-hosted, unchanged).** The existing `orchestrator/` (LangGraph router + memory
|
|
retriever + Composio connector + LangSmith tracing on the `orchestration` project) stays the **on-demand,
|
|
interactive** delegation path: one task -> retrieve memory -> route -> one agent -> result, API-billed. This is
|
|
the CLAUDE.md hybrid-delegation path Claude Code uses for cross-family review, large scans, fast coding, and
|
|
connector actions. It stays one-shot and stateless (Q1: not refactored for persistence).
|
|
- **R720 orchestrator (the agent-team coordinator, new).** The scheduled, unattended, multi-agent layer:
|
|
cadence, the coordinator brain, shared budget ledger, versioned rotation/coverage state, the clean-clone mirror
|
|
corpus, and per-role canaries. Claude work here runs **headless under subscription OAuth** (Agent SDK),
|
|
billing-mode-swappable (§3.1).
|
|
|
|
| Component | Today (LangSmith orchestrator, Mac) | After this plan |
|
|
|---|---|---|
|
|
| Trigger | On-demand from Claude Code / CLI | + scheduled (systemd timer) and SSH-invoked on-demand, on the R720 |
|
|
| Execution shape | One task -> one agent (stateless) | + multi-agent coordination with shared budget + versioned state (R720) |
|
|
| Claude billing | Metered API key | **Subscription OAuth on the R720** (swappable to api/bedrock per §3.1) |
|
|
| Non-Claude (GPT-4.1 / Gemini / DeepSeek) | `run.py` router, API-billed, LangSmith-traced | **unchanged in shape** — the R720 coordinator calls the **local** `~/orchestrator/run.py` (already rsync'd to the box) for these single-shot sub-tasks, so they keep API billing + LangSmith tracing |
|
|
| Memory retriever + embeddings cache | Mac | reused read-only by both (the box's rsync'd copy embeds the same memory store) |
|
|
| Composio connector (Slack/Notion/GitHub) | Mac | reused; the R720 routes Slack/Jira/Notion through it. **Confluence stays OUT of the connector** — native Atlassian MCP on the Mac for interactive edits, `confluence_mermaid.py` + REST for the box |
|
|
| Observability | LangSmith SaaS tracing (`orchestration` project) | **LangSmith deprecated (D12)** — local JSONL (`telemetry.py`) default, self-hosted Phoenix optional. LangGraph framework stays (it is not SaaS) |
|
|
| `models.py` factories + model-ID constants | Mac | shared code (rsync'd); single source of truth for both |
|
|
|
|
**What does NOT migrate (stays Mac / interactive):** the daily-driver Claude Code sessions, the hybrid on-demand
|
|
delegation, and interactive Confluence edits via the native Atlassian MCP.
|
|
|
|
**What is genuinely NEW on the R720 (not a migration — these never existed in the LangSmith orchestrator):**
|
|
scheduling, the coordinator + shared state/budget, canary/coverage, and subscription-OAuth Claude.
|
|
|
|
Net: the LangSmith orchestrator keeps its job (on-demand routing, non-Claude execution, tracing, connector); the
|
|
R720 becomes the host for everything **scheduled, stateful, and subscription-billed**, and it **reuses the
|
|
LangSmith orchestrator in place** (the local rsync'd copy) for the non-Claude single-shots rather than
|
|
re-implementing them.
|
|
|
|
### 3.3 Plane 2 — the agentic SDLC pipeline (the centerpiece)
|
|
|
|
A durable, human-gated task pipeline hosted on the R720. A task is a long-lived, resumable record; the
|
|
coordinator drives it through stages, asking Adam for input when it is not confident and handing off to the org
|
|
CI to actually build and verify.
|
|
|
|
```
|
|
INTAKE ─► CLARIFIER ─► PLANNER ─► REVIEW LOOP ─► BUILDERS ─► VERIFIERS ─► draft PR + report
|
|
│ │ │ │ │ │
|
|
Mac harness asks Adam phased plan GPT-4.1 + Claude spec org CI builds/tests/
|
|
(SSH-invoke) question- (Claude) multi-model + DeepSeek security-review;
|
|
GitHub issue sets until adversarial; edits ─► verifier reads results;
|
|
checker 98%+, then loops back candidate loops back to builders
|
|
finding HUMAN GATE to planner diff on failure
|
|
```
|
|
|
|
**Stages and model per stage:**
|
|
- **Intake** — a task enters from the Mac harness (D13, first), a GitHub issue (next), or a Plane-1 checker
|
|
finding. It is written as a new task record (LangGraph thread) with a unique `thread_id`.
|
|
- **Clarifier (Claude)** — gathers context (repo, memory, handbook), then asks Adam **question-sets until 98%+
|
|
confident**. This is a LangGraph `interrupt()`: the task suspends and checkpoints, a question-set is delivered
|
|
over the chosen transport (D10), and the task resumes via `Command(resume=...)` when the answer arrives. The
|
|
**human gate**: no progression to build without the clarifier clearing the bar and Adam approving the plan.
|
|
- **Planner (Claude)** — produces a phased plan (the format these design docs use).
|
|
- **Review loop (GPT-4.1 + optional multi-model)** — adversarial plan review (the `sh-plan-review` /
|
|
`cross_reviewer` discipline). Loops back to the planner on REQUEST CHANGES; escalates to Adam if it cannot
|
|
converge.
|
|
- **Builders (Claude spec + DeepSeek edits)** — turn the approved plan into a **candidate diff**. They do not
|
|
write to repos; per D2/D11 they emit the diff for CI.
|
|
- **Verifiers (org CI + a Claude/GPT reader)** — CI (Option-B OIDC) applies the diff on a branch, builds, runs
|
|
tests + the security review + lint; the verifier agent reads the CI results and either loops back to builders
|
|
or advances. Confirmed pass produces a **draft PR** plus a report to Adam.
|
|
|
|
**Durable state (D9).** LangGraph (local) + a SQLite checkpointer. Each stage transition is checkpointed, so a
|
|
crash, a budget pause, or an overnight wait on a human answer all resume cleanly instead of restarting. The task
|
|
record holds: status, current phase, the full Q&A history, the plan, review verdicts, the candidate diff, and CI
|
|
results.
|
|
|
|
**Human-in-the-loop (D10).** A small transport-agnostic responder service owns the notify+resume seam: it posts
|
|
the interrupt's question-set to the channel Adam chose for that task (Slack Block Kit / ticket comment / a Claude
|
|
Code session) and maps his reply back to the right `thread_id` to resume it. Adapters are independent so one can
|
|
ship first (Slack) and the others follow.
|
|
|
|
**Stability + autonomy bounds (the "stable" requirement).** Hard gates, not vibes: (1) no build before the
|
|
clarifier hits 98% AND Adam approves the plan; (2) draft PRs only, never auto-merge; (3) the verifier must pass
|
|
or the task loops/holds, never ships; (4) per-task budget cap inside the shared nightly cap (§6.1); (5) every
|
|
stage checkpointed so failures resume, not restart; (6) a task that stalls (no human answer within a window, or
|
|
N failed build loops) parks and ALARMs rather than spinning. Plane-1's canary/coverage discipline applies to the
|
|
pipeline's agents too.
|
|
|
|
### 3.3.1 Durable human-in-the-loop suspend/resume (resolves B2)
|
|
|
|
LangGraph `interrupt()` + the SQLite checkpointer suspend and resume the graph, but the checkpoint alone does not
|
|
track the human-interaction lifecycle (delivery, duplicate/late answers, expiry). So the pipeline adds one
|
|
durable source of truth, a SQLite `pending_questions` table, and resolves every race with an atomic
|
|
compare-and-set against it. This is the riskiest mechanic, so it is specified here and P1 must prove it.
|
|
|
|
- **Identity.** Each task is a graph `thread_id`. Each question-set gets a `question_id` (uuid) and a monotonic
|
|
`turn` within the task. The interrupt payload carries `{thread_id, question_id, turn, question_set, transport,
|
|
deadline}`.
|
|
- **Ledger.** `pending_questions(question_id PK, thread_id, turn, status[open|answered|expired|superseded],
|
|
transport, channel_ref, posted_at, deadline_at, answer_json, answered_at, answered_via)`. The LangGraph
|
|
checkpoint holds graph state; this table holds the question lifecycle and is what delivery, the responder, and
|
|
recovery read.
|
|
- **Delivery (and lost-post).** On interrupt, write the row `open` first, then post to the chosen transport and
|
|
store its `channel_ref` (Slack message ts / issue-comment id / Claude session id). The posted question embeds
|
|
the `question_id` (Slack `callback_id`; a `<!-- shq:<question_id> -->` marker in a GitHub comment). If the post
|
|
fails, the row stays `open` with no ref and a reconcile loop retries idempotently.
|
|
- **Answer mapping + idempotency (first-answer-wins).** Each transport's inbound adapter normalizes an answer to
|
|
`(question_id, answer, via)`. The responder then runs one atomic statement:
|
|
`UPDATE pending_questions SET status='answered', answer_json=?, answered_via=? WHERE question_id=? AND
|
|
status='open'`. rowcount 1 = first valid answer, enqueue a resume job; rowcount 0 = the question was not open
|
|
(already answered/expired/superseded), so the answer is a duplicate or late and is ignored with a "already
|
|
closed" reply. This single compare-and-set makes duplicate clicks, transport redelivery, answers via two
|
|
channels, and answer-after-timeout all safe. The statement runs inside a `BEGIN IMMEDIATE` transaction (SQLite's
|
|
default deferred isolation does not serialize concurrent responders, so the check-and-set must take the write
|
|
lock up front).
|
|
- **Resume (single-flight, turn-guarded).** A resume worker serializes per `thread_id` and calls
|
|
`graph.invoke(Command(resume=answer), {configurable:{thread_id}})`. Before resuming it checks the live
|
|
checkpoint is still interrupted on this `turn`; if the graph already advanced (stale/redelivered job) it marks
|
|
the question `superseded` and skips. A resume can never double-apply.
|
|
- **Deadline / no-answer.** Each open question has `deadline_at`. A timer loop flips overdue `open` rows to
|
|
`expired` (same compare-and-set) and applies the task policy: park + ALARM Adam, or apply a defined default
|
|
answer. An answer arriving for an already-`expired` question loses the compare-and-set and is ignored. Timeout
|
|
vs answer is a deterministic race on flipping `open`.
|
|
- **Restart recovery.** All state is durable (both SQLite stores), so a reboot converges via a startup sweep:
|
|
retry delivery for `open` rows lacking a ref; re-enqueue resume for `answered` rows whose graph is still
|
|
interrupted on that turn (idempotent via the turn guard); run the deadline policy for overdue `open` rows. No
|
|
in-memory-only state.
|
|
- **Parallel tasks + isolation (Q1).** Each task is its own `thread_id` with its own checkpoint and ledger rows;
|
|
the resume worker serializes per thread but runs different threads concurrently within the budget cap.
|
|
- **Manual path.** A small CLI over the ledger lets an operator list `open`/`parked` questions, re-deliver,
|
|
force-expire, or answer on a task's behalf; a stuck task parks rather than spins. Destructive CLI actions
|
|
(force-expire, answer-on-behalf, force-resume) are audit-logged and require an explicit confirmation flag.
|
|
- **Transport seam (Q2 fallback).** A `Transport` interface (`post_question(...) -> channel_ref`,
|
|
`parse_answer(raw) -> (question_id, answer, via)`) with Slack / GitHub / Claude Code adapters; the ledger +
|
|
resume logic are transport-independent. Adam picks the channel per task at intake. If the chosen transport is
|
|
unreachable, reconcile retries and, after N failures, falls back to a Slack ALARM pointing at the task.
|
|
|
|
### 3.3.2 CI-as-verifier trust boundary (resolves B4)
|
|
|
|
The builders are semi-trusted at best: an LLM that read repo content can be wrong or prompt-injected, so **the
|
|
candidate diff is treated as untrusted code.** The threat is that executing it in CI with org credentials lets a
|
|
bad diff exfiltrate secrets, assume the deploy role, or tamper with other repos. Five boundaries bound it:
|
|
|
|
1. **Split CI: untrusted execution is credential-less; privileged steps never see the patch.** The job that
|
|
checks out and runs the diff (install/build/test) runs with `permissions: contents: read`, **no secrets, no
|
|
OIDC, no write token**, and restricted network egress. The patch executes only here, where there is nothing
|
|
to steal and nothing to assume. Any privileged action (the OIDC role, authoritative status, opening the PR)
|
|
runs in a **separate job that does not check out or execute patch-controlled code**; it consumes the
|
|
build/test report as data only. This is the standard untrusted-code-in-CI ("pwn request") mitigation, so the
|
|
workflow must NOT use `pull_request_target` with a checkout of the head ref.
|
|
2. **The patch may not touch the trust-control surface.** A box-side check and a CI guard both reject any
|
|
candidate diff that modifies `.github/workflows/**`, IAM/policy/permission IaC (CDK/SAM), branch-protection /
|
|
`CODEOWNERS` / Dependabot config, or files outside the task's declared scope. Such a diff is escalated to
|
|
mandatory human review + GPT cross-review, never auto-built (those files are the mandatory-cross-review
|
|
surface regardless). The match is not naive: enforcement is a **CI-side hard fail** (not only the box check),
|
|
it resolves symlinks and canonicalizes paths, and it rejects renames into denied paths and build steps that
|
|
generate files into them, so a path match cannot be bypassed by indirection.
|
|
3. **Diff integrity, box to CI.** The builder hashes the candidate diff and records the hash in the task ledger.
|
|
The box has no write token (D2), so the diff reaches CI as a signed artifact / via a short-lived
|
|
single-purpose token that can only create the agent branch; CI verifies the diff hash matches the ledger
|
|
before applying. A tampered or substituted diff fails the hash check.
|
|
4. **Pass/fail is a pure-code gate over authenticated CI results, not the LLM verifier.** Mirroring secrev's "one
|
|
pure-code script owns the block decision," a deterministic gate reads the CI run conclusion via the GitHub
|
|
Checks/Actions API (authenticated as the box read-only PAT), keyed to the specific run id + diff hash. It
|
|
consumes **only** that authenticated, patch-independent conclusion; it never trusts a success/failure file or
|
|
artifact the patch could have written. The verifier **agent** only reads failures to propose the next fix; it
|
|
cannot declare success.
|
|
5. **Branch protection is the final enforcement.** The draft PR targets a protected branch; the locked-down
|
|
build/test/security-review are required status checks; merge needs them green plus the security-review, the
|
|
Claude Code App review, and human approval. The agent never auto-merges (D2).
|
|
|
|
The CI apply/verify workflow is itself IaC + untrusted-input handling, so before it ships it goes through
|
|
**`/sh-security-review` and the mandatory GPT-4.1 cross-review** (in P3), and its actions are SHA-pinned per the
|
|
handbook Pinning Principle. Provenance (diff hash, CI run id, checks consumed, gate decision) is recorded per
|
|
task; a diff that tried to touch the denylist, a build job that attempted egress, or a checks/hash mismatch is an
|
|
ALARM. Egress restriction in the untrusted job is enforced at the runner level (a harden-runner-style egress
|
|
policy or a self-hosted-runner network deny), but the primary mitigation is that the job holds no secrets, so a
|
|
successful exfil yields nothing of value.
|
|
|
|
**Threat-model honesty (the diff-hash does not cover a compromised box).** The hash protects the box->CI transit
|
|
(tamper/substitution), not a box that is itself compromised: the box is in the trusted computing base for what it
|
|
*proposes*. The real backstops against a bad box are the credential-less build job, the trust-control-surface
|
|
denylist, and above all the **human review of the draft PR** plus required checks (nothing merges without Adam).
|
|
A signed-artifact/attestation step hardens transit further but is not claimed to defend a compromised box.
|
|
|
|
## 4. Roster
|
|
|
|
### Tier 1 — checkers (read-only)
|
|
| Agent | Models | Cadence | Output / gate |
|
|
|---|---|---|---|
|
|
| **compliance-drift** | Gemini scan + Claude judge | nightly | Drift vs engineering-handbook (naming, secrets placement, CI/CD present, Dependabot, branch protection). Report + Slack ALARM on violations (no auto-Jira yet, D3) |
|
|
| **dependency-cve** | Claude + GPT tiebreak | nightly | Cross-ref lockfiles vs advisories org-wide; report + feed fixer. Complements Dependabot |
|
|
| **doc-drift** | Gemini (large context) | weekly | Flags repos whose architecture moved but Confluence/README did not |
|
|
|
|
### Tier 2 — aws-posture (resident, D5) + planner
|
|
| Agent | Models | Cadence | Output |
|
|
|---|---|---|---|
|
|
| **aws-posture** | Sonnet collectors + judge | weekly | Idle/anomalous spend (≈$330/mo flagged) + reasoning layer over baseline findings. Auths via **Roles Anywhere** (short-lived leaf certs, auto-rotated by step-ca). Complements existing GuardDuty/Security Hub/Config, does not replace them |
|
|
| **plan-groomer** | Claude | weekly | Drafts a groomed weekly plan **into the mode-600 report** for now (D3); auto-write to Notion/Jira is a later toggle once trusted |
|
|
| **confluence-doc** | Gemini scan + Claude judge | weekly (scheduled) + on-demand | **Scheduled:** diffs repos + AWS inventory + the page-ID map (`project_confluence_migration`) against Confluence, reports doc gaps / stale pages / missing runbooks (recommend-only, D3). **On-demand (SSH-invoked):** performs an actual update, including Mermaid map edits via `confluence_mermaid.py`. Writes as the IT-space-scoped `confluence-bot` (D6). Overlaps the existing `sh-confluence-audit`/`sh-confluence` skills; the box adds unattended cross-repo scope + the tested Mermaid script |
|
|
|
|
### Tier 3 — fixer (D2)
|
|
| Agent | Models | Trigger | Output |
|
|
|---|---|---|---|
|
|
| **fixer** | DeepSeek edit + Claude spec + GPT review | on a confirmed, low-risk finding | Emits a patch + opens an issue; org CI applies it and opens a **draft** PR. Never auto-merges; pre-push hooks + CI + Claude Code App gate it (F3) |
|
|
|
|
## 5. Coordination model
|
|
|
|
Nightly, after the mirror refresh, the coordinator: (1) loads the shared budget ledger and the **versioned**
|
|
rotation/coverage state (F1); (2) runs the **canary suite first**, one planted-fault corpus per role, a miss is a
|
|
COMPLACENCY ALARM and that role is skipped; (3) fans out the scheduled agents over the mirrors, each with a
|
|
per-call cap, all drawing from **one shared total cap** (critical for the Claude subscription draw, §6.1), with
|
|
budget-exhausted roles deferred via the rotation pointer (never dropped) and a COVERAGE ALARM if a role slips
|
|
past `MAX_CYCLE_NIGHTS`; (4) collects each agent's structured JSON, dedups across agents, prioritizes; (5) routes
|
|
per D3: Slack ALARM for confirmed criticals, everything else to the mode-600 report, and fix-specs to the fixer
|
|
queue if Tier 3 is enabled; (6) a fully clean night posts nothing.
|
|
|
|
**Escalation ladder (resolves Q3).** A confirmed critical that stays unaddressed escalates beyond a one-shot
|
|
Slack ALARM: it re-alarms on a backoff each night it persists, and after `N` nights (default 3) the coordinator
|
|
opens a tracking Jira ticket (INFRA) so it cannot quietly linger. The same ladder applies to a COMPLACENCY or
|
|
COVERAGE alarm that does not clear. Escalation stays ALARM-only in spirit (nothing posts on a clean state).
|
|
|
|
The coordinator holds its own state; it does not rely on the orchestrator (one-shot, stateless). It may *call*
|
|
`orchestrator/run.py` for GPT/Gemini/DeepSeek single-shot sub-tasks, or call those providers directly (Q1: the
|
|
team implements its own coordination; the orchestrator is not refactored for persistence).
|
|
|
|
## 6. Constraints and how this revision answers them
|
|
|
|
### 6.1 Subscription billing draw — shared cap (B1 resolved by D1/D4)
|
|
Headless under Max is permitted for now (D1). Every Claude SDK call still draws from the **same Max pool as
|
|
interactive Claude Code**, so the team runs under **one shared nightly cap across all agents**, and pushes volume
|
|
to Gemini/GPT/DeepSeek (own-account billing) where quality allows. The billing-mode abstraction (D4) lets us flip
|
|
to `api`/`bedrock` when the ToS cutover lands. `ANTHROPIC_API_KEY` stays unset in subscription mode.
|
|
|
|
### 6.2 Fixer write path (B2 resolved by D2)
|
|
Box stays read-only; CI applies the patch and opens the draft PR. The CI workflow + its OIDC role is **new IAM**
|
|
and goes through the **mandatory GPT-4.1 cross-review before it is built** (§7, B3).
|
|
|
|
### 6.3 step-ca + Roles Anywhere (D5)
|
|
New internal CA (step-ca) issues short-lived leaf certs auto-renewed by a systemd timer; the Roles Anywhere
|
|
**trust anchor + the read-only AWS role** are **new IAM** and go through the mandatory cross-review before build
|
|
(B3). The leaf is short-lived (self-expiring), which is stronger than the box's long-lived GitHub PAT.
|
|
|
|
### 6.4 Anti-complacency per role
|
|
Each agent ships with its own canary corpus, versioned in the repo. A role with a failing/stale canary is skipped
|
|
with a COMPLACENCY ALARM, never run silently degraded. **Canary update process (resolves F2):** canary corpora
|
|
are version-controlled **per agent/role** (no shared global corpus, to avoid cross-role confusion); a change to
|
|
any canary goes through a PR + review with a revert point, and because the canary runs every night, a canary edit
|
|
that silently weakens recall is itself caught on the next run.
|
|
|
|
### 6.5 Host capacity
|
|
4GB / 2 vCPU / 40GB. Work is I/O-bound, but more report history + step-ca may pressure disk. Re-check headroom
|
|
after Phase 1; size up the Hyper-V VM (snapshot first per `feedback_ec2_replacement_snapshot` discipline) if
|
|
needed rather than risking the secrev workload.
|
|
|
|
### 6.6 Claude budget realism + contention (resolves B1)
|
|
A multi-stage pipeline draws far more Claude than a single sweep (clarifier loop + planner + verifier-read per
|
|
task), all on the **same Max pool as Adam's interactive Claude Code**. Three controls:
|
|
- **Pre-build measurement (gate before P3 builds anything).** Using the existing `telemetry.py` token capture,
|
|
measure the per-stage Claude token draw on a representative task plus the worst-case clarifier loop, then
|
|
project per-task and daily aggregate at expected task volume. P1/P2 must emit these numbers before P3
|
|
proceeds; the design measures, it does not assume.
|
|
- **One shared daily Claude cap across ALL R720 Claude work** (pipeline + Plane-1 sweeps), tracked in the
|
|
persistent budget ledger. Non-Claude stages are pushed to GPT/Gemini/DeepSeek (own-account billing) to keep
|
|
the Claude draw down.
|
|
- **Interactive-first contention rule.** Adam's interactive Claude Code is never blocked. The box keeps a
|
|
reserve headroom; before starting a stage it checks remaining headroom, and if below the reserve it **parks
|
|
new pipeline tasks and ALARMs** rather than competing for the pool. A task already mid-flight checkpoints and
|
|
pauses at the next stage boundary (never killed). A clarifier is capped at `N` turns per task, then
|
|
escalates/parks, so an ambiguous task cannot loop-drain the pool.
|
|
- **Fairness + no starvation.** The reserve is a fixed configured fraction of the daily cap (not guessed at
|
|
runtime). Parked tasks are FIFO-aged with a `MAX_PARK` window; a task that exceeds it escalates (ALARM, and a
|
|
Jira ticket per §5) rather than starving silently, and an operator can force-resume or re-prioritize it via the
|
|
CLI. A clarifier parked at the turn cap is resumable the same way: Adam adds context and re-opens it, so it is
|
|
never an indefinite stall.
|
|
|
|
### 6.7 State durability + backup (resolves F1)
|
|
All durable state (the LangGraph SQLite checkpoint, the `pending_questions` ledger, the budget ledger, the
|
|
Plane-1 rotation/coverage pointer) is written atomically (write-temp-then-rename), integrity-checked on load, and
|
|
included in the nightly offsite backup (mode 600). On corruption the coordinator refuses to proceed silently: the
|
|
rotation pointer is rebuildable from the report history, and a corrupt task checkpoint parks that task with an
|
|
ALARM rather than restarting it blindly. "Integrity-checked" is concrete: schema-version match + a stored content
|
|
hash + a logical-consistency check (e.g. no `answered` question whose graph is already past that turn). After a
|
|
restore, a reconciliation step re-syncs against external state (in-flight CI runs, current GitHub PR status)
|
|
before any task resumes, so a restored backup cannot act on stale external assumptions.
|
|
|
|
## 7. Phased rollout (re-sequenced for provisioning order, rollback, cross-review, and docs-as-you-go)
|
|
|
|
- **Phase 0 — substrate factoring (with rollback, B4).** Back up `nightly_sweep.sh` (tag a revert point);
|
|
extract discovery/mirror/budget-ledger/rotation/Slack/canary into a shared module used by both secrev and the
|
|
team. **Gate:** secrev passes its canary + existing behavior after the refactor, else revert. No team behavior
|
|
yet.
|
|
- **Phase 1 — one checker end to end.** Build `compliance-drift` + its canary + the mode-600 report path + a
|
|
**routing dry-run** (F4) for the Slack alarm. Dry-run on the mirrors. Proves the substrate generalizes.
|
|
**Create the `project_r720_agent_team` memory now** (B5/docs-as-you-go).
|
|
- **Phase 2 — coordinator + second checker.** Add the coordinator (shared budget, dedup, versioned state F1) and
|
|
`dependency-cve`. **Run a forced budget-squeeze dry-run** to prove deferral-not-drop + COVERAGE ALARM (F2).
|
|
- **Phase 3 — doc-drift + step-ca/Roles Anywhere + aws-posture.** Stand up step-ca and the Roles Anywhere trust
|
|
anchor + read-only AWS role; **cross-review the IAM before building** the agent (B3). Wire aws-posture. Add
|
|
doc-drift.
|
|
- **Phase 4 — planner + confluence-doc.** `plan-groomer` writing into the report only (D3). For
|
|
`confluence-doc`: create the `confluence-bot` service account with IT-space-only edit rights (D6), put its
|
|
token in `~/secrev.env`; ship the scheduled gap-detection (read-only, recommend) first, then wire the
|
|
on-demand SSH-invoked write path. The Mermaid script (`~/.claude/scripts/confluence_mermaid.py`, already
|
|
written and offline-tested) must pass a **live dry-run against page 1540098** (verify it lists all 16 weweave
|
|
macros and that a no-op set is clean) before any `--apply`. Notion/Jira auto-write stays a later toggle.
|
|
- **Phase 5 — fixer (D2).** Build the org CI apply-and-open-draft-PR workflow; **cross-review its OIDC IAM
|
|
before building** (B3). Confirm fixer PRs hit pre-push hooks + CI + Claude Code App (F3). Start with the
|
|
narrowest fix class (dep bumps). Draft PRs only.
|
|
- **Phase 6 — document as standing infra.** Update Confluence (the team **and** the still-undocumented secrev
|
|
host) in the IT host/LAN inventory; write the operator runbook. **The runbook must cover incident handling
|
|
(resolves F4):** pipeline stalls, stuck/parked tasks, failed human-in-the-loop resumes, budget exhaustion
|
|
mid-pipeline, transport outages, and COMPLACENCY/COVERAGE alarms, each with the manual CLI recovery steps
|
|
(§3.3.1) and the escalation ladder (§5).
|
|
|
|
**Provisioning + rollback + cross-review gate (resolves B3/B5/F5), applied to every phase:**
|
|
- **Cross-review is a hard gate, not a note.** Any new IAM role/policy, trust anchor, OIDC role, or permission
|
|
change is provisioned AND passes the mandatory GPT-4.1 cross-review (plus `/sh-security-review` where it
|
|
touches the CI / untrusted-input surface) **before** any code that depends on it is built. A phase cannot
|
|
start its dependent work until that review is recorded.
|
|
- **Every stateful phase has a revert point.** Not just Phase 0: before standing up step-ca, the Roles Anywhere
|
|
trust anchor + AWS role, or the CI apply-workflow, capture a documented rollback (remove the role/CA, restore
|
|
the prior workflow, revert the cert config) and gate the phase on a successful dry-run. The rollback is
|
|
**exercised** (sandbox or simulated teardown/re-provision), not merely written, before the phase is accepted.
|
|
VM changes snapshot first per `feedback_ec2_replacement_snapshot`.
|
|
- **Docs land with the change, enforced as definition-of-done.** A phase is not "done" until its memory entry
|
|
and the relevant Confluence page are updated; that update is a checklist item in the phase, not deferred
|
|
(Phase 6 is only the final standing-infra writeup).
|
|
|
|
### 7.1 Pipeline track (Plane 2) — depends only on Phase 0 substrate
|
|
|
|
This track is largely independent of the Plane-1 checker phases (1-6); both build on the Phase 0 substrate.
|
|
Given the north star, **Adam may prioritize this track first.** Sequencing within it:
|
|
|
|
- **Phase P1 — skeleton + the human gate.** LangGraph graph + SQLite checkpointer on the box; one trivial task
|
|
type; Mac SSH-invoke intake; the **clarifier** with `interrupt()`/resume over **one** transport (Slack first).
|
|
Stops at an approved plan, no build yet. This proves durable suspend/resume across a real human answer (the
|
|
riskiest mechanic) before anything else. **Exit criteria (must demonstrate §3.3.1):** (a) kill the box
|
|
mid-wait and have the task resume after restart; (b) submit a duplicate answer and confirm it no-ops; (c)
|
|
submit an answer after the deadline expired and confirm it is rejected and the task parked; (d) two tasks
|
|
suspended concurrently resume independently to the correct thread.
|
|
- **Phase P2 — planner + review loop.** Wire the planner and the GPT-4.1 review loop (reuse `cross_reviewer`),
|
|
including loop-back and the escalate-to-Adam path.
|
|
- **Phase P3 — builders + verifier via org CI.** Builders emit a candidate diff; the Option-B OIDC workflow
|
|
builds/tests/security-reviews; the verifier reads CI results and produces a draft PR. Start with the narrowest
|
|
task class (e.g. a dependency bump or a single-file fix), draft PRs only. **Build the §3.3.2 trust boundary:**
|
|
split untrusted/privileged CI jobs, diff-hash integrity, the trust-control-surface denylist, and the pure-code
|
|
pass/fail gate. The CI apply/verify workflow + its OIDC role go through **`/sh-security-review` AND the
|
|
mandatory GPT-4.1 cross-review** before this phase ships (it is IaC/IAM + untrusted-input handling).
|
|
- **Phase P4 — more transports + GitHub intake.** Add the ticket-comment and Claude-Code responder adapters
|
|
(D10) and GitHub-issue intake (D13).
|
|
- **Phase P5 — checker findings as a task source.** Let a confirmed Plane-1 finding open a pipeline task, closing
|
|
the loop between the two planes.
|
|
|
|
Observability for both planes (D12): LangSmith stays off; the existing local JSONL (`telemetry.py`) covers the
|
|
LangChain/LangGraph path, the Agent-SDK path keeps its own run logs, and self-hosted Phoenix is an optional
|
|
later add if per-run trace UI is wanted.
|
|
|
|
## 8. Open items folded in (no longer blocking)
|
|
- Shared vs separate timer: **shared** with secrev (one discovery/mirror pass, one shared budget); error
|
|
isolation handled by per-role try/skip + canary, documented in the runbook (N2).
|
|
- Read-only PAT sufficiency (Q2): confirm the existing fine-grained PAT covers all mirrors before Phase 1; it
|
|
already clones every non-archived org repo for secrev, so this is a verification step, not a change.
|
|
|
|
## 9. Obligations on build (per global instructions)
|
|
- **Memory:** `project_r720_agent_team` created in Phase 1; cross-link `project_security_review_agent`,
|
|
`project_orchestration_migration`, `reference_claude_subscription_billing`, `feedback_cloudwatch_alarms`.
|
|
- **Confluence:** document the team (and secrev) as standing infra (always-on VM holding read-only org PAT + now
|
|
a Roles Anywhere AWS identity).
|
|
- **Handbook/naming:** kebab-case dirs/resources (`agent-team`), snake_case importable Python modules; secrets in
|
|
`.env`/Secrets Manager/`~/secrev.env` (mode 600), never committed; CI/CD for the fixer apply-workflow.
|
|
- **Cross-review:** the fixer CI OIDC role and the Roles Anywhere trust anchor + AWS read role each go through the
|
|
mandatory GPT-4.1 cross-review before their phase builds.
|
|
- **Service-account lifecycle (F3):** the `confluence-bot` token is rotated on a schedule (90 days, calendared
|
|
like the GitHub PAT), has a documented revocation step, keeps its edits attributable in Confluence page
|
|
history, and is decommissioned if the agent is retired.
|
|
- **Backups (F1):** the durable state stores (checkpoint, ledgers, rotation pointer) are included in the nightly
|
|
offsite backup.
|