This repository has been archived on 2026-08-04. You can view files and clone it, but cannot push or open issues or pull requests.
orchestrator/agent-team/agent_team/dispatcher.py
Adam Moussa 03b9a94881
feat(agent-team): P3-flip Phase 1 — CI trust-boundary hardening (WIP, gated) (#34)
* feat(agent-team): P3-flip Phase 1 — expand denylist vectors (§4.2) + runner-trust assertion (§4.1)

First controls of the P3-live-flip Phase-1 CI hardening (workflow stays INERT;
this only tightens the trust boundary). Whole Phase-1 surface is gated by
/sh-security-review + GPT-4.1 cross-review before any flip.

§4.2 — expand the trust-control denylist with direct code-execution / supply-chain
vectors, kept byte-identical across all three copies (ci_gate.DENYLIST_GLOBS + the
guard + post-build inline DENY_GLOBS), drift-guarded:
  .gitmodules, .husky/**, .githooks/**, .gitattributes, .npmrc, and generated/build
  artifacts (__generated__, *.generated.*, dist/**, build/**, *.min.js).
Deliberate: lockfiles are NOT wholesale denied — lockfile-postinstall RCE is already
contained by the credential-less egress-blocked build sandbox, and the Tier-3 dep-CVE
fixer rewrites lockfiles to produce its draft PRs; a blanket deny would make it
un-shippable. Flagged in-code for the security gate. Direct code-execution config
(hooks/filters/npmrc/submodules) is the actual §4.2 RCE surface.

§4.1 — runner-trust: assert no job (esp. the privileged gate-and-pr) can run on a
self-hosted/user-provided runner; all must be GitHub-hosted.

998 tests pass, ruff clean.

* feat(agent-team): P3-flip Phase 1 — gate-weakening detector (§4.5)

A diff that ADDS a lint/type/coverage/security suppression (noqa, type: ignore,
pragma: no cover, nosec, nosemgrep), a test skip/xfail, or a hook bypass
(--no-verify) could make CI pass falsely. The pure-code gate now flags these via
gate_weakening_violations() and BLOCKs in evaluate_ci_gate as a top-priority trust
violation (step 1b, alongside the denylist) — regardless of the authenticated CI
conclusion. A build cannot pass itself by disabling its own checks; flagged diffs
escalate to a human. Only ADDED lines are inspected (removing a suppression is fine).

1015 tests pass, ruff clean.

* feat(agent-team): P3-flip — diff transport (§4.3) + flip privileged apply path live

Completes the box->CI diff handoff and flips the apply/verify privileged job
live (gated behind the agent-apply environment's required reviewer).

Transport (§4.3): the read-only box (D2) emits a diff but holds no write token.
- New credential-less `materialize` job decodes the untrusted `diff_b64`
  dispatch input via env (CWE-94), fail-closed re-hashes it against
  `expected_diff_hash`, and uploads it as the named artifact so guard/build-test
  download it same-run. guard now `needs: materialize`.
- New `dispatcher.py` (the trusted apply path, operator/Mac-side — never the
  box): pushes the diff as a head branch then `gh workflow run`s the workflow.
  Pure input-assembly (sha256 == sha256sum, b64 round-trip, head ref) is
  unit-tested; git/gh are injected seams. Push-before-dispatch; fail-closed on
  empty diff/scope, unsafe task_id/owner/repo.

Flip: gate-and-pr binds `environment: agent-apply` (required reviewer
amoussa1229) + grants exactly `pull-requests: write`; the App-token + draft-PR
steps run only on `steps.gate.outputs.gate == 'pass'` (no more if:false); the
draft PR opens with an explicit `--head`; task_id/head_branch charset-validated
(§4.6). Updated the hardening tests from inert-state to live-state assertions +
added transport tests. 1039 tests, ruff clean, workflow YAML valid.

NOTE: workflow only runs on manual workflow_dispatch and the privileged job is
held at the required-reviewer gate, so nothing privileged runs unapproved.

* fix(agent-team): P3-flip — address GPT-4.1 cross-review (size bound, ref-traversal guard)

- BLOCK: cap candidate diff at 40 KB in the dispatcher (the diff rides a base64
  workflow_dispatch input; GitHub caps inputs at ~64 KB so an oversized diff
  cannot dispatch at all) + a defense-in-depth decoded-size bound in materialize.
- FIX: harden the draft-PR HEAD_BRANCH guard to reject leading/trailing slash,
  '..' segments, and '//' (CWE-88 git ref-traversal), not just bad charset.
- NIT: document the mandatory invariants on gate-and-pr (required-reviewer
  environment must stay; runs-on must stay GitHub-hosted).
- QUESTION (lockfiles): answered in-code — the build-test sandbox is
  credential-less + egress-blocked, so lockfile-postinstall RCE is contained.
Tests added for all guards. 1042 tests, ruff clean, YAML valid.

* fix(agent-team): P3-flip — resolve /sh-security-review findings (LOGIC-1/2/3)

High-recall fan-out (injection/logic/iac+secrets) + proof-or-kill on the LIVE
apply path found 3 real issues the cross-review missed; all fixed:

- LOGIC-2 (HIGH, was a live hole): build-test ran `ruff check . || echo` /
  `pytest -q || echo`, swallowing failures so the job was always 'success' and
  the gate would open draft PRs on RED builds. ruff/pytest now run
  authoritatively under set -e (pytest exit 5 'no tests' is the only non-fatal
  case); the exit code IS the build-test conclusion the gate keys on.
- LOGIC-1 (verified!=shipped): the dispatcher used `git apply` + `git add -A`,
  staging stray untracked content into the pushed PR head. Now `git apply
  --index` stages exactly the diff, so the head tree is precisely base+diff —
  bound to the bytes CI hash-verified.
- LOGIC-3 (§4.5 on the live path): gate-weakening was enforced only box-side;
  added a gate-weakening check to the guard job so the live PR-opening path
  rejects a diff that adds suppressions/skips, even on a green build.

Injection / secrets / least-privilege / flip-correctness / no-untrusted-checkout
all came back clean. 1044 tests, ruff clean, YAML valid.
2026-06-22 18:51:52 -04:00

319 lines
12 KiB
Python

"""Trusted apply-path dispatcher (§4.3): carry a box-emitted diff into org CI.
The read-only R720 box (D2) EMITS a candidate diff + the
``workflow_dispatch`` inputs but holds **no write token**. This dispatcher is
the ONLY component that writes, and it runs on a TRUSTED host (the operator's
Mac, where the GitHub App key / operator ``gh`` auth lives) — never on the box.
It does two things and nothing else:
1. Pushes the candidate diff as a short-lived **head branch** (the PR head).
2. Triggers the ``agent-team-apply-verify.yml`` ``workflow_dispatch``, passing
the diff as ``diff_b64`` plus the integrity inputs.
The CI workflow then RE-VERIFIES everything (materialize re-hashes; guard
re-hashes + denylist + scope; build-test builds/tests credential-less; the
pure-code gate decides pass/fail; the privileged job opens a DRAFT PR only on a
clean pass, gated by the ``agent-apply`` environment's required reviewer). This
dispatcher TRANSPORTS only — it makes no trust decision.
Design discipline (mirrors the rest of agent_team): the network/SDK/subprocess
side effects are behind INJECTED seams so the pure input-assembly + validation
logic is fully unit-testable with no git, no ``gh``, and no network. The real
default seams shell out to ``git`` / ``gh`` and are exercised only in
production.
"""
from __future__ import annotations
import base64
import re
from dataclasses import dataclass
from typing import Protocol
from agent_team.state_store import compute_content_hash
__all__ = [
"DispatchInputs",
"DispatcherError",
"build_dispatch_inputs",
"dispatch_apply_verify",
"head_branch_for",
]
WORKFLOW_FILE = "agent-team-apply-verify.yml"
# The artifact carries exactly this filename; the workflow's materialize/guard
# steps look for ``candidate.diff`` (mirrors nodes.fixer.CANDIDATE_DIFF_FILENAME).
CANDIDATE_DIFF_FILENAME = "candidate.diff"
# Max candidate-diff size. NOT arbitrary: the diff is carried as the base64
# `diff_b64` workflow_dispatch INPUT, and GitHub caps total dispatch inputs at
# ~65,535 bytes. base64 inflates ~4/3, so a diff over ~45 KB cannot be
# dispatched at all. We cap at 40 KB (leaving headroom for the other inputs) and
# fail closed with a clear error rather than letting GitHub reject the dispatch
# opaquely — and to bound resource use on the operator host. Larger diffs need a
# branch-only transport (future), not an inline input.
MAX_DIFF_BYTES = 40_000
# task_id must match the workflow's gate-and-pr safety guard charset (it is
# embedded in the head ref and the PR text). A coordinator thread_id is a uuid,
# well within this set; we validate to fail closed on anything else.
_TASK_ID_RE = re.compile(r"\A[A-Za-z0-9._-]{1,200}\Z")
# owner/repo path-segment charset (mirrors ci_fetcher / github_intake).
_OWNER_REPO_RE = re.compile(r"\A[A-Za-z0-9_.-]{1,100}\Z")
class DispatcherError(Exception):
"""Raised on structurally invalid dispatch inputs (fail closed, never proceed)."""
def head_branch_for(task_id: str) -> str:
"""Return the head-branch ref the dispatcher pushes for ``task_id``.
A stable, namespaced ref so a re-dispatch for the same task reuses/replaces
one branch rather than littering refs. Validated to the safe charset so the
ref can never carry shell/ref metacharacters.
"""
if not _TASK_ID_RE.match(task_id or ""):
raise DispatcherError(
f"invalid task_id {task_id!r}: must match {_TASK_ID_RE.pattern}"
)
return f"agent-team/apply/{task_id}"
@dataclass(frozen=True)
class DispatchInputs:
"""The exact ``workflow_dispatch`` inputs for the apply/verify workflow.
Field names match ``on.workflow_dispatch.inputs`` 1:1 so :meth:`as_inputs`
can be handed straight to ``gh workflow run -f k=v``.
"""
task_id: str
diff_artifact_name: str
expected_diff_hash: str
declared_scope: str
diff_b64: str
head_branch: str
def as_inputs(self) -> dict[str, str]:
return {
"task_id": self.task_id,
"diff_artifact_name": self.diff_artifact_name,
"expected_diff_hash": self.expected_diff_hash,
"declared_scope": self.declared_scope,
"diff_b64": self.diff_b64,
"head_branch": self.head_branch,
}
def build_dispatch_inputs(
*,
task_id: str,
diff_text: str,
declared_scope: str,
artifact_prefix: str = "agent-team-diff",
) -> DispatchInputs:
"""Assemble the dispatch inputs from a task id + the candidate diff (PURE).
Computes the sha256 the workflow binds to (``expected_diff_hash``), base64s
the diff (``diff_b64``, carried as an input so the credential-less
materialize job can re-create the artifact), derives the head branch, and
names the artifact. No I/O, no network — fully testable.
Fails closed (:class:`DispatcherError`) on an empty diff / empty scope /
unsafe task id, so a malformed task can never be transported.
"""
if not isinstance(diff_text, str) or not diff_text.strip():
raise DispatcherError("diff_text must be a non-empty diff")
if not isinstance(declared_scope, str) or not declared_scope.strip():
raise DispatcherError(
"declared_scope must be non-empty (an empty scope allows any path)"
)
head_branch = head_branch_for(task_id) # validates task_id
raw = diff_text.encode("utf-8")
if len(raw) > MAX_DIFF_BYTES:
raise DispatcherError(
f"diff too large ({len(raw)} bytes > {MAX_DIFF_BYTES}); the diff is "
"carried as a base64 workflow_dispatch input (GitHub caps inputs at "
"~64 KB). Split the change or use a branch-only transport."
)
return DispatchInputs(
task_id=task_id,
diff_artifact_name=f"{artifact_prefix}-{task_id}",
expected_diff_hash=compute_content_hash(raw),
declared_scope=declared_scope,
diff_b64=base64.b64encode(raw).decode("ascii"),
head_branch=head_branch,
)
class BranchPusher(Protocol):
"""Pushes the candidate diff as the head branch (the only WRITE)."""
def __call__(
self, *, owner: str, repo: str, base: str, head_branch: str, diff_text: str
) -> None: ...
class WorkflowDispatcher(Protocol):
"""Triggers the apply/verify ``workflow_dispatch`` with the assembled inputs."""
def __call__(
self, *, owner: str, repo: str, inputs: dict[str, str], ref: str
) -> None: ...
def dispatch_apply_verify(
*,
owner: str,
repo: str,
task_id: str,
diff_text: str,
declared_scope: str,
base: str = "main",
pusher: BranchPusher | None = None,
dispatcher: WorkflowDispatcher | None = None,
) -> DispatchInputs:
"""Transport one candidate diff into org CI: push the head branch, dispatch.
The trusted apply path. Validates owner/repo, assembles the dispatch inputs,
pushes the diff as the head branch via ``pusher``, then triggers the
workflow via ``dispatcher``. Returns the :class:`DispatchInputs` used (for
the ledger/audit). ``pusher`` / ``dispatcher`` are injected so this is
testable without git/gh/network; the real defaults shell out to git/gh.
NOTE: this never runs on the box (the box has no write token, D2). CI owns
every trust decision; this only moves bytes.
"""
if not _OWNER_REPO_RE.match(owner or "") or not _OWNER_REPO_RE.match(repo or ""):
raise DispatcherError(f"invalid owner/repo {owner!r}/{repo!r}")
inputs = build_dispatch_inputs(
task_id=task_id, diff_text=diff_text, declared_scope=declared_scope
)
push = pusher if pusher is not None else _default_branch_pusher()
fire = dispatcher if dispatcher is not None else _default_workflow_dispatcher()
# Push the head branch FIRST: the draft-PR step opens against an
# already-pushed --head, so the branch must exist before the run reaches it.
push(
owner=owner,
repo=repo,
base=base,
head_branch=inputs.head_branch,
diff_text=diff_text,
)
fire(owner=owner, repo=repo, inputs=inputs.as_inputs(), ref=base)
return inputs
def _default_branch_pusher() -> BranchPusher:
"""Real pusher: clone-free apply of the diff onto a fresh branch via git/gh.
Deferred to call time (no subprocess at import). Uses the operator's git
credentials / the GitHub App token present on the trusted host — NEVER a
box-held token. Implemented as a thin shell-out; the heavy lifting is the
pure :func:`build_dispatch_inputs` above, so this stays small.
"""
def _push(
*, owner: str, repo: str, base: str, head_branch: str, diff_text: str
) -> None:
import subprocess
import tempfile
from pathlib import Path
# Operator host only. Use a throwaway worktree, apply the diff, push the
# branch with the host's credentials. Kept intentionally minimal; the
# security comes from CI re-verifying the pushed content by hash.
with tempfile.TemporaryDirectory() as tmp:
tmpdir = Path(tmp)
diff_path = tmpdir / CANDIDATE_DIFF_FILENAME
diff_path.write_text(diff_text, encoding="utf-8")
clone = tmpdir / "repo"
subprocess.run(
[
"gh",
"repo",
"clone",
f"{owner}/{repo}",
str(clone),
"--",
"--depth",
"1",
"--branch",
base,
],
check=True,
capture_output=True,
)
subprocess.run(
["git", "-C", str(clone), "checkout", "-B", head_branch],
check=True,
capture_output=True,
)
# --index applies AND stages exactly the diff's changes (incl. new
# files) and NOTHING else — so the committed head tree is precisely
# base+diff, never stray untracked worktree content. This keeps the
# PR head bound to the same bytes CI hash-verified (LOGIC-1). No
# separate `git add -A` (which would stage unrelated content).
subprocess.run(
["git", "-C", str(clone), "apply", "--index", str(diff_path)],
check=True,
capture_output=True,
)
subprocess.run(
[
"git",
"-C",
str(clone),
"-c",
"user.name=agent-team",
"-c",
"user.email=agent-team@seahavenind.com",
"commit",
"-m",
f"agent-team apply: {head_branch}",
],
check=True,
capture_output=True,
)
subprocess.run(
[
"git",
"-C",
str(clone),
"push",
"--force-with-lease",
"origin",
head_branch,
],
check=True,
capture_output=True,
)
return _push
def _default_workflow_dispatcher() -> WorkflowDispatcher:
"""Real dispatcher: ``gh workflow run`` (operator auth has ``actions:write``)."""
def _fire(*, owner: str, repo: str, inputs: dict[str, str], ref: str) -> None:
import subprocess
args = [
"gh",
"workflow",
"run",
WORKFLOW_FILE,
"--repo",
f"{owner}/{repo}",
"--ref",
ref,
]
for key, value in inputs.items():
args += ["-f", f"{key}={value}"]
subprocess.run(args, check=True, capture_output=True)
return _fire