This repository has been archived on 2026-08-04. You can view files and clone it, but cannot push or open issues or pull requests.
orchestrator/agent-team/tests
Adam Moussa 03b9a94881
feat(agent-team): P3-flip Phase 1 — CI trust-boundary hardening (WIP, gated) (#34)
* feat(agent-team): P3-flip Phase 1 — expand denylist vectors (§4.2) + runner-trust assertion (§4.1)

First controls of the P3-live-flip Phase-1 CI hardening (workflow stays INERT;
this only tightens the trust boundary). Whole Phase-1 surface is gated by
/sh-security-review + GPT-4.1 cross-review before any flip.

§4.2 — expand the trust-control denylist with direct code-execution / supply-chain
vectors, kept byte-identical across all three copies (ci_gate.DENYLIST_GLOBS + the
guard + post-build inline DENY_GLOBS), drift-guarded:
  .gitmodules, .husky/**, .githooks/**, .gitattributes, .npmrc, and generated/build
  artifacts (__generated__, *.generated.*, dist/**, build/**, *.min.js).
Deliberate: lockfiles are NOT wholesale denied — lockfile-postinstall RCE is already
contained by the credential-less egress-blocked build sandbox, and the Tier-3 dep-CVE
fixer rewrites lockfiles to produce its draft PRs; a blanket deny would make it
un-shippable. Flagged in-code for the security gate. Direct code-execution config
(hooks/filters/npmrc/submodules) is the actual §4.2 RCE surface.

§4.1 — runner-trust: assert no job (esp. the privileged gate-and-pr) can run on a
self-hosted/user-provided runner; all must be GitHub-hosted.

998 tests pass, ruff clean.

* feat(agent-team): P3-flip Phase 1 — gate-weakening detector (§4.5)

A diff that ADDS a lint/type/coverage/security suppression (noqa, type: ignore,
pragma: no cover, nosec, nosemgrep), a test skip/xfail, or a hook bypass
(--no-verify) could make CI pass falsely. The pure-code gate now flags these via
gate_weakening_violations() and BLOCKs in evaluate_ci_gate as a top-priority trust
violation (step 1b, alongside the denylist) — regardless of the authenticated CI
conclusion. A build cannot pass itself by disabling its own checks; flagged diffs
escalate to a human. Only ADDED lines are inspected (removing a suppression is fine).

1015 tests pass, ruff clean.

* feat(agent-team): P3-flip — diff transport (§4.3) + flip privileged apply path live

Completes the box->CI diff handoff and flips the apply/verify privileged job
live (gated behind the agent-apply environment's required reviewer).

Transport (§4.3): the read-only box (D2) emits a diff but holds no write token.
- New credential-less `materialize` job decodes the untrusted `diff_b64`
  dispatch input via env (CWE-94), fail-closed re-hashes it against
  `expected_diff_hash`, and uploads it as the named artifact so guard/build-test
  download it same-run. guard now `needs: materialize`.
- New `dispatcher.py` (the trusted apply path, operator/Mac-side — never the
  box): pushes the diff as a head branch then `gh workflow run`s the workflow.
  Pure input-assembly (sha256 == sha256sum, b64 round-trip, head ref) is
  unit-tested; git/gh are injected seams. Push-before-dispatch; fail-closed on
  empty diff/scope, unsafe task_id/owner/repo.

Flip: gate-and-pr binds `environment: agent-apply` (required reviewer
amoussa1229) + grants exactly `pull-requests: write`; the App-token + draft-PR
steps run only on `steps.gate.outputs.gate == 'pass'` (no more if:false); the
draft PR opens with an explicit `--head`; task_id/head_branch charset-validated
(§4.6). Updated the hardening tests from inert-state to live-state assertions +
added transport tests. 1039 tests, ruff clean, workflow YAML valid.

NOTE: workflow only runs on manual workflow_dispatch and the privileged job is
held at the required-reviewer gate, so nothing privileged runs unapproved.

* fix(agent-team): P3-flip — address GPT-4.1 cross-review (size bound, ref-traversal guard)

- BLOCK: cap candidate diff at 40 KB in the dispatcher (the diff rides a base64
  workflow_dispatch input; GitHub caps inputs at ~64 KB so an oversized diff
  cannot dispatch at all) + a defense-in-depth decoded-size bound in materialize.
- FIX: harden the draft-PR HEAD_BRANCH guard to reject leading/trailing slash,
  '..' segments, and '//' (CWE-88 git ref-traversal), not just bad charset.
- NIT: document the mandatory invariants on gate-and-pr (required-reviewer
  environment must stay; runs-on must stay GitHub-hosted).
- QUESTION (lockfiles): answered in-code — the build-test sandbox is
  credential-less + egress-blocked, so lockfile-postinstall RCE is contained.
Tests added for all guards. 1042 tests, ruff clean, YAML valid.

* fix(agent-team): P3-flip — resolve /sh-security-review findings (LOGIC-1/2/3)

High-recall fan-out (injection/logic/iac+secrets) + proof-or-kill on the LIVE
apply path found 3 real issues the cross-review missed; all fixed:

- LOGIC-2 (HIGH, was a live hole): build-test ran `ruff check . || echo` /
  `pytest -q || echo`, swallowing failures so the job was always 'success' and
  the gate would open draft PRs on RED builds. ruff/pytest now run
  authoritatively under set -e (pytest exit 5 'no tests' is the only non-fatal
  case); the exit code IS the build-test conclusion the gate keys on.
- LOGIC-1 (verified!=shipped): the dispatcher used `git apply` + `git add -A`,
  staging stray untracked content into the pushed PR head. Now `git apply
  --index` stages exactly the diff, so the head tree is precisely base+diff —
  bound to the bytes CI hash-verified.
- LOGIC-3 (§4.5 on the live path): gate-weakening was enforced only box-side;
  added a gate-weakening check to the guard job so the live PR-opening path
  rejects a diff that adds suppressions/skips, even on a green build.

Injection / secrets / least-privilege / flip-correctness / no-untrusted-checkout
all came back clean. 1044 tests, ruff clean, YAML valid.
2026-06-22 18:51:52 -04:00
..
sim Prove P1 exit criteria against the real LangGraph graph; fix question_id stability 2026-06-17 15:16:12 -04:00
__init__.py Plane 2 foundation: interfaces, SQLite schemas, state-store, billing seam 2026-06-17 15:16:12 -04:00
conftest.py Plane 2 foundation: interfaces, SQLite schemas, state-store, billing seam 2026-06-17 15:16:12 -04:00
test_apply_verify_workflow_hardening.py feat(agent-team): P3-flip Phase 1 — CI trust-boundary hardening (WIP, gated) (#34) 2026-06-22 18:51:52 -04:00
test_billing.py Plane 2 foundation: interfaces, SQLite schemas, state-store, billing seam 2026-06-17 15:16:12 -04:00
test_build_verify_subgraph.py feat(agent-team): P3-inert build/verify subgraph topology (opt-in, no live CI) 2026-06-18 13:23:03 -04:00
test_builders.py Resolve security-review BLOCK: CI-guard bypasses, denylist parity, force-resume 2026-06-17 15:16:12 -04:00
test_builders_llm.py feat(agent-team): bind planner/review/builder/verifier nodes to their models 2026-06-18 12:56:42 -04:00
test_checker_intake.py feat(agent-team): P5 cross-plane loop — checker finding -> pipeline task (opt-in) (#18) 2026-06-18 15:56:52 -04:00
test_ci_fetcher.py feat(agent-team): P3-live CI apply/verify hardening + ci_fetcher (gate-passed, provisioning-gated) (#17) 2026-06-18 15:53:26 -04:00
test_ci_gate.py feat(agent-team): P3-flip Phase 1 — CI trust-boundary hardening (WIP, gated) (#34) 2026-06-22 18:51:52 -04:00
test_ci_gate_workflow.py feat(agent-team): P3-live CI apply/verify hardening + ci_fetcher (gate-passed, provisioning-gated) (#17) 2026-06-18 15:53:26 -04:00
test_clarifier.py Add Plane-2 leaf scaffold (pipeline graph, nodes, HITL, transports, CI) 2026-06-17 15:16:12 -04:00
test_clarifier_llm.py feat(agent-team): bind P1 clarifier to real Claude via subscription-OAuth invoker 2026-06-18 12:56:42 -04:00
test_claude_code_adapter.py Add Plane-2 leaf scaffold (pipeline graph, nodes, HITL, transports, CI) 2026-06-17 15:16:12 -04:00
test_claude_code_live.py feat(agent-team): P4 live github/claude_code transports + GitHub-issue intake 2026-06-18 13:23:02 -04:00
test_coordinator.py fix(agent-team): harden listener respawn/close, broaden handle_event guard, channel_ref partial-unique (#29) 2026-06-22 16:14:22 -04:00
test_deadline_timer.py Add Plane-2 leaf scaffold (pipeline graph, nodes, HITL, transports, CI) 2026-06-17 15:16:12 -04:00
test_dispatcher.py feat(agent-team): P3-flip Phase 1 — CI trust-boundary hardening (WIP, gated) (#34) 2026-06-22 18:51:52 -04:00
test_fixer.py feat(agent-team): Plane-1 fixer — finding→patch→CI draft-PR (dep-bumps, opt-in/inert) (#21) 2026-06-18 16:17:33 -04:00
test_github_adapter.py Add Plane-2 leaf scaffold (pipeline graph, nodes, HITL, transports, CI) 2026-06-17 15:16:12 -04:00
test_github_intake.py feat(agent-team): durable GitHub-issue intake de-dup + intake hardening (#32) 2026-06-22 17:20:43 -04:00
test_github_live.py feat(agent-team): P4 live github/claude_code transports + GitHub-issue intake 2026-06-18 13:23:02 -04:00
test_graph.py fix(agent-team): register QuestionSet with the langgraph checkpoint serializer (silence/avoid msgpack block) (#30) 2026-06-22 16:14:27 -04:00
test_invoker.py feat(agent-team): bind P1 clarifier to real Claude via subscription-OAuth invoker 2026-06-18 12:56:42 -04:00
test_ledger.py Add Plane-2 leaf scaffold (pipeline graph, nodes, HITL, transports, CI) 2026-06-17 15:16:12 -04:00
test_operator_cli.py fix(agent-team): harden listener respawn/close, broaden handle_event guard, channel_ref partial-unique (#29) 2026-06-22 16:14:22 -04:00
test_planner.py Add Plane-2 leaf scaffold (pipeline graph, nodes, HITL, transports, CI) 2026-06-17 15:16:12 -04:00
test_recovery.py Add Plane-2 leaf scaffold (pipeline graph, nodes, HITL, transports, CI) 2026-06-17 15:16:12 -04:00
test_responder.py Add Plane-2 leaf scaffold (pipeline graph, nodes, HITL, transports, CI) 2026-06-17 15:16:12 -04:00
test_resume_worker.py Add Plane-2 leaf scaffold (pipeline graph, nodes, HITL, transports, CI) 2026-06-17 15:16:12 -04:00
test_review_loop.py feat(agent-team): bind planner/review/builder/verifier nodes to their models 2026-06-18 12:56:42 -04:00
test_review_loop_llm.py feat(agent-team): bind planner/review/builder/verifier nodes to their models 2026-06-18 12:56:42 -04:00
test_run_team.py feat(agent-team): Plane-1 fixer — finding→patch→CI draft-PR (dep-bumps, opt-in/inert) (#21) 2026-06-18 16:17:33 -04:00
test_schema.py feat(agent-team): durable GitHub-issue intake de-dup + intake hardening (#32) 2026-06-22 17:20:43 -04:00
test_slack_adapter.py Add Plane-2 leaf scaffold (pipeline graph, nodes, HITL, transports, CI) 2026-06-17 15:16:12 -04:00
test_slack_listener.py fix(agent-team): harden listener respawn/close, broaden handle_event guard, channel_ref partial-unique (#29) 2026-06-22 16:14:22 -04:00
test_slack_live.py test(agent-team): make slack_live token test independent of slack_sdk presence 2026-06-18 13:00:34 -04:00
test_state_store.py Plane 2 foundation: interfaces, SQLite schemas, state-store, billing seam 2026-06-17 15:16:12 -04:00
test_task_model.py Plane 2 foundation: interfaces, SQLite schemas, state-store, billing seam 2026-06-17 15:16:12 -04:00
test_transport_base.py Plane 2 foundation: interfaces, SQLite schemas, state-store, billing seam 2026-06-17 15:16:12 -04:00
test_verifier.py Add Plane-2 leaf scaffold (pipeline graph, nodes, HITL, transports, CI) 2026-06-17 15:16:12 -04:00
test_verifier_llm.py feat(agent-team): bind planner/review/builder/verifier nodes to their models 2026-06-18 12:56:42 -04:00