First full VM run: canary recall 18 with the expanded Node/.NET corpus, and a real
agentic repo cost ~$3.50 (vs the $1.4 testbed). Raise CANARY_FLOOR 10->14 (catches a
language-blindness recall collapse to ~9 while keeping margin under 18) and MAX_CYCLE_NIGHTS
4->6 (at ~4-6 agentic repos/night a full 22-repo rotation takes ~4-5 nights; 4 would
false-fire the coverage alarm). Both stay env-overridable and re-tunable as data accrues.
scan_scanners communicates results via globals (T1_BLOCK/T1_CRIT/T1_HIGH); its last
statement was a bare [ $rc -eq 1 ] && T1_BLOCK=1. On a PASS (rc=0) that test is false,
so the function returned non-zero and set -e killed the whole sweep at the first passing
repo in tier 1 (right after the unbound-variable fix let it get that far). Add an explicit
return 0. Audited the rest of the tier1/tier2/summary path; scan_agentic and the others
already end on a zero-status command.
scan_scanners referenced ${slug} inside the SAME local statement that defines it
(local target=... slug=$2 result_json=...${slug}...). bash expands the local's
arguments before the builtin assigns them, so under set -u ${slug} is unbound and
the sweep died right after the canary, before tier 1 ever ran. Split the local so
slug exists first. Fix the same latent self-reference in mirror_repo (dir=...$name),
which only worked by accident because the discovery loop left a global $name.
Per the machine-level-suppressions policy, the .env.example FP suppression now lives at
~/.config/sea-haven/security-review/orchestrator/suppressions.json (resolved first by the
hooks), not a committed .security-review/suppressions.json. Gate still passes via the
machine-level file.
The root README covered only the router and omitted the security-review/ subsystem
entirely (Path A skill, review.sh gate, Path B nightly sweep, global hooks). Add a
Security Review section summarizing both paths and linking the subsystem README +
DEPLOY-R720 runbook.
The live global pre-push hook was hand-edited to resolve suppressions from a
machine-level file (${SH_SECURITY_SUPPRESSIONS_DIR:-~/.config/sea-haven/security-review}/
<repo-basename>/suppressions.json) kept out of repo history, falling back to a
repo-local .security-review/suppressions.json. The repo-sourced hooks lacked it, so
install-hooks.sh --global would overwrite the live hook and lose the feature.
Port the prefer-machine/fallback-repo-local block into hooks/pre-push, align
hooks/pre-commit to the same (else a machine-suppressed finding passes at push but
blocks at commit), and document the path + SH_SECURITY_SUPPRESSIONS_DIR override +
basename-collision caveat in the README.
run_headless.py is the Path B engine the nightly sweep calls: the 6 fresh-context
detectors + proof-or-kill verifier from /sh-security-review, run unattended via the
Claude Agent SDK on subscription OAuth (pops ANTHROPIC_API_KEY so the API key can't
silently win). Read-only tools, hermetic, per-call + total budget caps, fails toward
over-reporting. Emits the finding schema that review.sh --agent-findings consumes.
Rewrite README + DEPLOY-R720 for the clean-clone mirror model, the read-only GH_TOKEN
PAT recipe, the global hook installer, and the no-CI-by-design decision.
gitleaks git-mode scans committed history, so the .env.example placeholder flags from
history and would block the orchestrator's own pre-push gate. Suppress it with a written
justification. Also gitignore stray .adf_final*.json left by an unrelated tool.
Replace the opt-in sweep-targets allowlist with zero-wiring discovery: enumerate org
repos via the GitHub REST API (curl + read-only GH_TOKEN, no gh dependency) and mirror
each as a shallow clean clone (git clone --depth=1, default branch from the API) into
~/repo-mirrors. Scanning server-side clones keeps local .env secrets out of scope.
Tier 1 runs deterministic scanners over every repo nightly ($0 Claude); tier 2 runs the
agentic pass over a budget-bounded round-robin rotation with a persistent cycle pointer,
so the draw on the shared Max limits stays bounded and coverage never goes silently
incomplete (COVERAGE ALARM if the rotation falls behind). Skip = committed marker or
central list (marker-skips logged). Redact secrets from Slack; reports mode 600. Raise
the systemd timeout to 6h for the longer two-tier run.
The global pre-push hook, the /sh-security-review prompt, and finding.schema.json
previously lived only in ~/.config/git and ~/.claude (untracked) — unreproducible.
Source them here: add hooks/pre-push, rewrite install-hooks.sh with a --global mode
(lays down both hooks, sets core.hooksPath, links skill+schema into ~/.claude) and a
per-repo mode. Align pre-commit with pre-push (honor skip marker + suppressions). Add
semgrep p/javascript so the scanners cover the org's Node/.NET repos.
CI was swapped for the global git hooks + nightly VM sweep (solo dev), so
the parked ci/*.yml and CI-BACKSTOP-NOTES.md were dead weight — a defective
workflow in-tree is a foot-gun. Recover from history if the team grows.
Trigger-agnostic pure-code gate that merges deterministic-scanner findings
(semgrep/gitleaks/checkov/cfn-lint/pip-audit) with agent findings from
/sh-security-review, dedups, applies justification-required suppressions, and
makes the block decision (exit 1 on confirmed critical/high). Phase 2 of the
Sea Haven security-review agent; Path B (CI/headless) wiring lands in Phase 3.
- Add 6 retrieval-augmented routing tests (3 live retrieval, 3 off-topic
fake memories) to unblock Phase 5
- Defer Composio tool loading and graph construction to first use so
expired or missing keys don't crash imports
- Atomic cache write in retriever via temp file (open item #2)
- Log rotation in weekly_summary.py, pruning JSONL >90 days (open item #3)
Captures end-of-session state after Phases 1-2-4 merge: current
architecture, shipped commits, three documented open items
(retrieval-augmented routing test before Phase 5; non-atomic cache
write; log rotation), unscheduled-but-ready items (weekly digest + log
prune), telemetry collection window, roadmap status, and a pickup
prompt template for the next session.
Dated filename so future handoffs can coexist without overwriting.
CI's `pytest --collect-only` was exiting 5 (no tests collected) because
the test module used `pytest.skip(..., allow_module_level=True)` at
import time — skipped modules never enter the collection phase.
Switch to a per-test `skipif` marker driven by ANTHROPIC_API_KEY +
COMPOSIO_API_KEY presence, and lazy-import `graph.router_node` inside
the test body so module import works without COMPOSIO_API_KEY.
Result: 21 tests collect in both environments; live tests run only
when both keys are present. Verified locally with env -i.
claude-sonnet-4-20250514 reaches EOL 2026-06-15. Bumping ahead of the
deadline while the centralized models.py constant makes it a one-line
change. Smoke-tested via run.py --route-only — no deprecation warning,
router behavior unchanged.
Closes the no-CI gap surfaced in the post-merge retrospective. The org
reusable workflows live under Sea-Haven-Industries and assume SAM/CDK
projects — orchestrator is a personal CLI tool with neither, so this is
a standalone workflow on the same conventions (actions/checkout@v6,
Python 3.12, ruff).
- lint job: ruff check + ruff format --check.
- test-collect job: installs requirements + runs `pytest --collect-only`.
Catches import errors and golden-set test discovery regressions without
needing live ANTHROPIC/COMPOSIO secrets — full pytest stays a local
pre-push responsibility.
Also adds .DS_Store to the local .gitignore (also covered by the user
global gitignore, but belt-and-suspenders).
Follow-up to the Phases 1-2-4 modernization PR.
- models.py: extend retry coverage to Gemini transient errors
(google.api_core: ResourceExhausted, InternalServerError,
ServiceUnavailable, DeadlineExceeded, GatewayTimeout). Before this,
a 429 or 5xx from Google would crash the scanner route. Retriable
count went from 6 to 11.
- retriever.py: batch cache-miss embeddings into a single
embed_documents() call instead of one embed_query() per memory.
Cold rebuild of 87 memories went from ~30s to ~10s; one HTTPS
round-trip instead of 87.
- telemetry.py + scripts/weekly_summary.py: use UTC date for log
filenames so they align with the UTC `timestamp` field inside each
record. Eliminates the off-by-one near local midnight and matches
Sea Haven's UTC-for-logs convention.
- scripts/weekly_summary.py: include failed runs in the total cost
estimate (they consumed tokens too) and surface a `(incl. $X on
failed runs)` breakdown so outages are visible in the digest.
Validated: ruff clean; weekly_summary.py still prints; retriever cold
rebuild + cache hit both verified end-to-end.
Observability slim layer. Every full run appends one JSON line to
~/.claude/logs/orchestrator/YYYY-MM-DD.jsonl with timestamp, sha256-prefix
task hash (raw task is never logged), retrieved memory names, router
choice, runtime, tokens in/out, success/error. risk_class and confidence
fields are reserved nulls for Phase 5.
- telemetry.py: log_run(), build_record(), task_hash(), token-usage
extraction from AIMessage.usage_metadata. log_run swallows all
exceptions — telemetry never kills a run.
- run.py: wraps app.invoke in try/except with a monotonic-clock window;
logs on both success and failure. --route-only path is left unlogged
(no agent work, doesn't represent a "run").
- scripts/weekly_summary.py: scans the last 7 days of JSONL and prints
a markdown digest (routes, unknown rate, cross-review rate, success
rate, total spend, mean tokens/route). Schedule via /schedule and
pipe stdout to Slack from the scheduler.
Cost rates per route are rough Sonnet/Haiku/GPT/Gemini/DeepSeek defaults
suitable for spotting runaway prompts, not finance. Router tokens for
structured-output calls aren't captured (they don't surface through the
message trail); agent tokens are the dominant component anyway.
Validated: golden-set 21/21 still passing; full-run smoke writes
expected fields; weekly_summary.py prints clean markdown from a 2-run
log.
Plugs the orchestrator into Adam's existing memory store at
~/.claude/projects/-Users-adammoussa-Documents-repositories/memory/. Every
run starts with a top-3 retrieval pass that is then surfaced in the CLI
output and injected as system context into the router and downstream agent.
- retriever.py: load *.md memories (skipping the MEMORY.md index), embed
with text-embedding-3-small, cache to .cache/embeddings.json keyed on
file mtime. Cosine similarity, top-k=3 default. Reads only — never
writes back to the memory store.
- state.py: add `retrieved: list[dict]` to OrchestratorState; relax to
total=False to match LangGraph's partial-update semantics.
- graph.py: new retriever_node wired as START -> retriever -> router.
router_node and connector_node now inject retrieved memories into their
SystemMessage. Retrieval failures are caught and the run continues with
empty memory context (logged).
- agents.py: make_agent_node injects retrieved memories into each agent's
system prompt.
- run.py: prints `[retrieved: name1, name2, name3]` (or `[retrieved: none]`)
before route/result for both --route-only and full-run modes, so bad
retrieval is visible at a glance.
- .gitignore: add .cache/, .pytest_cache/, .ruff_cache/.
Validated: golden-set still 21/21 passing; smoke tests retrieve plausible
memories ("Send a Slack message to ops about the new exec-aide deploy" ->
project_exec_aide, feedback_exec_aide_vip_management, project_seahaven_slack_bot).
Phase 1 stabilization. Removes the four-copy prompt/agent-description drift
surface and the silent router fallback.
- models.py: hoist model IDs to module-level constants; add with_retries()
helper (2 retries on Anthropic+OpenAI transient errors via with_retry).
- agents.py: single AGENTS dict (model_fn, prompt, description) and a
make_agent_node() factory that collapses six near-identical node functions.
- graph.py: router prompt is generated from AGENTS; router_node uses
with_structured_output(RouteDecision) and returns an explicit "unknown"
route instead of the silent "researcher" fallback. New unknown_node wires
to END. All LLM invocations go through with_retries.
- state.py: add "unknown" to the route Literal.
- run.py: --route-only now imports router_node from graph.py, killing the
fourth prompt copy.
- tests/: pytest golden-set (20 labelled tasks + size guard). Skips cleanly
without ANTHROPIC_API_KEY or COMPOSIO_API_KEY. Validated 21/21 passing.
Multi-model agent routing with 7 agent nodes (Sonnet, Haiku,
GPT-4.1, Gemini, DeepSeek) and 15 pre-loaded Composio tools
for Slack, Notion, GitHub, and Google Drive integration.
