From 537e83975b6127abe7c6111043265a6600b9df71 Mon Sep 17 00:00:00 2001 From: Adam Moussa Date: Tue, 16 Jun 2026 15:00:00 -0400 Subject: [PATCH] Rewrite nightly sweep as two-tier clean-clone auto-discovery Replace the opt-in sweep-targets allowlist with zero-wiring discovery: enumerate org repos via the GitHub REST API (curl + read-only GH_TOKEN, no gh dependency) and mirror each as a shallow clean clone (git clone --depth=1, default branch from the API) into ~/repo-mirrors. Scanning server-side clones keeps local .env secrets out of scope. Tier 1 runs deterministic scanners over every repo nightly ($0 Claude); tier 2 runs the agentic pass over a budget-bounded round-robin rotation with a persistent cycle pointer, so the draw on the shared Max limits stays bounded and coverage never goes silently incomplete (COVERAGE ALARM if the rotation falls behind). Skip = committed marker or central list (marker-skips logged). Redact secrets from Slack; reports mode 600. Raise the systemd timeout to 6h for the longer two-tier run. --- security-review/nightly_sweep.sh | 410 ++++++++++++++++++ security-review/sweep-targets.txt | 15 + .../systemd/sea-haven-secrev.service | 49 +++ .../systemd/sea-haven-secrev.timer | 20 + 4 files changed, 494 insertions(+) create mode 100755 security-review/nightly_sweep.sh create mode 100644 security-review/sweep-targets.txt create mode 100644 security-review/systemd/sea-haven-secrev.service create mode 100644 security-review/systemd/sea-haven-secrev.timer diff --git a/security-review/nightly_sweep.sh b/security-review/nightly_sweep.sh new file mode 100755 index 0000000..999ee3d --- /dev/null +++ b/security-review/nightly_sweep.sh @@ -0,0 +1,410 @@ +#!/usr/bin/env bash +# nightly_sweep.sh — Sea Haven Path B nightly security sweep (R720 / sh-secrev VM). +# +# TWO-TIER, CLEAN-CLONE AUTO-DISCOVERY (no per-repo wiring): +# Discovery: enumerate ALL Sea-Haven-Industries org repos via the GitHub REST API +# (curl + a read-only fine-grained PAT in GH_TOKEN — no gh CLI dependency), then +# mirror each into ~/repo-mirrors as a shallow clean clone (git clone --depth=1, +# default branch from the API). Scanning server-side clones (not developer working +# trees) structurally avoids surfacing local gitignored .env secrets. +# TIER 1 (every repo, every night, $0 Claude): review.sh --scanners-only over every +# mirror — complete deterministic baseline coverage. +# TIER 2 (bounded agentic): the expensive run_headless.py detector+verifier pass runs +# over a deterministic ROUND-ROBIN rotation that fits TOTAL_BUDGET_USD, with a +# persistent cycle pointer so every repo gets a deep pass within MAX_CYCLE_NIGHTS. +# This bounds the draw on the SHARED Max subscription limits (see memory +# reference-claude-subscription-billing): a clean night never scans all repos +# agentically. +# +# Anti-complacency: the canary testbed is ALWAYS scanned agentically first (block + +# recall floor). Reporting is Slack ALARM-ONLY (a clean night posts NOTHING — see +# memory feedback_cloudwatch_alarms). Secret-shaped values are redacted from the Slack +# string; on-disk reports are written mode 600. +# +# Skip a repo: a .security-review-skip file committed at its root, OR an entry in the +# central skip list ($CENTRAL_SKIP_FILE). Repos skipped via their OWN committed marker +# are LOGGED in the report (auditable — a sensitive repo cannot silently self-exclude). +# +# Contract notes: +# - run_headless.py REQUIRES CLAUDE_CODE_OAUTH_TOKEN and pops ANTHROPIC_API_KEY. +# Source ~/secrev.env before invoking (the systemd unit does this via EnvironmentFile). +# - review.sh re-derives the block decision (exit 1 = BLOCK). This script makes NO +# block decision itself; it only reports. +# +# Config (env, all optional except auth): +# GH_TOKEN read-only fine-grained PAT (Contents: read) — REQUIRED for discovery +# GH_ORG org to enumerate (default: Sea-Haven-Industries) +# MIRROR_DIR clean-clone mirror root (default: ~/repo-mirrors) +# CENTRAL_SKIP_FILE one repo name per line, # comments (default: ~/.secrev-skip.txt) +# TARGETS space-separated paths to scan INSTEAD of discovery (manual override) +# TESTBED canary corpus dir (default: ~/security-review-testbed) +# CANARY_FLOOR min confirmed crit+high the canary MUST surface (default: 10) +# TOTAL_BUDGET_USD hard agentic spend ceiling across the night (default: 20) +# PER_TARGET_BUDGET_USD passed to run_headless --total-budget-usd (default: 12) +# MAX_CYCLE_NIGHTS alarm if the agentic rotation hasn't covered every repo in this many nights (default: 4) +# MAX_AGENTIC_PER_NIGHT cap on repos given the deep agentic pass per night, for wall-clock bounding +# (default: 0 = unlimited, bounded only by TOTAL_BUDGET_USD) +# REPORT_ROOT base dir for logs+JSON (default: ~/sweep-reports) +# SLACK_WEBHOOK_URL incoming-webhook URL; if unset, alarms are logged only +# ENABLE_XMODEL_HOOK 1 to run the cross-family critical tiebreak (default: 0) +# ORCHESTRATOR_DIR orchestrator repo root (default: ~/orchestrator) +# VENV_PY python in the SDK venv (default: ~/orchestrator/.venv/bin/python) +# +# Exit: 0 = sweep completed (whether or not it alarmed); 2 = setup/usage error. +set -euo pipefail +export PATH="$HOME/.local/bin:/opt/homebrew/bin:/usr/local/bin:$PATH" + +log() { echo "[nightly_sweep] $*" >&2; } +die() { echo "[nightly_sweep] FATAL: $*" >&2; exit 2; } + +# --- Config + defaults -------------------------------------------------------- +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ORCHESTRATOR_DIR="${ORCHESTRATOR_DIR:-$HOME/orchestrator}" +VENV_PY="${VENV_PY:-$ORCHESTRATOR_DIR/.venv/bin/python}" +RUN_HEADLESS="$HERE/run_headless.py" +REVIEW_SH="$HERE/review.sh" +GH_ORG="${GH_ORG:-Sea-Haven-Industries}" +MIRROR_DIR="${MIRROR_DIR:-$HOME/repo-mirrors}" +CENTRAL_SKIP_FILE="${CENTRAL_SKIP_FILE:-$HOME/.secrev-skip.txt}" +TESTBED="${TESTBED:-$HOME/security-review-testbed}" +CANARY_FLOOR="${CANARY_FLOOR:-10}" +TOTAL_BUDGET_USD="${TOTAL_BUDGET_USD:-20}" +PER_TARGET_BUDGET_USD="${PER_TARGET_BUDGET_USD:-12}" +MAX_CYCLE_NIGHTS="${MAX_CYCLE_NIGHTS:-4}" +MAX_AGENTIC_PER_NIGHT="${MAX_AGENTIC_PER_NIGHT:-0}" +REPORT_ROOT="${REPORT_ROOT:-$HOME/sweep-reports}" +ENABLE_XMODEL_HOOK="${ENABLE_XMODEL_HOOK:-0}" + +command -v jq >/dev/null || die "jq is required" +command -v curl >/dev/null || die "curl is required for org discovery" +command -v git >/dev/null || die "git is required" +[ -x "$VENV_PY" ] || die "venv python not found/executable: $VENV_PY" +[ -f "$RUN_HEADLESS" ] || die "run_headless.py not found: $RUN_HEADLESS" +[ -x "$REVIEW_SH" ] || die "review.sh not found/executable: $REVIEW_SH" +[ -n "${CLAUDE_CODE_OAUTH_TOKEN:-}" ] || die "CLAUDE_CODE_OAUTH_TOKEN not set (source ~/secrev.env)" + +UTC_DATE="$(date -u +%Y-%m-%d)" +UTC_STAMP="$(date -u +%Y-%m-%dT%H:%M:%SZ)" +REPORT_DIR="$REPORT_ROOT/$UTC_DATE" +mkdir -p "$REPORT_DIR"; chmod 700 "$REPORT_ROOT" "$REPORT_DIR" 2>/dev/null || true +ROTATION_STATE="$REPORT_ROOT/.rotation-state.json" +SWEEP_LOG="$REPORT_DIR/sweep.log" +exec > >(tee -a "$SWEEP_LOG") 2>&1 +umask 077 # on-disk reports/logs are not world-readable + +log "=== nightly sweep $UTC_STAMP (two-tier auto-discovery) ===" +log "org=$GH_ORG mirror=$MIRROR_DIR report=$REPORT_DIR total-budget=\$$TOTAL_BUDGET_USD canary-floor=$CANARY_FLOOR" + +# --- Aggregate state ---------------------------------------------------------- +TOTAL_SPEND="0"; BUDGET_HIT=0 +declare -a ALARM_LINES=(); declare -a XMODEL_LINES=(); declare -a MARKER_SKIPS=() +add_spend() { TOTAL_SPEND="$(jq -n --argjson a "$TOTAL_SPEND" --argjson b "${1:-0}" '$a + $b')"; } +over_budget() { jq -n --argjson s "$TOTAL_SPEND" --argjson c "$TOTAL_BUDGET_USD" -e '$c > 0 and $s >= $c' >/dev/null; } + +# --- Secret redaction for the Slack string (defense-in-depth; reports stay on the VM) -- +redact() { + sed -E \ + -e 's/AKIA[0-9A-Z]{16}/AKIA****REDACTED****/g' \ + -e 's/gh[pousr]_[A-Za-z0-9]{20,}/gh*_****REDACTED****/g' \ + -e 's/(xox[baprs]-)[A-Za-z0-9-]{10,}/\1****REDACTED****/g' \ + -e 's/[A-Za-z0-9/+]{40,}/****REDACTED-HIENTROPY****/g' +} + +# --- Discovery: enumerate non-archived org repos via the REST API ------------- +# Emits "nameclone_urldefault_branch" per repo. Returns non-zero on failure. +discover_repos() { + [ -n "${GH_TOKEN:-}" ] || { log "GH_TOKEN unset — cannot enumerate org"; return 1; } + local page=1 got body + while :; do + body="$(curl -fsS \ + -H "Authorization: Bearer $GH_TOKEN" \ + -H "Accept: application/vnd.github+json" \ + -H "X-GitHub-Api-Version: 2022-11-28" \ + "https://api.github.com/orgs/$GH_ORG/repos?per_page=100&type=all&page=$page" 2>>"$REPORT_DIR/discover.log")" || return 1 + echo "$body" | jq -e 'type=="array"' >/dev/null 2>&1 || return 1 + got="$(echo "$body" | jq -r '[.[] | select(.archived==false)] | .[] | [.name, .clone_url, .default_branch] | @tsv')" + [ -n "$got" ] && echo "$got" + [ "$(echo "$body" | jq 'length')" -lt 100 ] && break + page=$((page+1)) + done + return 0 +} + +# --- Mirror one repo as a shallow clean clone ------------------------------------------- +# The token is NEVER persisted to .git/config: the fetch path passes the auth URL inline +# (transient, command-args only), and the clone path scrubs origin immediately after. So a +# failed fetch cannot leave GH_TOKEN at rest on disk. (Residual: the token is briefly visible +# in process args to a local `ps`; acceptable on this single-user unattended box.) +mirror_repo() { # name clone_url default_branch -> 0 ok / 1 fail + local name="$1" url="$2" branch="$3" dir="$MIRROR_DIR/$name" + local auth_url="https://x-access-token:${GH_TOKEN}@${url#https://}" + if [ -d "$dir/.git" ]; then + git -C "$dir" fetch --depth=1 "$auth_url" "$branch" >/dev/null 2>&1 || return 1 + git -C "$dir" reset --hard FETCH_HEAD >/dev/null 2>&1 || return 1 + git -C "$dir" clean -fdq >/dev/null 2>&1 || true + else + git clone --depth=1 --branch "$branch" "$auth_url" "$dir" >/dev/null 2>&1 || return 1 + git -C "$dir" remote set-url origin "$url" >/dev/null 2>&1 || true # clone wrote auth URL → scrub it + fi + return 0 +} + +# --- Skip resolution: "" = scan, else reason ("marker"|"central") -------------- +declare -a CENTRAL_SKIP=() +if [ -f "$CENTRAL_SKIP_FILE" ]; then + while IFS= read -r line; do line="${line%%#*}"; line="$(echo "$line" | xargs || true)" + [ -n "$line" ] && CENTRAL_SKIP+=( "$line" ); done < "$CENTRAL_SKIP_FILE" +fi +skip_reason() { # name dir + local name="$1" dir="$2" + [ -f "$dir/.security-review-skip" ] && { echo "marker"; return; } + for s in ${CENTRAL_SKIP[@]+"${CENTRAL_SKIP[@]}"}; do [ "$s" = "$name" ] && { echo "central"; return; }; done + echo "" +} + +# --- xmodel cross-family critical tiebreak (GUARDED, never fails the sweep) ---- +xmodel_check_criticals() { + local label="$1" result_json="$2" + [ "$ENABLE_XMODEL_HOOK" = "1" ] || return 0 + [ -n "${OPENAI_API_KEY:-}" ] || { log " xmodel hook: OPENAI_API_KEY unset — skipping"; return 0; } + "$VENV_PY" -c 'import langchain_openai' >/dev/null 2>&1 || { log " xmodel hook: deps missing — skipping"; return 0; } + local crits n; crits="$(jq -c '[.findings[]? | select(.status=="confirmed" and .severity=="critical")]' "$result_json" 2>/dev/null || echo '[]')" + n="$(echo "$crits" | jq 'length')"; [ "${n:-0}" -gt 0 ] || return 0 + log " xmodel hook: re-checking $n confirmed critical(s) for $label" + local i=0 + while [ "$i" -lt "$n" ]; do + local summary; summary="$(echo "$crits" | jq -r --argjson i "$i" '.[$i] | "\(.cwe // "n/a") \(.file):\(.line // 0) — \(.title // .id) :: \(.data_flow // "")"')" + local verdict + if verdict="$(cd "$ORCHESTRATOR_DIR" && "$VENV_PY" run.py "Independently assess whether this is a real exploitable vulnerability (yes/no) and why: $summary" 2>>"$REPORT_DIR/xmodel.log")"; then + if echo "$verdict" | grep -qiE '(^|[^a-z])no([^a-z]|$)|not (a |an )?(real |exploitable )?vuln'; then + XMODEL_LINES+=( "DISAGREEMENT on $label critical: $summary (cross_reviewer says NOT a vuln)" ) + fi + else log " xmodel hook: run.py failed for a critical (logged) — continuing"; fi + i=$((i+1)) + done +} + +# --- TIER 1: deterministic scanners over a target dir ------------------------- +# Sets T1_BLOCK/T1_CRIT/T1_HIGH. review.sh exit 0 pass / 1 BLOCK / 2 setup. +T1_BLOCK=0; T1_CRIT=0; T1_HIGH=0 +scan_scanners() { # target slug + local target="$1" slug="$2" result_json="$REPORT_DIR/${slug}.scanners.json" + T1_BLOCK=0; T1_CRIT=0; T1_HIGH=0 + local sup=() + [ -f "$target/.security-review/suppressions.json" ] && sup=(--suppressions "$target/.security-review/suppressions.json") + set +e + "$REVIEW_SH" --scanners-only ${sup[@]+"${sup[@]}"} --json-out "$result_json" "$target" >"$REPORT_DIR/${slug}.scanners.log" 2>&1 + local rc=$? + set -e + [ "$rc" -eq 2 ] && { ALARM_LINES+=( "*$slug*: review.sh scanner setup error. See \`$REPORT_DIR/${slug}.scanners.log\`." ); return; } + T1_CRIT="$(jq -r '(.summary.confirmed_critical // 0)' "$result_json" 2>/dev/null || echo 0)" + T1_HIGH="$(jq -r '(.summary.confirmed_high // 0)' "$result_json" 2>/dev/null || echo 0)" + [ "$rc" -eq 1 ] && T1_BLOCK=1 +} + +# --- TIER 2: agentic run_headless + full review.sh over a target dir ---------- +# Sets LAST_BLOCK/LAST_CRIT/LAST_HIGH/LAST_REASON/LAST_RESULT_JSON/LAST_ERRORS. +LAST_BLOCK=0; LAST_CRIT=0; LAST_HIGH=0; LAST_REASON=""; LAST_RESULT_JSON=""; LAST_ERRORS=0 +scan_agentic() { # target slug + local target="$1" slug="$2" + LAST_BLOCK=0; LAST_CRIT=0; LAST_HIGH=0; LAST_REASON=""; LAST_RESULT_JSON=""; LAST_ERRORS=0 + [ -d "$target" ] || { ALARM_LINES+=( "Target *$slug* ($target) missing — could not scan." ); LAST_ERRORS=1; return; } + local agent_json="$REPORT_DIR/${slug}.agent.json" result_json="$REPORT_DIR/${slug}.result.json" runner_log="$REPORT_DIR/${slug}.runner.log" + LAST_RESULT_JSON="$result_json" + log " [$slug] run_headless.py (per-target budget \$$PER_TARGET_BUDGET_USD)" + if ! "$VENV_PY" "$RUN_HEADLESS" "$target" --out "$agent_json" --total-budget-usd "$PER_TARGET_BUDGET_USD" >>"$runner_log" 2>&1; then + ALARM_LINES+=( "*$slug*: run_headless.py failed (setup error). See \`$runner_log\`." ); LAST_ERRORS=1; return + fi + [ -f "$agent_json" ] || { ALARM_LINES+=( "*$slug*: run_headless produced no JSON." ); LAST_ERRORS=1; return; } + local spend errs; spend="$(jq -r '(._meta.spend_usd // 0)' "$agent_json")"; errs="$(jq -r '(._meta.errors // []) | length' "$agent_json")" + add_spend "$spend"; LAST_ERRORS="$errs" + log " [$slug] spend \$$spend, runner errors $errs, total \$$TOTAL_SPEND" + [ "${errs:-0}" -gt 0 ] && ALARM_LINES+=( "*$slug*: run_headless reported $errs error(s): $(jq -r '(._meta.errors // []) | join("; ")' "$agent_json")" ) + local sup=() + [ -f "$target/.security-review/suppressions.json" ] && sup=(--suppressions "$target/.security-review/suppressions.json") + set +e + "$REVIEW_SH" --agent-findings "$agent_json" ${sup[@]+"${sup[@]}"} --json-out "$result_json" "$target" >"$REPORT_DIR/${slug}.review.log" 2>&1 + local rc=$? + set -e + [ "$rc" -eq 2 ] && { ALARM_LINES+=( "*$slug*: review.sh setup error. See \`$REPORT_DIR/${slug}.review.log\`." ); LAST_ERRORS=$((LAST_ERRORS+1)); return; } + LAST_CRIT="$(jq -r '(.summary.confirmed_critical // 0)' "$result_json" 2>/dev/null || echo 0)" + LAST_HIGH="$(jq -r '(.summary.confirmed_high // 0)' "$result_json" 2>/dev/null || echo 0)" + if [ "$rc" -eq 1 ]; then LAST_BLOCK=1; LAST_REASON="confirmed crit=$LAST_CRIT high=$LAST_HIGH"; log " [$slug] BLOCK ($LAST_REASON)" + else log " [$slug] PASS (crit=$LAST_CRIT high=$LAST_HIGH)"; fi +} + +# ============================== 1) CANARY ==================================== +CANARY_OK=1 +if [ -d "$TESTBED" ]; then + log "--- canary (anti-complacency): $TESTBED ---" + scan_agentic "$TESTBED" "canary" + CANARY_CONFIRMED=0 + if [ -n "$LAST_RESULT_JSON" ] && [ -f "$LAST_RESULT_JSON" ]; then + CANARY_CONFIRMED="$(jq -r '[.findings[]? | select(.status=="confirmed" and (.severity|IN("critical","high")))] | length' "$LAST_RESULT_JSON" 2>/dev/null || echo 0)" + fi + log "canary: block=$LAST_BLOCK confirmed(crit+high)=$CANARY_CONFIRMED (floor=$CANARY_FLOOR)" + if [ "$LAST_BLOCK" -ne 1 ]; then + CANARY_OK=0; ALARM_LINES+=( "*COMPLACENCY ALARM*: canary testbed did NOT block. Result: \`$LAST_RESULT_JSON\`" ) + elif [ "${CANARY_CONFIRMED:-0}" -lt "$CANARY_FLOOR" ]; then + CANARY_OK=0; ALARM_LINES+=( "*COMPLACENCY ALARM*: canary recall $CANARY_CONFIRMED < floor $CANARY_FLOOR. Result: \`$LAST_RESULT_JSON\`" ) + fi + xmodel_check_criticals "canary" "$LAST_RESULT_JSON" +else + CANARY_OK=0; ALARM_LINES+=( "*COMPLACENCY ALARM*: canary testbed missing at $TESTBED." ) +fi + +# ============================== 2) DISCOVER + MIRROR ========================= +declare -a REPO_NAMES=() # scan order (discovery order) +declare -A REPO_DIR=() +if [ -n "${TARGETS:-}" ]; then + # Manual override: scan explicit paths, no discovery/cloning. + # shellcheck disable=SC2206 + arr=( $TARGETS ) + for p in "${arr[@]}"; do + p="${p/#\~/$HOME}"; nm="$(basename "$p")" + REPO_NAMES+=( "$nm" ); REPO_DIR["$nm"]="$p" + done + log "manual TARGETS override: ${REPO_NAMES[*]}" +else + mkdir -p "$MIRROR_DIR" + DISCOVERED="$REPORT_DIR/discovered.tsv" + if discover_repos > "$DISCOVERED" 2>>"$REPORT_DIR/discover.log" && [ -s "$DISCOVERED" ]; then + NREPO="$(wc -l < "$DISCOVERED" | tr -d ' ')" + log "discovered $NREPO non-archived repo(s) in $GH_ORG" + while IFS=$'\t' read -r name url branch; do + [ -n "$name" ] || continue + if mirror_repo "$name" "$url" "$branch"; then + REPO_NAMES+=( "$name" ); REPO_DIR["$name"]="$MIRROR_DIR/$name" + else + log " mirror FAILED: $name"; ALARM_LINES+=( "*$name*: clone/pull failed — not scanned this night. See \`$REPORT_DIR/discover.log\`." ) + fi + done < "$DISCOVERED" + log "mirrored ${#REPO_NAMES[@]} repo(s) into $MIRROR_DIR" + else + ALARM_LINES+=( "*DISCOVERY ALARM*: org enumeration failed (GH_TOKEN missing/invalid or API error). Falling back to existing mirrors; coverage may be stale. See \`$REPORT_DIR/discover.log\`." ) + log "discovery failed — falling back to existing mirrors in $MIRROR_DIR" + if [ -d "$MIRROR_DIR" ]; then + for d in "$MIRROR_DIR"/*/; do [ -d "$d/.git" ] || continue; nm="$(basename "$d")"; REPO_NAMES+=( "$nm" ); REPO_DIR["$nm"]="${d%/}"; done + fi + fi +fi + +# Resolve skips up front (so both tiers honor them and marker-skips are auditable). +declare -a SCANNABLE=() +for nm in ${REPO_NAMES[@]+"${REPO_NAMES[@]}"}; do + reason="$(skip_reason "$nm" "${REPO_DIR[$nm]}")" + if [ "$reason" = "marker" ]; then MARKER_SKIPS+=( "$nm" ); log " skip $nm (repo-committed .security-review-skip)" + elif [ "$reason" = "central" ]; then log " skip $nm (central skip list)" + else SCANNABLE+=( "$nm" ); fi +done +if [ "${#MARKER_SKIPS[@]}" -gt 0 ]; then + ALARM_LINES+=( "*self-excluded repos* (committed .security-review-skip, FYI/audit): ${MARKER_SKIPS[*]}" ) +fi +log "scannable repos: ${#SCANNABLE[@]} (skipped: $(( ${#REPO_NAMES[@]} - ${#SCANNABLE[@]} )))" + +# ============================== 3) TIER 1: scanners over ALL ================== +declare -a BLOCKED_T1=() +for nm in ${SCANNABLE[@]+"${SCANNABLE[@]}"}; do + scan_scanners "${REPO_DIR[$nm]}" "scan-$nm" + if [ "$T1_BLOCK" -eq 1 ]; then + BLOCKED_T1+=( "$nm" ) + ALARM_LINES+=( "*$nm* TIER1/scanners BLOCK: crit=$T1_CRIT high=$T1_HIGH. Result: \`$REPORT_DIR/scan-$nm.scanners.json\`" ) + fi +done +log "tier1 complete: ${#SCANNABLE[@]} scanned, ${#BLOCKED_T1[@]} blocked" + +# ============================== 4) TIER 2: agentic rotation =================== +# Persistent cycle state: {cycle_start, scanned:[names]}. Reset the cycle once every +# scannable repo has had a deep pass; alarm if a cycle runs longer than MAX_CYCLE_NIGHTS. +[ -f "$ROTATION_STATE" ] || echo "{\"cycle_start\":\"$UTC_DATE\",\"scanned\":[]}" > "$ROTATION_STATE" +SCANNED_JSON="$(jq -c '.scanned // []' "$ROTATION_STATE" 2>/dev/null || echo '[]')" +CYCLE_START="$(jq -r '.cycle_start // empty' "$ROTATION_STATE" 2>/dev/null || echo "$UTC_DATE")" +[ -n "$CYCLE_START" ] || CYCLE_START="$UTC_DATE" +if [ "${#SCANNABLE[@]}" -gt 0 ]; then + SCANNABLE_JSON="$(printf '%s\n' "${SCANNABLE[@]}" | jq -R . | jq -cs .)" +else + SCANNABLE_JSON="[]" +fi +# If every scannable repo is already in scanned[], the cycle is complete -> start fresh. +if jq -e -n --argjson sc "$SCANNED_JSON" --argjson all "$SCANNABLE_JSON" '($all - $sc) | length == 0' >/dev/null 2>&1 \ + && [ "$(echo "$SCANNABLE_JSON" | jq 'length')" -gt 0 ]; then + log "agentic rotation: cycle complete ($CYCLE_START) — starting a new cycle" + SCANNED_JSON="[]"; CYCLE_START="$UTC_DATE" +fi +# This night's agentic candidates = scannable repos not yet scanned this cycle, discovery order. +PENDING_JSON="$(jq -c -n --argjson all "$SCANNABLE_JSON" --argjson sc "$SCANNED_JSON" '$all - $sc')" +declare -a BLOCKED_T2=(); AGENTIC_DONE=0 +if over_budget; then + BUDGET_HIT=1; ALARM_LINES+=( "*BUDGET ALARM*: ceiling \$$TOTAL_BUDGET_USD hit after canary (\$$TOTAL_SPEND). No agentic rotation this night." ) +else + while read -r nm; do + [ -n "$nm" ] || continue + if over_budget; then BUDGET_HIT=1; log "budget ceiling hit (\$$TOTAL_SPEND) — pausing rotation"; break; fi + if [ "$MAX_AGENTIC_PER_NIGHT" -gt 0 ] && [ "$AGENTIC_DONE" -ge "$MAX_AGENTIC_PER_NIGHT" ]; then + log "per-night agentic cap ($MAX_AGENTIC_PER_NIGHT) reached — pausing rotation"; break; fi + log "--- agentic: $nm ---" + scan_agentic "${REPO_DIR[$nm]}" "scan-$nm" + SCANNED_JSON="$(echo "$SCANNED_JSON" | jq -c --arg n "$nm" '. + [$n] | unique')" + AGENTIC_DONE=$((AGENTIC_DONE+1)) + if [ "$LAST_BLOCK" -eq 1 ]; then + BLOCKED_T2+=( "$nm" ) + ALARM_LINES+=( "*$nm* TIER2/agentic BLOCK: $LAST_REASON. Result: \`$LAST_RESULT_JSON\`" ) + xmodel_check_criticals "$nm" "$LAST_RESULT_JSON" + fi + done < <(echo "$PENDING_JSON" | jq -r '.[]') +fi +# Persist rotation state. +jq -n --arg cs "$CYCLE_START" --argjson sc "$SCANNED_JSON" '{cycle_start:$cs, scanned:$sc}' > "$ROTATION_STATE" +# Coverage accounting + lag alarm. +REMAINING="$(jq -n --argjson all "$SCANNABLE_JSON" --argjson sc "$SCANNED_JSON" '($all - $sc) | length')" +to_epoch() { date -u -d "$1" +%s 2>/dev/null || date -u -j -f '%Y-%m-%d' "$1" +%s 2>/dev/null || echo 0; } +CYCLE_AGE=$(( ( $(to_epoch "$UTC_DATE") - $(to_epoch "$CYCLE_START") ) / 86400 )) +log "agentic rotation: scanned $AGENTIC_DONE this night, $REMAINING still pending in cycle (started $CYCLE_START, age ${CYCLE_AGE}d)" +if [ "$REMAINING" -gt 0 ] && [ "$CYCLE_AGE" -ge "$MAX_CYCLE_NIGHTS" ]; then + ALARM_LINES+=( "*COVERAGE ALARM*: agentic rotation behind — $REMAINING repo(s) not deep-scanned in ${CYCLE_AGE}d (cycle since $CYCLE_START, max $MAX_CYCLE_NIGHTS). Raise budget or check for failures." ) +fi + +# Fold xmodel disagreements into the alarm set. +for x in ${XMODEL_LINES[@]+"${XMODEL_LINES[@]}"}; do ALARM_LINES+=( "$x" ); done + +# ============================== 5) ALARM-ONLY REPORT ========================= +ALARM=0 +[ "${#BLOCKED_T1[@]}" -gt 0 ] && ALARM=1 +[ "${#BLOCKED_T2[@]}" -gt 0 ] && ALARM=1 +[ "$CANARY_OK" -ne 1 ] && ALARM=1 +[ "$BUDGET_HIT" -eq 1 ] && ALARM=1 +[ "${#ALARM_LINES[@]}" -gt 0 ] && ALARM=1 + +SUMMARY_LINE="sweep $UTC_STAMP: scannable=${#SCANNABLE[@]} tier1_blocked=${#BLOCKED_T1[@]} tier2_scanned=$AGENTIC_DONE tier2_blocked=${#BLOCKED_T2[@]} canary_ok=$CANARY_OK spend=\$$TOTAL_SPEND/\$$TOTAL_BUDGET_USD alarm=$ALARM report=$REPORT_DIR" +echo "$SUMMARY_LINE" + +if [ "$ALARM" -ne 1 ]; then + log "clean night — no alarm conditions. Posting NOTHING to Slack (ALARM-only policy)." + exit 0 +fi + +ALARM_BODY="$(printf '%s\n' ${ALARM_LINES[@]+"${ALARM_LINES[@]}"} | sed 's/^/• /')" +SLACK_TEXT=":rotating_light: *Sea Haven nightly security sweep — ALARM* ($UTC_STAMP) +$ALARM_BODY + +Coverage: tier1 scanners ${#SCANNABLE[@]} repos · tier2 agentic $AGENTIC_DONE this night ($REMAINING pending) · canary_ok=$CANARY_OK +Spend: \$$TOTAL_SPEND (ceiling \$$TOTAL_BUDGET_USD) +Reports + JSON: \`$REPORT_DIR\` (on sh-secrev VM)" +SLACK_TEXT="$(echo "$SLACK_TEXT" | redact)" + +log "ALARM conditions present — composing Slack post" +echo "$SLACK_TEXT" >&2 + +if [ -n "${SLACK_WEBHOOK_URL:-}" ] && command -v curl >/dev/null; then + PAYLOAD="$(jq -n --arg t "$SLACK_TEXT" '{text:$t}')" + if curl -fsS -X POST -H 'Content-Type: application/json' --data "$PAYLOAD" "$SLACK_WEBHOOK_URL" >/dev/null 2>>"$REPORT_DIR/slack.log"; then + log "Slack alarm posted." + else + log "Slack POST FAILED — see $REPORT_DIR/slack.log. Alarm text is in $SWEEP_LOG." + fi +else + log "SLACK_WEBHOOK_URL unset (or curl missing) — alarm logged to $SWEEP_LOG only." +fi + +# An alarm is a reportable condition, not a script crash. Exit 0 so systemd shows success. +exit 0 diff --git a/security-review/sweep-targets.txt b/security-review/sweep-targets.txt new file mode 100644 index 0000000..9be820e --- /dev/null +++ b/security-review/sweep-targets.txt @@ -0,0 +1,15 @@ +# sweep-targets.txt — one repo path per line for the nightly Path B sweep. +# Lines starting with '#' and blank lines are ignored. ~ is expanded. +# Override at runtime with the TARGETS env var (space-separated paths). +# +# NOTE: the testbed canary corpus is ALWAYS scanned by nightly_sweep.sh as the +# anti-complacency check; do NOT list it here (it is handled separately). +# +# TODO (Phase 5): add the first real hardened repo here once it is cloned on the +# VM (candidates: payments-dashboard / proposal-system / procurement-ingest). +# +# Deliberately EMPTY by default = canary-only nights. Do NOT scan ~/orchestrator: +# it holds ~/orchestrator/.env with live provider API keys, which the agentic +# detector could surface into sweep reports / Slack. Only add repos with no +# plaintext secrets (or scrub/exclude secret files first). +# ~/orchestrator diff --git a/security-review/systemd/sea-haven-secrev.service b/security-review/systemd/sea-haven-secrev.service new file mode 100644 index 0000000..3e40fb0 --- /dev/null +++ b/security-review/systemd/sea-haven-secrev.service @@ -0,0 +1,49 @@ +# sea-haven-secrev.service — Path B nightly security sweep (sh-secrev VM, user adam). +# +# Install (on the VM, as root): +# sudo cp sea-haven-secrev.service /etc/systemd/system/ +# sudo cp sea-haven-secrev.timer /etc/systemd/system/ +# sudo systemctl daemon-reload +# sudo systemctl enable --now sea-haven-secrev.timer # timer drives the run; do NOT enable the .service +# systemctl list-timers sea-haven-secrev.timer # confirm next run +# sudo systemctl start sea-haven-secrev.service # optional: run once now to smoke-test +# journalctl -u sea-haven-secrev.service -e # logs (also under ~/sweep-reports//) +# +# Secrets come from the EnvironmentFiles (the leading '-' = optional, no failure if absent): +# ~/secrev.env -> CLAUDE_CODE_OAUTH_TOKEN (required by run_headless.py), +# GH_TOKEN (read-only fine-grained PAT — REQUIRED for org auto-discovery), +# SLACK_WEBHOOK_URL +# ~/orchestrator/.env -> OPENAI_API_KEY etc. (only needed if ENABLE_XMODEL_HOOK=1) +# +# GH_TOKEN must be a fine-grained PAT scoped to the Sea-Haven-Industries org with READ-ONLY +# Contents (and Metadata) permission — nothing else. It enumerates repos and clones them into +# ~/repo-mirrors. Never give this unattended box a write-capable token. + +[Unit] +Description=Sea Haven Path B nightly security sweep +After=network-online.target +Wants=network-online.target + +[Service] +Type=oneshot +User=adam +WorkingDirectory=/home/adam/orchestrator +EnvironmentFile=-/home/adam/secrev.env +EnvironmentFile=-/home/adam/orchestrator/.env +# Tune ceilings/targets here without editing the script (uncomment to override defaults): +# Environment=TOTAL_BUDGET_USD=20 +# Environment=PER_TARGET_BUDGET_USD=12 +# Environment=CANARY_FLOOR=10 +# Environment=MAX_CYCLE_NIGHTS=4 +# Environment=MAX_AGENTIC_PER_NIGHT=0 +# Environment=GH_ORG=Sea-Haven-Industries +# Environment=MIRROR_DIR=/home/adam/repo-mirrors +# Environment=ENABLE_XMODEL_HOOK=0 +ExecStart=/home/adam/orchestrator/security-review/nightly_sweep.sh +# Two-tier sweep (scanners over every repo + a budget-bounded agentic rotation) runs for hours; +# 6h ceiling bounds a hang without killing a healthy long night. Spend is capped by TOTAL_BUDGET_USD. +TimeoutStartSec=21600 +Nice=10 + +[Install] +WantedBy=multi-user.target diff --git a/security-review/systemd/sea-haven-secrev.timer b/security-review/systemd/sea-haven-secrev.timer new file mode 100644 index 0000000..b377585 --- /dev/null +++ b/security-review/systemd/sea-haven-secrev.timer @@ -0,0 +1,20 @@ +# sea-haven-secrev.timer — fires the nightly sweep at ~02:00 local, user adam. +# +# Install: see the header of sea-haven-secrev.service. In short: +# sudo systemctl enable --now sea-haven-secrev.timer +# systemctl list-timers sea-haven-secrev.timer +# +# Persistent=true → if the VM was off at 02:00, the sweep runs at next boot. +# RandomizedDelaySec spreads load off an exact-minute spike. + +[Unit] +Description=Run the Sea Haven Path B security sweep nightly (~02:00) + +[Timer] +OnCalendar=*-*-* 02:00:00 +Persistent=true +RandomizedDelaySec=600 +Unit=sea-haven-secrev.service + +[Install] +WantedBy=timers.target