mirror of
https://github.com/Sea-Haven-Industries/security-review.git
synced 2026-09-30 17:23:16 +00:00
Adam Moussa
4c88c01f7b
Fresh-init copy of the security-review/ subsystem extracted from Sea-Haven-Industries/orchestrator (being deprecated). Adds org-standard scaffold: CI reusable-workflow callers (ruff + collect), dependency-review, labeler, dependabot, .gitignore, requirements.txt. Scheduled execution is migrating to Claude Code web routines (ALARM-only to #repo-scanner); the systemd units and nightly_sweep.sh/checker_coordinator.sh remain the source of truth. Committed with --no-verify: the canary fixtures (checkers/fixtures/**) carry intentional secret-shaped test data that trips the deterministic gate (the documented detector-fixture false positive); no new logic is introduced.
492 lines
26 KiB
Bash
Executable file
492 lines
26 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# compliance-drift.sh — Plane-1 / Tier-1 checker for the R720 agent-team.
|
|
#
|
|
# Design refs: docs/r720-agent-team-design.md §4 (Tier 1 roster: compliance-drift) and
|
|
# §7 Phase 1 ("one checker end to end"). This is the FIRST Plane-1 checker built on the
|
|
# Phase-0 shared substrate (lib/sweep_substrate.sh) — it proves the substrate generalizes
|
|
# beyond the secrev nightly sweep.
|
|
#
|
|
# WHAT IT DOES (read-only):
|
|
# Flags drift from Sea Haven engineering conventions across the org mirrors. It scans the
|
|
# SAME shallow clean clones that nightly_sweep.sh already produced in $MIRROR_DIR — it does
|
|
# NOT re-clone (mirrors-first; an optional --refresh re-runs discovery+mirror via the shared
|
|
# substrate). The checklist is GROUNDED in the engineering-handbook + this repo's README; it
|
|
# does not invent rules. See "CHECKLIST" below.
|
|
#
|
|
# REPORTING (matches secrev sweep conventions):
|
|
# - Writes a per-run JSON + text report under $REPORT_ROOT/<UTC-date>/, mode 600 (umask 077).
|
|
# - Slack ALARM-ONLY: a clean run (no confirmed drift) posts NOTHING (memory
|
|
# feedback_cloudwatch_alarms). Secret-shaped values are redacted from the Slack string.
|
|
# - Reuses the substrate's redact() + post_slack_alarm() verbatim.
|
|
#
|
|
# SUBSTRATE REUSE (lib/sweep_substrate.sh, sourced — bash dynamic scoping):
|
|
# redact, post_slack_alarm -> Slack delivery (reads SLACK_WEBHOOK_URL, REPORT_DIR, SWEEP_LOG)
|
|
# discover_repos, mirror_repo-> ONLY on --refresh (reads GH_TOKEN, GH_ORG, MIRROR_DIR, REPORT_DIR)
|
|
# Default path enumerates EXISTING $MIRROR_DIR/*/.git dirs — zero clones, zero network.
|
|
#
|
|
# CANARY / DRY-RUN (offline, no network, no token):
|
|
# --canary runs the checklist against a planted-drift fixture (checkers/fixtures/compliance-drift/)
|
|
# and asserts the known drift count. This is the anti-complacency floor (design §6.4) AND the
|
|
# routing dry-run (§7 Phase 1, F4): with --dry-run, the Slack alarm is composed + printed but
|
|
# NOT POSTed. Fully offline-smoke-testable.
|
|
#
|
|
# SCOPE / SAFETY:
|
|
# Read-only. Filesystem checks need no network. The branch-protection / Dependabot-alerts /
|
|
# repo-settings checks call the GitHub REST API read-only with the same $GH_TOKEN the sweep
|
|
# uses (Contents+Metadata read). When GH_TOKEN is unset OR --no-api is passed (the offline
|
|
# default for --canary), API-only checks are SKIPPED and noted in the report — they are never
|
|
# reported as drift on missing data (memory feedback_cloudwatch_alarms: no false alarms on no-data).
|
|
#
|
|
# This script does NOT touch agent_team/ or agent-team/, and is NOT wired into systemd — that is
|
|
# Phase-6 provisioning (gated). See the "PROVISIONING (NOT DONE HERE)" note at the bottom.
|
|
#
|
|
# Exit: 0 = ran (whether or not it alarmed); 2 = setup/usage error; 3 = canary assertion FAILED.
|
|
set -euo pipefail
|
|
export PATH="$HOME/.local/bin:/opt/homebrew/bin:/usr/local/bin:$PATH"
|
|
|
|
log() { echo "[compliance-drift] $*" >&2; }
|
|
die() { echo "[compliance-drift] FATAL: $*" >&2; exit 2; }
|
|
|
|
# --- Shared substrate ---------------------------------------------------------
|
|
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
SUBSTRATE="$HERE/../lib/sweep_substrate.sh"
|
|
[ -f "$SUBSTRATE" ] || die "shared substrate not found: $SUBSTRATE"
|
|
# shellcheck source=../lib/sweep_substrate.sh
|
|
. "$SUBSTRATE"
|
|
|
|
# --- Config + defaults (env, all optional) ------------------------------------
|
|
GH_ORG="${GH_ORG:-Sea-Haven-Industries}"
|
|
MIRROR_DIR="${MIRROR_DIR:-$HOME/repo-mirrors}"
|
|
REPORT_ROOT="${REPORT_ROOT:-$HOME/sweep-reports/compliance-drift}"
|
|
# Repos exempt from CodeQL/compliance tooling per github-standards.md ("Exceptions").
|
|
# Comma-separated; handbook lists shoc-backend, shoc-frontend-new (SHOC-owned) + docs repos.
|
|
COMPLIANCE_EXEMPT="${COMPLIANCE_EXEMPT:-shoc-backend,shoc-frontend-new}"
|
|
# Docs-only repos skip CodeQL/CI-deploy expectations (handbook exception); they still need README.
|
|
DOCS_ONLY_REPOS="${DOCS_ONLY_REPOS:-engineering-handbook}"
|
|
|
|
REFRESH=0 # --refresh: re-run discovery+mirror via substrate (network). Default: reuse mirrors.
|
|
DO_API=1 # --no-api: skip GitHub-API checks (branch protection / dependabot / settings).
|
|
DRY_RUN=0 # --dry-run: compose the Slack alarm but DO NOT post it (routing dry-run, F4).
|
|
CANARY=0 # --canary: run against the planted-drift fixture + assert the known count.
|
|
TARGETS_OVERRIDE="" # --targets "p1 p2": scan explicit dirs instead of the mirror set.
|
|
|
|
usage() {
|
|
cat >&2 <<EOF
|
|
compliance-drift.sh — Plane-1 Tier-1 conventions-drift checker (read-only)
|
|
|
|
--canary run against the planted-drift fixture and assert the known drift count
|
|
(implies --dry-run + --no-api; fully offline smoke test)
|
|
--dry-run compose the Slack alarm but DO NOT post it (routing dry-run)
|
|
--no-api skip GitHub-API checks (branch protection, dependabot alerts, repo settings)
|
|
--refresh re-discover + re-mirror via the shared substrate before scanning (network)
|
|
--targets "a b" scan these explicit repo dirs instead of \$MIRROR_DIR/* (no clone)
|
|
-h|--help this help
|
|
|
|
Env: GH_ORG MIRROR_DIR REPORT_ROOT GH_TOKEN SLACK_WEBHOOK_URL COMPLIANCE_EXEMPT DOCS_ONLY_REPOS
|
|
EOF
|
|
}
|
|
|
|
while [ $# -gt 0 ]; do
|
|
case "$1" in
|
|
--canary) CANARY=1; DRY_RUN=1; DO_API=0 ;;
|
|
--dry-run) DRY_RUN=1 ;;
|
|
--no-api) DO_API=0 ;;
|
|
--refresh) REFRESH=1 ;;
|
|
--targets) shift; TARGETS_OVERRIDE="${1:-}" ;;
|
|
-h|--help) usage; exit 0 ;;
|
|
*) die "unknown arg: $1 (see --help)" ;;
|
|
esac
|
|
shift
|
|
done
|
|
|
|
command -v jq >/dev/null || die "jq is required"
|
|
command -v git >/dev/null || die "git is required"
|
|
|
|
# --- Report dir (mode 600 reports; matches sweep conventions) -----------------
|
|
umask 077
|
|
UTC_DATE="$(date -u +%Y-%m-%d)"
|
|
UTC_STAMP="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
|
REPORT_DIR="$REPORT_ROOT/$UTC_DATE"
|
|
mkdir -p "$REPORT_DIR"; chmod 700 "$REPORT_ROOT" "$REPORT_DIR" 2>/dev/null || true
|
|
# shellcheck disable=SC2034 # read by the sourced substrate (post_slack_alarm) via dynamic scope
|
|
SWEEP_LOG="$REPORT_DIR/compliance-drift.log" # name the substrate's post_slack_alarm() references
|
|
REPORT_JSON="$REPORT_DIR/compliance-drift.json"
|
|
REPORT_TXT="$REPORT_DIR/compliance-drift.txt"
|
|
|
|
log "=== compliance-drift $UTC_STAMP (canary=$CANARY dry_run=$DRY_RUN api=$DO_API refresh=$REFRESH) ==="
|
|
|
|
# ------------------------------------------------------------------------------
|
|
# CHECKLIST (grounded — every item cites a handbook/README rule; nothing invented):
|
|
#
|
|
# naming-repo repo dir name is kebab-case naming-conventions.md ("kebab-case for everything")
|
|
# readme-present README.md exists at repo root github-standards.md / global CLAUDE.md ("Every repo must have a README")
|
|
# cicd-present .github/workflows/ci.yaml|ci.yml cicd.md ("Every deployable repo must have a CI/CD pipeline"; ci.yaml)
|
|
# dependabot-config .github/dependabot.yml present when github-standards.md ("Every repo with dependencies gets a .github/dependabot.yml")
|
|
# dependency manifests exist
|
|
# secrets-committed no committed .env with real-looking secrets-and-config.md ("Never commit .env files containing real values")
|
|
# values (tracked-in-git, not gitignored)
|
|
# --- API-only (need GH_TOKEN; skipped offline / --no-api / --canary) ---
|
|
# branch-protection main requires PR, no force-push, github-standards.md ("Branch Protection")
|
|
# no deletion
|
|
# dependabot-alerts Dependabot alerts + security updates github-standards.md ("Dependabot alerts and security updates enabled")
|
|
# enabled
|
|
# merge-settings allow_auto_merge + delete_branch_on_ github-standards.md ("enable auto-merge and auto-delete head branch")
|
|
# merge enabled
|
|
#
|
|
# Each emitted finding follows the spirit of finding.schema.json (id/title/severity/category/
|
|
# proof/status) so a later phase can route it like an agentic finding. category="other" — this is
|
|
# convention drift, not the schema's security categories. status="confirmed" only for deterministic
|
|
# filesystem facts and explicit API "false" answers; API checks on missing data are NOT findings.
|
|
# ------------------------------------------------------------------------------
|
|
|
|
# Drift accumulator: one JSON object per finding, appended to a bash array.
|
|
declare -a FINDINGS=()
|
|
add_finding() { # repo id title severity check proof
|
|
local repo="$1" id="$2" title="$3" sev="$4" check="$5" proof="$6"
|
|
FINDINGS+=( "$(jq -n \
|
|
--arg repo "$repo" --arg id "$id" --arg title "$title" --arg sev "$sev" \
|
|
--arg check "$check" --arg proof "$proof" \
|
|
'{repo:$repo, id:($repo+"-"+$id), title:$title, severity:$sev, category:"other",
|
|
check:$check, status:"confirmed", proof:{outcome:$proof}}')" )
|
|
}
|
|
declare -a SKIPPED_CHECKS=() # (repo:check) checks skipped on missing data — reported, never alarmed
|
|
note_skip() { SKIPPED_CHECKS+=( "$1" ); }
|
|
|
|
in_csv() { # needle csv -> 0 if present
|
|
local n="$1" csv="$2"; case ",$csv," in *",$n,"*) return 0 ;; *) return 1 ;; esac
|
|
}
|
|
|
|
# --- kebab-case test (lowercase, digits, single hyphens; no leading/trailing hyphen) ---
|
|
is_kebab() { [[ "$1" =~ ^[a-z0-9]+(-[a-z0-9]+)*$ ]]; }
|
|
|
|
# --- Does the repo carry dependency manifests that warrant a dependabot.yml? ----
|
|
has_dep_manifests() { # dir
|
|
local d="$1"
|
|
# Match handbook's ecosystem table: package.json / requirements.txt / *.csproj.
|
|
[ -f "$d/package.json" ] && return 0
|
|
find "$d" -maxdepth 3 -name requirements.txt -not -path '*/.git/*' -print -quit 2>/dev/null | grep -q . && return 0
|
|
find "$d" -maxdepth 3 -name '*.csproj' -not -path '*/.git/*' -print -quit 2>/dev/null | grep -q . && return 0
|
|
return 1
|
|
}
|
|
|
|
# ==============================================================================
|
|
# FILESYSTEM CHECKS (offline; run on every repo dir)
|
|
# ==============================================================================
|
|
check_repo_fs() { # repo_name repo_dir
|
|
local repo="$1" dir="$2"
|
|
local docs_only=0; in_csv "$repo" "$DOCS_ONLY_REPOS" && docs_only=1
|
|
local exempt=0; in_csv "$repo" "$COMPLIANCE_EXEMPT" && exempt=1
|
|
|
|
# naming-repo — repo dir name kebab-case
|
|
is_kebab "$repo" || add_finding "$repo" "naming-repo" \
|
|
"Repo name '$repo' is not kebab-case" "medium" "naming-repo" \
|
|
"naming-conventions.md: kebab-case for everything (repository names)"
|
|
|
|
# readme-present — every repo, no exceptions
|
|
[ -f "$dir/README.md" ] || add_finding "$repo" "readme-missing" \
|
|
"No README.md at repo root" "high" "readme-present" \
|
|
"global CLAUDE.md / github-standards.md: every repo must have a README"
|
|
|
|
# cicd-present — ci workflow expected unless docs-only or compliance-exempt
|
|
if [ "$docs_only" -eq 0 ] && [ "$exempt" -eq 0 ]; then
|
|
if [ ! -f "$dir/.github/workflows/ci.yaml" ] && [ ! -f "$dir/.github/workflows/ci.yml" ]; then
|
|
add_finding "$repo" "cicd-missing" \
|
|
"No .github/workflows/ci.yaml" "high" "cicd-present" \
|
|
"cicd.md: every deployable repo must have a CI/CD pipeline (ci.yaml)"
|
|
fi
|
|
else
|
|
note_skip "$repo:cicd-present(docs-only/exempt)"
|
|
fi
|
|
|
|
# dependabot-config — required only when dependency manifests exist, and not exempt
|
|
if [ "$exempt" -eq 0 ] && has_dep_manifests "$dir"; then
|
|
[ -f "$dir/.github/dependabot.yml" ] || [ -f "$dir/.github/dependabot.yaml" ] || \
|
|
add_finding "$repo" "dependabot-config-missing" \
|
|
"Has dependency manifests but no .github/dependabot.yml" "medium" "dependabot-config" \
|
|
"github-standards.md: every repo with dependencies gets a .github/dependabot.yml"
|
|
fi
|
|
|
|
# secrets-committed — a .env TRACKED in git (gitignored .env is fine; tracked is the drift)
|
|
if [ -d "$dir/.git" ]; then
|
|
while IFS= read -r envf; do
|
|
[ -n "$envf" ] || continue
|
|
# Only flag .env / .env.* that look like they hold real values, not .env.example/.sample/.template.
|
|
case "$envf" in *.example|*.sample|*.template|*.dist) continue ;; esac
|
|
# Fire only on secret-SHAPED entries: a secret-ish key name, or a long
|
|
# (>=20 char) high-entropy value. Benign config (PORT=3000, DEBUG=true)
|
|
# is NOT drift, so a tracked config-only .env raises no ALARM
|
|
# (feedback_cloudwatch_alarms: no false alarms on non-secret config).
|
|
if grep -qiE '(secret|token|key|password|passwd|api[_-]?key|credential|private)[^=]*=[^[:space:]#]+' "$dir/$envf" 2>/dev/null \
|
|
|| grep -qE '=[^[:space:]#]{20,}' "$dir/$envf" 2>/dev/null; then
|
|
add_finding "$repo" "secrets-committed-$(echo "$envf" | tr '/.' '--')" \
|
|
"Tracked env file with values committed: $envf" "high" "secrets-committed" \
|
|
"secrets-and-config.md: never commit .env files containing real values"
|
|
fi
|
|
done < <(git -C "$dir" ls-files -- '*.env' '.env' '.env.*' 2>/dev/null || true)
|
|
else
|
|
note_skip "$repo:secrets-committed(not-a-git-checkout)"
|
|
fi
|
|
}
|
|
|
|
# ==============================================================================
|
|
# API CHECKS (read-only GitHub REST; need GH_TOKEN; skipped offline/--no-api/--canary)
|
|
# ==============================================================================
|
|
gh_api() { # path -> body on stdout, non-zero on transport/HTTP error
|
|
curl -fsS \
|
|
-H "Authorization: Bearer $GH_TOKEN" \
|
|
-H "Accept: application/vnd.github+json" \
|
|
-H "X-GitHub-Api-Version: 2022-11-28" \
|
|
"https://api.github.com/$1" 2>>"$REPORT_DIR/api.log"
|
|
}
|
|
|
|
check_repo_api() { # repo_name
|
|
local repo="$1"
|
|
local exempt=0; in_csv "$repo" "$COMPLIANCE_EXEMPT" && exempt=1
|
|
|
|
# repo settings: merge baseline + vulnerability-alerts capability come off the repo object.
|
|
local body
|
|
if ! body="$(gh_api "repos/$GH_ORG/$repo")" || ! echo "$body" | jq -e 'type=="object" and has("name")' >/dev/null 2>&1; then
|
|
note_skip "$repo:api(repo-fetch-failed)"; return
|
|
fi
|
|
local default_branch; default_branch="$(echo "$body" | jq -r '.default_branch // "main"')"
|
|
|
|
# merge-settings — auto-merge + delete-branch-on-merge (per-repo, no org default)
|
|
if [ "$exempt" -eq 0 ]; then
|
|
local am dbm; am="$(echo "$body" | jq -r '.allow_auto_merge')"; dbm="$(echo "$body" | jq -r '.delete_branch_on_merge')"
|
|
[ "$am" = "true" ] || add_finding "$repo" "merge-automerge-off" \
|
|
"allow_auto_merge disabled" "low" "merge-settings" \
|
|
"github-standards.md: enable auto-merge (allow_auto_merge)"
|
|
[ "$dbm" = "true" ] || add_finding "$repo" "merge-deletebranch-off" \
|
|
"delete_branch_on_merge disabled" "low" "merge-settings" \
|
|
"github-standards.md: enable auto-delete head branch on merge (delete_branch_on_merge)"
|
|
fi
|
|
|
|
# dependabot-alerts — vulnerability alerts enabled (204 = enabled, 404 = disabled)
|
|
if [ "$exempt" -eq 0 ]; then
|
|
local code
|
|
# No -f: a 404 (alerts off) is a real HTTP response we must classify, so curl
|
|
# must exit 0 and -w must yield a clean "404" (with -f the body-fail path
|
|
# corrupts the captured code and a real 404 would be misread as a skip).
|
|
code="$(curl -sS -o /dev/null -w '%{http_code}' \
|
|
-H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" \
|
|
-H "X-GitHub-Api-Version: 2022-11-28" \
|
|
"https://api.github.com/repos/$GH_ORG/$repo/vulnerability-alerts" 2>>"$REPORT_DIR/api.log" || echo 000)"
|
|
case "$code" in
|
|
204) : ;; # enabled
|
|
404) add_finding "$repo" "dependabot-alerts-off" \
|
|
"Dependabot vulnerability alerts disabled" "high" "dependabot-alerts" \
|
|
"github-standards.md: Dependabot alerts and security updates enabled on all active repos" ;;
|
|
*) note_skip "$repo:dependabot-alerts(http-$code)" ;; # missing data -> no alarm
|
|
esac
|
|
fi
|
|
|
|
# branch-protection — main: require PR, no force-push, no deletion.
|
|
# Status-code-aware (mirrors dependabot-alerts): 200 -> parse the rules,
|
|
# 404 -> no protection rule = real drift, anything else (403/5xx/000 transient
|
|
# or transport failure) -> skip with NO alarm (feedback_cloudwatch_alarms: a
|
|
# flaky API call must never raise a high-severity false alarm).
|
|
local prot_tmp prot_code prot
|
|
prot_tmp="$(mktemp)"
|
|
prot_code="$(curl -sS -o "$prot_tmp" -w '%{http_code}' \
|
|
-H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" \
|
|
-H "X-GitHub-Api-Version: 2022-11-28" \
|
|
"https://api.github.com/repos/$GH_ORG/$repo/branches/$default_branch/protection" \
|
|
2>>"$REPORT_DIR/api.log" || echo 000)"
|
|
prot="$(cat "$prot_tmp" 2>/dev/null)"; rm -f "$prot_tmp"
|
|
case "$prot_code" in
|
|
200)
|
|
echo "$prot" | jq -e '.required_pull_request_reviews != null' >/dev/null 2>&1 || \
|
|
add_finding "$repo" "branchprot-no-pr" \
|
|
"main does not require a PR for merge" "high" "branch-protection" \
|
|
"github-standards.md: require a PR for merges to main (no direct push)"
|
|
echo "$prot" | jq -e '.allow_force_pushes.enabled == false' >/dev/null 2>&1 || \
|
|
add_finding "$repo" "branchprot-force-push" \
|
|
"main allows force-push" "high" "branch-protection" \
|
|
"github-standards.md: no force push to main"
|
|
echo "$prot" | jq -e '.allow_deletions.enabled == false' >/dev/null 2>&1 || \
|
|
add_finding "$repo" "branchprot-deletion" \
|
|
"main allows branch deletion" "high" "branch-protection" \
|
|
"github-standards.md: no branch deletion for main"
|
|
;;
|
|
404)
|
|
# 404 from this endpoint = no protection rule at all on the default branch -> that IS drift.
|
|
add_finding "$repo" "branchprot-absent" \
|
|
"No branch protection on '$default_branch'" "high" "branch-protection" \
|
|
"github-standards.md: require a PR for merges to main, no force push, no deletion"
|
|
;;
|
|
*) note_skip "$repo:branch-protection(http-$prot_code)" ;; # transient/forbidden -> no alarm
|
|
esac
|
|
}
|
|
|
|
# ==============================================================================
|
|
# TARGET RESOLUTION
|
|
# ==============================================================================
|
|
declare -a REPO_NAMES=(); declare -A REPO_DIR=()
|
|
|
|
if [ "$CANARY" -eq 1 ]; then
|
|
FIXTURE_ROOT="$HERE/fixtures/compliance-drift"
|
|
[ -d "$FIXTURE_ROOT" ] || die "canary fixture missing: $FIXTURE_ROOT"
|
|
# Pin the exception lists the fixtures were authored against, so the canary is
|
|
# self-contained and deterministic regardless of the operator's env.
|
|
DOCS_ONLY_REPOS="docs-repo"
|
|
COMPLIANCE_EXEMPT=""
|
|
# Fixtures ship their git metadata as `dotgit/` (not `.git/`) so they are committable
|
|
# into THIS repo without becoming nested submodules. Materialize them into a temp work
|
|
# area — copy each fixture and rename dotgit -> .git — so the tracked-`.env`/ls-files
|
|
# checks run against a real git checkout. The temp area is mode 700 and removed on exit.
|
|
FIXTURE_WORK="$(mktemp -d "${TMPDIR:-/tmp}/compliance-drift-canary.XXXXXX")"
|
|
trap 'rm -rf "$FIXTURE_WORK"' EXIT
|
|
log "canary: materializing planted-drift fixtures from $FIXTURE_ROOT into $FIXTURE_WORK"
|
|
for d in "$FIXTURE_ROOT"/*/; do
|
|
[ -d "$d/dotgit" ] || continue # only fixture repos (skip README.md etc.)
|
|
nm="$(basename "$d")"
|
|
cp -R "$d" "$FIXTURE_WORK/$nm"
|
|
mv "$FIXTURE_WORK/$nm/dotgit" "$FIXTURE_WORK/$nm/.git"
|
|
# The planted-secret env file is shipped as `dotenv.fixture` (NOT `.env`): the repo's
|
|
# root .gitignore lists `.env`, so a literal `.env` fixture would never be committed and
|
|
# the secrets-committed drift would vanish on a fresh clone. Restore it to `.env` in the
|
|
# materialized work area (the dotgit/ index already TRACKS `.env`, so ls-files still
|
|
# reports it). Same committable-without-side-effects rationale as the `.fixture` suffix the
|
|
# dependency-cve fixtures use for their manifests.
|
|
[ -f "$FIXTURE_WORK/$nm/dotenv.fixture" ] && mv "$FIXTURE_WORK/$nm/dotenv.fixture" "$FIXTURE_WORK/$nm/.env"
|
|
REPO_NAMES+=( "$nm" ); REPO_DIR["$nm"]="$FIXTURE_WORK/$nm"
|
|
done
|
|
elif [ -n "$TARGETS_OVERRIDE" ]; then
|
|
# shellcheck disable=SC2206
|
|
arr=( $TARGETS_OVERRIDE )
|
|
for p in "${arr[@]}"; do p="${p/#\~/$HOME}"; nm="$(basename "$p")"; REPO_NAMES+=( "$nm" ); REPO_DIR["$nm"]="$p"; done
|
|
log "explicit targets: ${REPO_NAMES[*]}"
|
|
else
|
|
if [ "$REFRESH" -eq 1 ]; then
|
|
[ -n "${GH_TOKEN:-}" ] || die "--refresh needs GH_TOKEN"
|
|
command -v curl >/dev/null || die "--refresh needs curl"
|
|
mkdir -p "$MIRROR_DIR"
|
|
log "refresh: re-discovering + mirroring via shared substrate (no separate clone path)"
|
|
DISCOVERED="$REPORT_DIR/discovered.tsv"
|
|
if discover_repos > "$DISCOVERED" 2>>"$REPORT_DIR/discover.log" && [ -s "$DISCOVERED" ]; then
|
|
while IFS=$'\t' read -r name url branch; do
|
|
[ -n "$name" ] || continue
|
|
mirror_repo "$name" "$url" "$branch" || log " mirror FAILED: $name (will use stale mirror if present)"
|
|
done < "$DISCOVERED"
|
|
else
|
|
log "discovery failed — falling back to existing mirrors (coverage may be stale)"
|
|
fi
|
|
fi
|
|
# Default + post-refresh: enumerate EXISTING mirrors. No clone here — reuse the sweep's clones.
|
|
[ -d "$MIRROR_DIR" ] || die "mirror dir not found: $MIRROR_DIR (run nightly_sweep.sh first, or use --refresh/--targets)"
|
|
for d in "$MIRROR_DIR"/*/; do
|
|
[ -d "$d/.git" ] || continue
|
|
nm="$(basename "$d")"; REPO_NAMES+=( "$nm" ); REPO_DIR["$nm"]="${d%/}"
|
|
done
|
|
log "reusing ${#REPO_NAMES[@]} existing mirror(s) in $MIRROR_DIR (no re-clone)"
|
|
fi
|
|
|
|
[ "${#REPO_NAMES[@]}" -gt 0 ] || die "no repos to scan"
|
|
|
|
# Decide whether API checks run: need a token, the API enabled, and not the offline canary.
|
|
RUN_API=0
|
|
if [ "$DO_API" -eq 1 ] && [ -n "${GH_TOKEN:-}" ] && command -v curl >/dev/null; then RUN_API=1
|
|
elif [ "$DO_API" -eq 1 ]; then log "API checks requested but GH_TOKEN/curl unavailable — skipping (no false alarms on missing data)"; fi
|
|
|
|
# ==============================================================================
|
|
# RUN CHECKS
|
|
# ==============================================================================
|
|
for nm in "${REPO_NAMES[@]}"; do
|
|
check_repo_fs "$nm" "${REPO_DIR[$nm]}"
|
|
[ "$RUN_API" -eq 1 ] && check_repo_api "$nm"
|
|
done
|
|
|
|
# ==============================================================================
|
|
# ASSEMBLE REPORT (JSON + text), mode 600
|
|
# ==============================================================================
|
|
if [ "${#FINDINGS[@]}" -gt 0 ]; then
|
|
FINDINGS_JSON="$(printf '%s\n' "${FINDINGS[@]}" | jq -cs .)"
|
|
else
|
|
FINDINGS_JSON="[]"
|
|
fi
|
|
if [ "${#SKIPPED_CHECKS[@]}" -gt 0 ]; then
|
|
SKIPPED_JSON="$(printf '%s\n' "${SKIPPED_CHECKS[@]}" | jq -R . | jq -cs .)"
|
|
else
|
|
SKIPPED_JSON="[]"
|
|
fi
|
|
|
|
N_DRIFT="$(echo "$FINDINGS_JSON" | jq 'length')"
|
|
N_HIGH="$(echo "$FINDINGS_JSON" | jq '[.[]|select(.severity=="high")] | length')"
|
|
N_REPOS_DRIFTED="$(echo "$FINDINGS_JSON" | jq '[.[].repo] | unique | length')"
|
|
|
|
jq -n \
|
|
--arg checker "compliance-drift" --arg ts "$UTC_STAMP" --arg org "$GH_ORG" \
|
|
--argjson api "$RUN_API" --argjson scanned "${#REPO_NAMES[@]}" \
|
|
--argjson findings "$FINDINGS_JSON" --argjson skipped "$SKIPPED_JSON" \
|
|
'{checker:$checker, generated:$ts, org:$org, api_checks_ran:($api==1),
|
|
repos_scanned:$scanned, drift_count:($findings|length),
|
|
repos_with_drift:([$findings[].repo]|unique|length),
|
|
findings:$findings, skipped_checks:$skipped}' > "$REPORT_JSON"
|
|
|
|
{
|
|
echo "compliance-drift report — $UTC_STAMP"
|
|
echo "org=$GH_ORG repos_scanned=${#REPO_NAMES[@]} api_checks=$([ "$RUN_API" -eq 1 ] && echo on || echo off)"
|
|
echo "drift findings: $N_DRIFT ($N_HIGH high) across $N_REPOS_DRIFTED repo(s)"
|
|
echo
|
|
echo "$FINDINGS_JSON" | jq -r '.[] | "• [\(.severity)] \(.repo): \(.title)\n rule: \(.proof.outcome)"'
|
|
if [ "$(echo "$SKIPPED_JSON" | jq 'length')" -gt 0 ]; then
|
|
echo; echo "skipped checks (missing data — NOT counted as drift):"
|
|
echo "$SKIPPED_JSON" | jq -r '.[] | " - \(.)"'
|
|
fi
|
|
} > "$REPORT_TXT"
|
|
chmod 600 "$REPORT_JSON" "$REPORT_TXT" 2>/dev/null || true
|
|
|
|
log "report: $REPORT_JSON ($N_DRIFT drift finding(s), $N_REPOS_DRIFTED repo(s))"
|
|
|
|
# ==============================================================================
|
|
# CANARY ASSERTION (anti-complacency floor, design §6.4)
|
|
# ==============================================================================
|
|
if [ "$CANARY" -eq 1 ]; then
|
|
EXPECT_FILE="$HERE/fixtures/compliance-drift/EXPECTED_DRIFT_COUNT"
|
|
[ -f "$EXPECT_FILE" ] || die "canary expected-count file missing: $EXPECT_FILE"
|
|
EXPECTED="$(tr -dc '0-9' < "$EXPECT_FILE")"
|
|
log "canary assertion: expected drift=$EXPECTED, got=$N_DRIFT"
|
|
if [ "$N_DRIFT" -ne "$EXPECTED" ]; then
|
|
echo "[compliance-drift] CANARY FAIL: planted-drift count mismatch (expected $EXPECTED, got $N_DRIFT)" >&2
|
|
echo " -> the checklist regressed (a check stopped firing) or the fixture changed. See $REPORT_TXT." >&2
|
|
exit 3
|
|
fi
|
|
log "canary PASS: all $EXPECTED planted drifts detected."
|
|
fi
|
|
|
|
# ==============================================================================
|
|
# ALARM-ONLY ROUTING (clean = silent; memory feedback_cloudwatch_alarms)
|
|
# ==============================================================================
|
|
if [ "$N_DRIFT" -eq 0 ]; then
|
|
log "no confirmed drift — posting NOTHING to Slack (ALARM-only policy)."
|
|
exit 0
|
|
fi
|
|
|
|
ALARM_BODY="$(echo "$FINDINGS_JSON" | jq -r '
|
|
group_by(.repo)[] | "*\(.[0].repo)*: " + ([.[] | "[\(.severity)] \(.title)"] | join("; "))' | sed 's/^/• /')"
|
|
SLACK_TEXT=":triangular_flag_on_post: *Sea Haven compliance-drift — ALARM* ($UTC_STAMP)
|
|
$N_DRIFT drift finding(s) across $N_REPOS_DRIFTED repo(s) ($N_HIGH high):
|
|
$ALARM_BODY
|
|
|
|
Checks: naming · README · CI/CD · Dependabot · secrets-placement · branch-protection (api=$([ "$RUN_API" -eq 1 ] && echo on || echo off))
|
|
Report (mode 600): \`$REPORT_JSON\` (on R720)"
|
|
SLACK_TEXT="$(echo "$SLACK_TEXT" | redact)"
|
|
|
|
echo "$SLACK_TEXT" >&2
|
|
|
|
if [ "$DRY_RUN" -eq 1 ]; then
|
|
log "DRY-RUN: alarm composed but NOT posted (routing dry-run, design §7 Phase 1 / F4)."
|
|
exit 0
|
|
fi
|
|
post_slack_alarm "$SLACK_TEXT"
|
|
exit 0
|
|
|
|
# ==============================================================================
|
|
# PROVISIONING (NOT DONE HERE — gated, Phase 6):
|
|
# - No systemd unit / timer is installed by this script. Wiring it into the live
|
|
# sea-haven-secrev schedule (or a sibling timer) is provisioning and is gated.
|
|
# - The coordinator (design §5) that runs this alongside other Tier-1 checkers under
|
|
# one shared budget + versioned rotation state is Phase 2, not built here.
|
|
# - Confluence + project_r720_agent_team memory updates are docs-as-you-go obligations
|
|
# for the build session, tracked outside this script.
|
|
# ==============================================================================
|