mirror of
https://github.com/Sea-Haven-Industries/security-review.git
synced 2026-09-30 06:53:15 +00:00
a0bd8632b5
`${DIGEST:?}` aborts the script when DIGEST is unset/empty (i.e. when no
upstream checker findings exist). Replace with null-safe `${DIGEST:-}` so
the report-only mode exits cleanly on a clean week.
316 lines
16 KiB
Bash
Executable file
316 lines
16 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# plan-groomer.sh — Plane-1 planner for the R720 agent-team (REPORT-ONLY).
|
|
#
|
|
# Design refs: docs/r720-agent-team-design.md §4 (planner roster: plan-groomer —
|
|
# "Drafts a groomed weekly plan INTO the mode-600 report for now (D3); auto-write to
|
|
# Notion/Jira is a later toggle once trusted") and §7 Phase 4 ("planner + confluence-doc").
|
|
# This mirrors compliance-drift.sh / dependency-cve.sh conventions VERBATIM so the
|
|
# coordinator (§5) can drive it identically — BUT its output discipline is different:
|
|
# it is REPORT-ONLY, not ALARM-only.
|
|
#
|
|
# WHAT IT DOES (read-only):
|
|
# Aggregates the actionable items the OTHER Plane-1 checkers already produced into a
|
|
# single prioritized "groomed weekly plan". It does NOT re-scan repos or hit any network:
|
|
# it reads the LATEST per-checker JSON reports under $REPORT_ROOT_BASE/<checker>/<latest-date>/.
|
|
# Sources consumed (each optional — a missing checker is noted, never invented as work):
|
|
# compliance-drift/<date>/compliance-drift.json (.findings[])
|
|
# dependency-cve/<date>/dependency-cve.json (.findings[])
|
|
# doc-drift/<date>/doc-drift.json (.findings[], if Phase-3 doc-drift exists)
|
|
# confluence-doc/<date>/confluence-doc.json (.findings[], the Phase-4 sibling)
|
|
# Each finding is normalized to a plan item {repo, severity, source, title, action},
|
|
# DEDUPED (same repo+source+title collapses), grouped by severity then repo, and written
|
|
# into a prioritized plan in the mode-600 report (JSON + human text).
|
|
#
|
|
# REPORTING (D3 — REPORT-ONLY, the key difference from the ALARM-only checkers):
|
|
# - Writes a per-run JSON + text report under $REPORT_ROOT/<UTC-date>/, mode 600 (umask 077).
|
|
# - Slack: posts NOTHING by default. A groomed plan is a digest, not an alarm — auto-write
|
|
# to Notion/Jira (or a Slack digest) is a later toggle once signal quality is trusted (D3).
|
|
# There is intentionally NO post_slack_alarm() call in the default path; --notify is a
|
|
# future seam left inert here. A clean week (zero items) still writes an (empty) plan.
|
|
# - Reuses the substrate's redact() for the in-report digest string (defense-in-depth).
|
|
#
|
|
# SUBSTRATE REUSE (lib/sweep_substrate.sh, sourced — bash dynamic scoping):
|
|
# redact -> mask any secret-shaped value that leaked into an upstream report title.
|
|
# (discover_repos/mirror_repo/post_slack_alarm are intentionally NOT used: plan-groomer
|
|
# neither clones nor alarms — it only reads sibling reports and writes one mode-600 plan.)
|
|
#
|
|
# CANARY / DRY-RUN (offline, no network, no token):
|
|
# --canary points $REPORT_ROOT_BASE at a fixture set of sample checker reports
|
|
# (checkers/fixtures/plan-groomer/sample-reports/<checker>/<date>/<checker>.json) and asserts
|
|
# the groomed-plan ITEM COUNT equals EXPECTED_PLAN_ITEMS (anti-complacency floor, design §6.4).
|
|
# If aggregation/dedup regresses, the count drifts and the canary FAILS (exit 3). --canary
|
|
# implies --dry-run. Fully offline + deterministic. --dry-run also suppresses the (inert)
|
|
# --notify seam.
|
|
#
|
|
# SCOPE / SAFETY:
|
|
# Read-only. No network, no token, no clones, no agent_team/ writes, no systemd wiring —
|
|
# that is provisioning (gated). See "PROVISIONING (NOT DONE HERE)" at the bottom.
|
|
#
|
|
# Exit: 0 = ran (always, report-only); 2 = setup/usage error; 3 = canary assertion FAILED.
|
|
set -euo pipefail
|
|
export PATH="$HOME/.local/bin:/opt/homebrew/bin:/usr/local/bin:$PATH"
|
|
|
|
log() { echo "[plan-groomer] $*" >&2; }
|
|
die() { echo "[plan-groomer] FATAL: $*" >&2; exit 2; }
|
|
|
|
# --- Shared substrate ---------------------------------------------------------
|
|
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
SUBSTRATE="$HERE/../lib/sweep_substrate.sh"
|
|
[ -f "$SUBSTRATE" ] || die "shared substrate not found: $SUBSTRATE"
|
|
# shellcheck source=../lib/sweep_substrate.sh
|
|
. "$SUBSTRATE"
|
|
|
|
# --- Config + defaults (env, all optional) ------------------------------------
|
|
GH_ORG="${GH_ORG:-Sea-Haven-Industries}"
|
|
# Base under which each checker writes its own <checker>/<date>/ report tree (the parent of
|
|
# the per-checker REPORT_ROOTs the other checkers default to: $HOME/sweep-reports).
|
|
REPORT_ROOT_BASE="${REPORT_ROOT_BASE:-$HOME/sweep-reports}"
|
|
# plan-groomer's OWN report tree (separate from the checkers it reads).
|
|
REPORT_ROOT="${REPORT_ROOT:-$HOME/sweep-reports/plan-groomer}"
|
|
# Which sibling checkers to aggregate (space-separated; missing ones are noted, never invented).
|
|
SOURCE_CHECKERS="${SOURCE_CHECKERS:-compliance-drift dependency-cve doc-drift confluence-doc}"
|
|
|
|
DRY_RUN=0 # --dry-run: suppress the (inert) --notify seam (report still written).
|
|
CANARY=0 # --canary: read the fixture report set + assert the known plan-item count.
|
|
NOTIFY=0 # --notify: INERT future seam (post the digest somewhere). Off by default (D3).
|
|
TARGETS_OVERRIDE="" # --targets "a b": restrict the groomed plan to these repo names only.
|
|
|
|
usage() {
|
|
cat >&2 <<EOF
|
|
plan-groomer.sh — Plane-1 weekly planner (read-only, REPORT-ONLY per D3)
|
|
|
|
--canary aggregate the fixture report set and assert the known plan-item count
|
|
(implies --dry-run; fully offline smoke test)
|
|
--dry-run suppress the (inert) --notify seam (the report is still written)
|
|
--notify INERT future seam (D3): a later toggle to push the digest to Slack/Notion/Jira.
|
|
Does nothing in this phase; left here to mark the seam.
|
|
--targets "a b" restrict the groomed plan to these repo names only
|
|
-h|--help this help
|
|
|
|
Env: GH_ORG REPORT_ROOT_BASE REPORT_ROOT SOURCE_CHECKERS
|
|
EOF
|
|
}
|
|
|
|
while [ $# -gt 0 ]; do
|
|
case "$1" in
|
|
--canary) CANARY=1; DRY_RUN=1 ;;
|
|
--dry-run) DRY_RUN=1 ;;
|
|
--notify) NOTIFY=1 ;;
|
|
--targets) shift; TARGETS_OVERRIDE="${1:-}" ;;
|
|
-h|--help) usage; exit 0 ;;
|
|
*) die "unknown arg: $1 (see --help)" ;;
|
|
esac
|
|
shift
|
|
done
|
|
|
|
command -v jq >/dev/null || die "jq is required"
|
|
|
|
# --- Report dir (mode 600 reports; matches sweep conventions) -----------------
|
|
umask 077
|
|
UTC_DATE="$(date -u +%Y-%m-%d)"
|
|
UTC_STAMP="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
|
|
|
# Canary redirects the SOURCE base at the fixture report set; output stays in a temp area so
|
|
# the canary writes nothing under $HOME.
|
|
if [ "$CANARY" -eq 1 ]; then
|
|
FIXTURE_ROOT="$HERE/fixtures/plan-groomer"
|
|
[ -d "$FIXTURE_ROOT/sample-reports" ] || die "canary fixture missing: $FIXTURE_ROOT/sample-reports"
|
|
REPORT_ROOT_BASE="$FIXTURE_ROOT/sample-reports"
|
|
CANARY_OUT="$(mktemp -d "${TMPDIR:-/tmp}/plan-groomer-canary.XXXXXX")"
|
|
trap 'rm -rf "$CANARY_OUT"' EXIT
|
|
REPORT_ROOT="$CANARY_OUT/plan-groomer"
|
|
# Pin the source list the fixtures were authored against (deterministic regardless of env).
|
|
SOURCE_CHECKERS="compliance-drift dependency-cve doc-drift confluence-doc"
|
|
fi
|
|
|
|
REPORT_DIR="$REPORT_ROOT/$UTC_DATE"
|
|
mkdir -p "$REPORT_DIR"; chmod 700 "$REPORT_ROOT" "$REPORT_DIR" 2>/dev/null || true
|
|
# shellcheck disable=SC2034 # named for parity with the ALARM-only checkers' substrate contract
|
|
SWEEP_LOG="$REPORT_DIR/plan-groomer.log"
|
|
REPORT_JSON="$REPORT_DIR/plan-groomer.json"
|
|
REPORT_TXT="$REPORT_DIR/plan-groomer.txt"
|
|
|
|
log "=== plan-groomer $UTC_STAMP (canary=$CANARY dry_run=$DRY_RUN notify=$NOTIFY) ==="
|
|
|
|
# ------------------------------------------------------------------------------
|
|
# Severity ordering: the jq sort below maps severity->rank; no shell helper needed.
|
|
# ------------------------------------------------------------------------------
|
|
in_csv() { # needle space-list -> 0 if present
|
|
local n="$1" list="$2" t; for t in $list; do [ "$t" = "$n" ] && return 0; done; return 1
|
|
}
|
|
|
|
# --- Resolve the LATEST date dir for one checker under $REPORT_ROOT_BASE -------
|
|
latest_report_json() { # checker_name -> path to its latest <date>/<checker>.json, or "" if none
|
|
local checker="$1"
|
|
local base="$REPORT_ROOT_BASE/$checker" d name latest=""
|
|
[ -d "$base" ] || { echo ""; return 0; }
|
|
# Date dirs are YYYY-MM-DD; lexical sort == chronological. Glob the dirs, keep only
|
|
# YYYY-MM-DD names, sort newest-first, pick the newest that actually has the JSON.
|
|
for d in $(for p in "$base"/*/; do
|
|
[ -d "$p" ] || continue
|
|
name="$(basename "$p")"
|
|
[[ "$name" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]] && echo "$name"
|
|
done | sort -r); do
|
|
if [ -f "$base/$d/$checker.json" ]; then latest="$base/$d/$checker.json"; break; fi
|
|
done
|
|
echo "$latest"
|
|
}
|
|
|
|
# ==============================================================================
|
|
# AGGREGATE: pull findings from each sibling checker's latest report into plan items.
|
|
# ==============================================================================
|
|
declare -a PLAN_ITEMS=() # one normalized JSON plan-item per upstream finding
|
|
declare -a MISSING_SOURCES=() # checkers with no report found — noted, NEVER invented as work
|
|
note_missing() { MISSING_SOURCES+=( "$1" ); }
|
|
|
|
for checker in $SOURCE_CHECKERS; do
|
|
rj="$(latest_report_json "$checker")"
|
|
if [ -z "$rj" ]; then
|
|
note_missing "$checker(no-report-found)"
|
|
log " [$checker] no report under $REPORT_ROOT_BASE/$checker — skipped (not invented as work)"
|
|
continue
|
|
fi
|
|
if ! jq -e '.findings | type=="array"' "$rj" >/dev/null 2>&1; then
|
|
note_missing "$checker(report-unparseable)"
|
|
log " [$checker] report $rj has no .findings[] array — skipped"
|
|
continue
|
|
fi
|
|
cnt="$(jq '.findings | length' "$rj")"
|
|
log " [$checker] $rj -> $cnt finding(s)"
|
|
# Normalize each finding to a plan item. Title/severity/repo come straight off the finding
|
|
# schema the checkers emit (see finding.schema.json spirit). The "action" is a stable,
|
|
# source-derived hint (no fabrication — it just labels what kind of remediation this is).
|
|
while IFS= read -r item; do
|
|
[ -n "$item" ] && PLAN_ITEMS+=( "$item" )
|
|
done < <(jq -c --arg src "$checker" '
|
|
.findings[]
|
|
| { repo: (.repo // "unknown"),
|
|
severity: (.severity // "medium"),
|
|
source: $src,
|
|
title: (.title // .id // "untitled"),
|
|
action: ( if $src=="dependency-cve" then "upgrade vulnerable dependency"
|
|
elif $src=="compliance-drift" then "fix convention drift"
|
|
elif $src=="doc-drift" then "reconcile docs with code"
|
|
elif $src=="confluence-doc" then "close documentation gap"
|
|
else "review finding" end ) }
|
|
' "$rj")
|
|
done
|
|
|
|
# --- Optional repo-scope restriction (--targets) ------------------------------
|
|
if [ -n "$TARGETS_OVERRIDE" ] && [ "${#PLAN_ITEMS[@]}" -gt 0 ]; then
|
|
declare -a FILTERED=()
|
|
for item in "${PLAN_ITEMS[@]}"; do
|
|
r="$(echo "$item" | jq -r '.repo')"
|
|
in_csv "$r" "$TARGETS_OVERRIDE" && FILTERED+=( "$item" )
|
|
done
|
|
PLAN_ITEMS=( "${FILTERED[@]+"${FILTERED[@]}"}" )
|
|
log "targets filter '$TARGETS_OVERRIDE' -> ${#PLAN_ITEMS[@]} item(s)"
|
|
fi
|
|
|
|
# ==============================================================================
|
|
# DEDUP + PRIORITIZE: collapse identical (repo|source|title), then sort by severity desc,
|
|
# then repo, then source. Add a stable rank int so downstream consumers can re-sort.
|
|
# ==============================================================================
|
|
if [ "${#PLAN_ITEMS[@]}" -gt 0 ]; then
|
|
RAW_JSON="$(printf '%s\n' "${PLAN_ITEMS[@]}" | jq -cs .)"
|
|
else
|
|
RAW_JSON="[]"
|
|
fi
|
|
|
|
GROOMED_JSON="$(echo "$RAW_JSON" | jq -c '
|
|
# dedup on repo|source|title
|
|
( reduce .[] as $x ({}; .[($x.repo+"|"+$x.source+"|"+$x.title)] //= $x) ) | [ .[] ]
|
|
| map(. + { rank: ( {critical:4, high:3, medium:2, low:1}[.severity] // 0 ) })
|
|
| sort_by([ (-.rank), .repo, .source, .title ])
|
|
')"
|
|
|
|
N_ITEMS="$(echo "$GROOMED_JSON" | jq 'length')"
|
|
N_CRIT_HIGH="$(echo "$GROOMED_JSON" | jq '[.[]|select(.severity=="critical" or .severity=="high")]|length')"
|
|
N_REPOS="$(echo "$GROOMED_JSON" | jq '[.[].repo]|unique|length')"
|
|
|
|
if [ "${#MISSING_SOURCES[@]}" -gt 0 ]; then
|
|
MISSING_JSON="$(printf '%s\n' "${MISSING_SOURCES[@]}" | jq -R . | jq -cs .)"
|
|
else
|
|
MISSING_JSON="[]"
|
|
fi
|
|
|
|
# ==============================================================================
|
|
# ASSEMBLE REPORT (JSON + text), mode 600
|
|
# ==============================================================================
|
|
jq -n \
|
|
--arg planner "plan-groomer" --arg ts "$UTC_STAMP" --arg org "$GH_ORG" \
|
|
--arg sources "$SOURCE_CHECKERS" \
|
|
--argjson items "$GROOMED_JSON" --argjson missing "$MISSING_JSON" \
|
|
'{planner:$planner, generated:$ts, org:$org, mode:"report-only",
|
|
sources_considered:($sources|split(" ")),
|
|
plan_item_count:($items|length),
|
|
crit_high_count:([$items[]|select(.severity=="critical" or .severity=="high")]|length),
|
|
repos_in_plan:([$items[].repo]|unique|length),
|
|
plan:$items, missing_sources:$missing}' > "$REPORT_JSON"
|
|
|
|
{
|
|
echo "plan-groomer — groomed weekly plan — $UTC_STAMP"
|
|
echo "org=$GH_ORG sources=[$SOURCE_CHECKERS] mode=report-only (D3: no auto-write)"
|
|
echo "plan items: $N_ITEMS ($N_CRIT_HIGH crit/high) across $N_REPOS repo(s)"
|
|
echo
|
|
if [ "$N_ITEMS" -gt 0 ]; then
|
|
echo "PRIORITIZED PLAN (severity desc, then repo):"
|
|
echo "$GROOMED_JSON" | jq -r '.[] | "• [\(.severity)] \(.repo) — \(.title)\n action: \(.action) (source: \(.source))"'
|
|
else
|
|
echo "No actionable items aggregated this run (clean week, or no upstream reports)."
|
|
fi
|
|
if [ "$(echo "$MISSING_JSON" | jq 'length')" -gt 0 ]; then
|
|
echo; echo "sources with no report (NOT invented as work):"
|
|
echo "$MISSING_JSON" | jq -r '.[] | " - \(.)"'
|
|
fi
|
|
} > "$REPORT_TXT"
|
|
chmod 600 "$REPORT_JSON" "$REPORT_TXT" 2>/dev/null || true
|
|
|
|
# Defense-in-depth: the digest line that a future --notify seam would push is redacted now.
|
|
DIGEST="$(echo "$GROOMED_JSON" | jq -r '
|
|
group_by(.repo)[] | "*\(.[0].repo)*: " + ([.[] | "[\(.severity)] \(.title)"] | join("; "))' \
|
|
| sed 's/^/• /' | redact)"
|
|
|
|
log "report: $REPORT_JSON ($N_ITEMS plan item(s), $N_REPOS repo(s))"
|
|
|
|
# ==============================================================================
|
|
# CANARY ASSERTION (anti-complacency floor, design §6.4)
|
|
# ==============================================================================
|
|
if [ "$CANARY" -eq 1 ]; then
|
|
EXPECT_FILE="$HERE/fixtures/plan-groomer/EXPECTED_PLAN_ITEMS"
|
|
[ -f "$EXPECT_FILE" ] || die "canary expected-count file missing: $EXPECT_FILE"
|
|
EXPECTED="$(tr -dc '0-9' < "$EXPECT_FILE")"
|
|
log "canary assertion: expected plan items=$EXPECTED, got=$N_ITEMS"
|
|
if [ "$N_ITEMS" -ne "$EXPECTED" ]; then
|
|
echo "[plan-groomer] CANARY FAIL: groomed-plan item count mismatch (expected $EXPECTED, got $N_ITEMS)" >&2
|
|
echo " -> aggregation or dedup regressed, or the fixture changed. See $REPORT_TXT." >&2
|
|
exit 3
|
|
fi
|
|
log "canary PASS: groomed plan has all $EXPECTED expected item(s) (dedup intact)."
|
|
fi
|
|
|
|
# ==============================================================================
|
|
# REPORT-ONLY ROUTING (D3): the plan lives in the mode-600 report. Post NOTHING.
|
|
# ==============================================================================
|
|
if [ "$NOTIFY" -eq 1 ] && [ "$DRY_RUN" -eq 0 ]; then
|
|
# INERT future seam: when D3's "once trusted" toggle flips, this is where the digest would
|
|
# be pushed to Slack/Notion/Jira. It is intentionally a no-op in this phase — plan-groomer
|
|
# is REPORT-ONLY and must not auto-write. The composed digest is available in $DIGEST.
|
|
log "--notify requested but inert in this phase (D3: report-only; auto-write is a later toggle). No push."
|
|
fi
|
|
DIGEST="${DIGEST:-}" # DIGEST is composed for the future seam; keep it referenced.
|
|
log "REPORT-ONLY: groomed plan written to the mode-600 report; nothing posted (D3)."
|
|
exit 0
|
|
|
|
# ==============================================================================
|
|
# PROVISIONING (NOT DONE HERE — gated, later phases):
|
|
# - REPORT-ONLY by design (D3). The auto-write path (push the groomed plan to Notion/Jira,
|
|
# or a weekly Slack digest) is a LATER TOGGLE, flipped only once signal quality is trusted.
|
|
# The --notify seam above is intentionally inert; wiring a real destination is provisioning.
|
|
# - No systemd unit / timer is installed by this script. Wiring it into the weekly schedule
|
|
# (alongside the other Plane-1 checkers under the coordinator) is provisioning and is gated.
|
|
# - The coordinator (design §5, checker_coordinator.sh) registers + drives this planner; that
|
|
# registry edit is done centrally, NOT in this script.
|
|
# - Confluence + project_r720_agent_team memory updates are docs-as-you-go obligations for the
|
|
# build session, tracked outside this script.
|
|
# ==============================================================================
|