mirror of
https://github.com/Sea-Haven-Industries/security-review.git
synced 2026-09-30 08:03:17 +00:00
Adam Moussa
f0492b7555
review.sh only applied suppressions when handed an explicit --suppressions
FILE, so only the pre-push hook resolved them. Every other entry point (the
Open SWE daily-report automation, nightly sweep, on-demand/CI, agent runs)
called review.sh without it and therefore suppressed nothing, re-surfacing
every already-adjudicated false positive as HIGH.
When --suppressions is not passed, resolve by repo basename and MERGE both
suppression locations (machine-level first, wins id collisions):
- machine-level: ${SH_SECURITY_SUPPRESSIONS_DIR:-~/.config/sea-haven/security-review}/<basename>/suppressions.json
- repo-local: <repo>/.security-review/suppressions.json
An explicit --suppressions still overrides, so the hook and existing callers
are unaffected. Degrades gracefully off-Mac (repo-local only); fail-safe on an
unparseable file (suppresses nothing → blocks).
SECURITY (/sh-security-review, 2026-07-13): fan-out + proof-or-kill confirmed
one HIGH — the git-tracked repo-local suppressions.json lets anyone who can
commit to a scanned repo suppress a real finding and PASS an automated run
(verified by an actual exploit run; same posture nightly_sweep already had).
ACCEPTED-RISK per Adam on the condition that the automated scanners only ever
target trusted repos (no unreviewed untrusted contributions). Documented in the
auto-resolve block, README trust-model note, and a hard warning in
sweep-targets.txt. Four other candidates downgraded to low/pre-existing.
Verified: machine-level and repo-local both auto-resolve and suppress; explicit
--suppressions override still blocks; simulated off-Mac host keeps repo-local
and correctly re-blocks machine-level-only FPs.
149 lines
10 KiB
Markdown
149 lines
10 KiB
Markdown
# security-review
|
|
|
|
[](https://github.com/Sea-Haven-Industries/security-review/actions/workflows/ci.yaml)
|
|
[](https://github.com/Sea-Haven-Industries/security-review/actions/workflows/dependency-review.yml)
|
|
|
|
|
|
The Sea Haven security-review gate. One pure-code script (`review.sh`) is the decision-maker; everything
|
|
else (hooks, the interactive skill, the headless runner, the nightly sweep) is a trigger that feeds it.
|
|
See memory `project-security-review-agent` for the full design.
|
|
|
|
## Pieces
|
|
- `review.sh` — merges deterministic-scanner findings + agent findings, dedups, applies suppressions
|
|
(justification required), and makes the **block decision** (no agent decides). Exit 1 = BLOCK.
|
|
- `hooks/pre-commit`, `hooks/pre-push` + `install-hooks.sh` — fast `--scanners-only` gates. Install once
|
|
globally for every repo, or per-repo (see below).
|
|
- `skill/sh-security-review.md` — the interactive agentic detector/verifier prompt (Path A, Max-covered).
|
|
`finding.schema.json` — the structured finding contract both paths emit. `install-hooks.sh --global`
|
|
links these into `~/.claude/` (this repo is the source of truth).
|
|
- `run_headless.py` — the Path B headless detector fan-out + proof-or-kill verifier (Claude Agent SDK,
|
|
subscription OAuth). Self-contained: prompts are inline, so the VM needs no `~/.claude` assets to run it.
|
|
- `nightly_sweep.sh` + `systemd/` — the unattended two-tier sweep on the `sh-secrev` VM (R720).
|
|
|
|
## Triggers (one script, many entry points)
|
|
- **On-demand (primary):** run `/sh-security-review` in a Claude Code session (Max-covered), have it
|
|
write its schema JSON, then `review.sh --agent-findings out.json <repo>` to gate. Required before
|
|
pushing payments/auth/IaC/input-handling changes (see the global CLAUDE.md security-review rule).
|
|
- **Pre-commit / pre-push:** the global git hooks run deterministic scanners automatically.
|
|
- **Nightly:** the VM sweep (Path B) is the unattended backstop.
|
|
|
|
### Installing the hooks
|
|
```
|
|
# Global — gate EVERY repo on this machine, and link the skill + schema into ~/.claude:
|
|
./install-hooks.sh --global
|
|
|
|
# Per-repo — for a repo that sets its own core.hooksPath (e.g. husky) and would shadow the global hook:
|
|
./install-hooks.sh /path/to/repo
|
|
```
|
|
The global mode sets `git config --global core.hooksPath ~/.config/git/hooks`. Skip a repo with a
|
|
`.security-review-skip` file at its root; bypass once with `git push --no-verify`. Caveat: a repo with
|
|
its own local `core.hooksPath` overrides the global hook — install per-repo there. See memory
|
|
`reference_global_security_review_hook`.
|
|
|
|
**Suppressing a false positive.** A written justification is required and is surfaced in the report.
|
|
When no `--suppressions FILE` is passed, `review.sh` auto-resolves and **merges** suppressions from
|
|
two locations (an explicit `--suppressions` still overrides both):
|
|
1. **Machine-level (preferred), kept out of repo history:**
|
|
`${SH_SECURITY_SUPPRESSIONS_DIR:-~/.config/sea-haven/security-review}/<repo-basename>/suppressions.json`
|
|
(override the base dir with `SH_SECURITY_SUPPRESSIONS_DIR`). Keeps a suppression from becoming a
|
|
permanent in-history "ignore."
|
|
2. **Repo-local:** `<repo>/.security-review/suppressions.json` (tracked; travels with the repo).
|
|
|
|
Both files' `.suppressions[]` are concatenated (machine-level first, so it wins any id collision).
|
|
This means every entry point — the pre-push hook, the nightly sweep, on-demand/CI runs, and the
|
|
Open SWE daily-report automation — resolves suppressions identically. **Portability note:** hosts that
|
|
cannot see the Mac's `~/.config` (the Open SWE automation, the R720 VM) get only the tracked repo-local
|
|
file, so a suppression that must be honored off-Mac has to live repo-local. Fail-safe: if a file is
|
|
present but unparseable, nothing is suppressed (the gate blocks).
|
|
|
|
> **⚠️ Trust model — automated scanners must target TRUSTED repos only.** The repo-local
|
|
> `.security-review/suppressions.json` is git-tracked, so anyone who can land a commit in a scanned
|
|
> repo can suppress a real finding by committing it alongside the vuln (scanner IDs are deterministic).
|
|
> `/sh-security-review` confirmed this as a HIGH gate-bypass; it is **accepted** on the condition that
|
|
> the automated scanners (Open SWE daily report, nightly sweep) only ever scan repos that do **not**
|
|
> merge untrusted contributions without human review. See `sweep-targets.txt`. Relaxing that scoping
|
|
> requires a trust check first (signed / CODEOWNERS-verified repo-local suppressions).
|
|
|
|
Same JSON either place: `{"suppressions":[{"id":"<review.sh finding id>","justification":"…"}]}`. Caveat:
|
|
machine-level files are keyed by **repo basename**, so two repos sharing a name collide — fine for the
|
|
current single-namespace layout under `~/Documents/repositories`.
|
|
|
|
## CI
|
|
The CI here (`.github/workflows/`) is standard org code-hygiene for **this repo's own source** (ruff
|
|
lint/format + a `pytest --collect-only` import check, via the `Sea-Haven-Industries/.github` reusable
|
|
workflows) — it is **not** a security gate over other repos. The gate itself is intentionally *not*
|
|
CI-wired: for a solo dev the git hooks plus the scheduled sweeps are the backstop. The parked CI-backstop
|
|
drafts (`ci/*.yml`, `CI-BACKSTOP-NOTES.md`) were removed earlier; recover them from history if the team
|
|
ever goes multi-dev.
|
|
|
|
## Scanners
|
|
`review.sh` runs whatever is installed and logs the rest with install commands (no silent skips):
|
|
`semgrep` (`p/security-audit` + `p/secrets` + `p/javascript`), `gitleaks` (git-mode — scans committed
|
|
history, respects `.gitignore`), `checkov`, `cfn-lint`, `pip-audit`, `npm audit`. Each is normalized into
|
|
the finding schema. Install the full set:
|
|
```
|
|
pipx install semgrep pip-audit checkov # SAST / vulnerable Python deps / IaC misconfig
|
|
brew install gitleaks # hardcoded secrets
|
|
# cfn-lint via pip; Node.js provides npm audit
|
|
```
|
|
|
|
## Scheduled execution — migrating from the VM to Claude Code web routines
|
|
The unattended runs are moving off the `sh-secrev` VM into **Claude Code web scheduled routines** (which
|
|
post ALARM-only to Slack `#repo-scanner`): one routine for the agentic two-tier sweep, and a second that
|
|
runs the deterministic `checker_coordinator.sh` (the script owns the findings + ALARM decision; the
|
|
routine relays its output verbatim, never re-judging). `nightly_sweep.sh` / `checker_coordinator.sh` and
|
|
the `systemd/` units below remain the source of truth and the VM-deployment path; the VM timers are being
|
|
retired once the routines are validated.
|
|
|
|
## Nightly sweep (Path B) — two-tier, clean-clone auto-discovery
|
|
`nightly_sweep.sh` runs on the `sh-secrev` Ubuntu VM (R720) and needs **no per-repo wiring**. It:
|
|
|
|
1. **Discovers** every non-archived Sea-Haven-Industries repo via the GitHub REST API (`curl` + a
|
|
read-only `GH_TOKEN`; no `gh` CLI dependency) and **mirrors** each as a shallow clean clone
|
|
(`git clone --depth=1`, default branch from the API) into `~/repo-mirrors`. Scanning server-side
|
|
clones — not developer working trees — structurally keeps local gitignored `.env` secrets out of scope.
|
|
2. **Tier 1 (every repo, every night, $0 Claude):** `review.sh --scanners-only` over every mirror —
|
|
complete deterministic baseline coverage.
|
|
3. **Tier 2 (bounded agentic):** `run_headless.py` over a deterministic **round-robin rotation** that
|
|
fits `TOTAL_BUDGET_USD`, with a persistent cycle pointer so every repo gets a deep pass within
|
|
`MAX_CYCLE_NIGHTS`. This bounds the draw on the shared Max limits (a clean night never deep-scans all
|
|
repos). A `COVERAGE ALARM` fires if the rotation falls behind.
|
|
|
|
It is **ALARM-only**: a clean night posts nothing. See memory `feedback_cloudwatch_alarms`.
|
|
|
|
### Skip / override
|
|
- A repo is skipped if it commits a `.security-review-skip` marker **or** is listed in the central skip
|
|
file (`~/.secrev-skip.txt`, one repo name per line). Repos skipped via their own committed marker are
|
|
**logged in the report** so a sensitive repo can't silently self-exclude.
|
|
- `TARGETS="/path/a /path/b"` overrides discovery entirely (scan explicit paths, no cloning).
|
|
- The canary corpus (`~/security-review-testbed`) is **always** scanned agentically first as the
|
|
anti-complacency check — independent of the skip filter.
|
|
|
|
### Anti-complacency + guards
|
|
- **Canary check:** the testbed MUST block AND surface ≥ `CANARY_FLOOR` (default 10) confirmed crit/high.
|
|
Otherwise → COMPLACENCY ALARM. The corpus now includes Node + .NET fixtures (see the testbed key);
|
|
re-tune the floor after the first VM canary run reports the expanded recall number.
|
|
- **Budget ceiling:** `TOTAL_BUDGET_USD` (default 120 — full deep-pass coverage of every repo per night) caps aggregate agentic spend; `PER_TARGET_BUDGET_USD`
|
|
(default 12) caps each repo; `MAX_AGENTIC_PER_NIGHT` (default 0 = unlimited) optionally caps wall-clock.
|
|
With the SDK-billing split deferred (memory `reference-claude-subscription-billing`), spend draws from
|
|
the Max subscription limits, so the two-tier design keeps full coverage cheap and bounds the agentic draw.
|
|
- **Two-model hook (optional, off):** `ENABLE_XMODEL_HOOK=1` re-checks each confirmed CRITICAL with the
|
|
orchestrator cross-family reviewer (GPT-4.1) and flags disagreement; skips gracefully, never fails the sweep.
|
|
- **Redaction:** secret-shaped values are masked in the Slack ALARM string; on-disk reports are mode 600.
|
|
|
|
### Secrets / env (`~/secrev.env`, mode 600)
|
|
- `CLAUDE_CODE_OAUTH_TOKEN` — required (`run_headless.py` pops `ANTHROPIC_API_KEY`).
|
|
- `GH_TOKEN` — **read-only fine-grained PAT** scoped to the org (Contents + Metadata: read-only, nothing
|
|
else) for discovery + cloning. Never give this unattended box a write-capable token.
|
|
- `SLACK_WEBHOOK_URL` — alarms (plain incoming-webhook). `~/orchestrator/.env` → `OPENAI_API_KEY` (xmodel hook only).
|
|
- Reports + per-target JSON land under `~/sweep-reports/<UTC-date>/`.
|
|
|
|
### Install the timer
|
|
Units are in `systemd/`; full runbook is `DEPLOY-R720.md`. On the VM:
|
|
```
|
|
sudo cp systemd/sea-haven-secrev.{service,timer} /etc/systemd/system/
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl enable --now sea-haven-secrev.timer # the timer drives it; do not enable the .service
|
|
sudo systemctl start sea-haven-secrev.service # optional one-off smoke test
|
|
```
|
|
The timer fires nightly at ~02:00 local (`Persistent=true` catches missed runs after downtime).
|