shoc-pr-review-runner

Private, cloud-hosted PR review runner for the SHOC project (shoc-frontend-new + shoc-backend). A manually-dispatched GitHub Actions workflow checks out the exact PR head(s) in a clean environment, runs the repositories' real build/test gates, generates a truthful evidence report, invokes a Fireworks-hosted review model with the SHOC review skill, validates the output deterministically, and publishes the review + evidence as run artifacts for a human to copy into GitHub.

The runner never writes to the product repositories or their PRs. The workflow token has contents: read only, product-repo access uses a read-only GitHub App installation token, and no write-scoped credential exists in this repository.

Running a review

1. Actions → Review PR → Run workflow.
2. Choose review_type (frontend | backend | paired) and enter the PR number(s) or URL(s). Optional: ticket, reviewer notes, model.
3. When the run finishes, the review appears in the job summary and in the review-<run-id> artifact together with review-evidence.md and all logs.
4. Copy review.md into the GitHub PR manually. The runner never posts it.

Single-repo reviews check out the companion repository at its dev head for contract context; it is not built or reviewed.

Phase status

Phase 1 (current): input validation, exact-head checkout, clean build/test gates, mocked Playwright, evidence report, agent invocation, output validation, artifacts. Not yet implemented (spec Phases 2–4): disposable SQL Server + migrations, backend/frontend startup + health gates, live Playwright against a real backend, stacked/paired PR intelligence. The evidence report marks all of these NOT_RUN — reviews cannot claim them.

Layout

Path	Purpose
skills/pr-review/	Coordinating skill + frontend/backend checklists + output contract
.github/workflows/review-pr.yml	The review workflow (workflow_dispatch)
.github/workflows/ci.yaml	Repo CI caller (emits the required ci / ci context)
.github/workflows/ci-runner-checks.yaml	Reusable CI: shellcheck, actionlint, schema check, bash tests
scripts/	Orchestration scripts (see headers in each)
review/runner-config.yml	Source-of-truth config record (mirrored by the workflow env)
review/schemas/	Input contract schema
templates/	Evidence / request / failure-summary templates
tests/	Bash test suites + fixtures (run in CI)

Required secrets

Secret	Purpose
SHOC_REVIEW_APP_ID	GitHub App ID (read-only app, see below)
SHOC_REVIEW_APP_PRIVATE_KEY	The App's private key (PEM)
FIREWORKS_API_KEY	Fireworks inference API key

run-review-agent.sh fails fast with a clear error when the Fireworks key is missing or rejected.

GitHub App

The App (shoc-review-runner) must have exactly these permissions and nothing else, installed on only shoc-frontend-new and shoc-backend:

• Repository permissions: Contents Read-only, Pull requests Read-only, Checks Read-only, Metadata Read-only.

Checks read is what lets the runner report the reviewed PR's own CI status. Without it the runner records that signal as unknown rather than guessing.

Verify after installing (and re-verify when the App changes):

gh api /orgs/Sea-Haven-Industries/installations \
  --jq '.installations[] | select(.app_slug=="shoc-review-runner")
        | {permissions, repository_selection}'
# permissions must be exactly contents/pull_requests/checks/metadata = "read",
# and repository_selection must be "selected"

Listing the selected repositories requires the installation token itself (GET /installation/repositories), which only exists inside a workflow run. In practice the token mint is the check: actions/create-github-app-token requests shoc-frontend-new,shoc-backend explicitly and fails the run if the App is not installed on both.

Security architecture

Reviewing a PR means building it, and building it means executing code the PR author wrote (npm lifecycle scripts, eslint/vite/vitest configs, MSBuild targets). The workflow is therefore split into two jobs:

Job	Executes PR code	Secrets present
gates	yes	App token, revoked before the first build command runs
review	no	Fireworks API key only

Step-level env: is not an isolation boundary inside a job — PR code can append to $GITHUB_ENV to alter later steps, read a later step's process environment, or overwrite the runner's own scripts. The job split is what makes those attacks worthless: by the time any PR code runs, the gates job holds no usable credential, and the review job re-checks out this repository fresh so tampered scripts cannot follow it.

Further controls:

• Product repos are checked out under $RUNNER_TEMP, never beside this repo's scripts/.
• The App token is downscoped at mint time (permission-contents: read and friends), so it stays read-only even if the App installation is later granted broader permissions.
• Fork PR heads are refused outright.
• Gate results are recorded once per key outside the workspace, and every decision path calls assert_gate_table_intact first: a duplicate key means something other than the runner wrote the table, and the run fails closed rather than trusting a forged PASS.
• Changed-file contents are read from git objects, never the filesystem, so a symlink committed in a PR cannot pull host files into the prompt.
• The review prompt fences untrusted material with a per-run nonce, and validate-review-output.sh re-checks every claim against the recorded gate table rather than trusting the model.

Residual risk, stated plainly: someone with push access to a product repo can make their own PR's gates report success by having the build fake it. The integrity check turns the obvious forms of that into a hard failure, but a CI system that builds untrusted code cannot fully certify its own results. The review is a reviewing aid, not an authority — a human still reads the diff.

Data flow to third parties

The PR diff and the contents of changed files are sent to Fireworks AI (api.fireworks.ai) as the review prompt. This is private SHOC source leaving the Sea Haven boundary to an external inference provider. Confirm the Fireworks account has training and retention disabled before reviewing anything sensitive.

Artifacts are split so raw source is not retained as long as the review:

Artifact	Contents	Retention
review-<run-id>	review, evidence report, gate table, gate logs	30 days
review-context-<run-id>	prompt, model responses, raw diffs	2 days

Anyone with read access to this repository can read those artifacts. Keep this repository's read audience no broader than both product repositories'.

Artifact hygiene

scripts/redact-check.sh scans every staged artifact for the run's secret values before upload and blocks the upload on any hit. If sensitive content is ever discovered in a published artifact, delete it immediately:

gh api repos/Sea-Haven-Industries/shoc-pr-review-runner/actions/artifacts \
  --jq '.artifacts[] | {id, name, created_at}'
gh api -X DELETE \
  repos/Sea-Haven-Industries/shoc-pr-review-runner/actions/artifacts/<id>

Artifact retention is 30 days.

Provisioning notes

• Repo settings: allow_auto_merge + delete_branch_on_merge enabled; org Code Security Configuration "Sea Haven Standard" attached.
• CodeQL exemption: this repository contains only shell, YAML, Markdown, and JSON — no CodeQL-supported language — so CodeQL default setup is not enabled. Revisit if a supported language is ever added.
• Dependabot: github-actions ecosystem, weekly.

Review instructions live here, not in the product repos

The frontend and backend review checklists are deliberately not committed to shoc-frontend-new or shoc-backend (spec §27.1). They were ported from the reviewers' local .cursor/commands/pr-review.md files on 2026-07-29; this repository is now their source of truth.