shoc-pr-review-runner/README.md

# shoc-pr-review-runner

Private, cloud-hosted PR review runner for the SHOC project
(`shoc-frontend-new` + `shoc-backend`). A manually-dispatched GitHub Actions
workflow checks out the exact PR head(s) in a clean environment, runs the
repositories' real build/test gates, generates a truthful evidence report,
invokes a Fireworks-hosted review model with the SHOC review skill, validates
the output deterministically, and publishes the review + evidence as run
artifacts for a human to copy into GitHub.

**The runner never writes to the product repositories or their PRs.** The
workflow token has `contents: read` only, product-repo access uses a read-only
GitHub App installation token, and no write-scoped credential exists in this
repository.

## Running a review

1. Actions → **Review PR** → *Run workflow*.
2. Choose `review_type` (`frontend` | `backend` | `paired`) and enter the PR
   number(s) or URL(s). Optional: ticket, reviewer notes, model.
3. When the run finishes, the review appears in the job summary and in the
   `review-<run-id>` artifact together with `review-evidence.md` and all logs.
4. Copy `review.md` into the GitHub PR manually. The runner never posts it.

Single-repo reviews check out the companion repository at its `dev` head for
contract context; it is not built or reviewed.

## Phase status

Phase 1 (current): input validation, exact-head checkout, clean build/test
gates, mocked Playwright, evidence report, agent invocation, output validation,
artifacts. **Not yet implemented** (spec Phases 2–4): disposable SQL Server +
migrations, backend/frontend startup + health gates, live Playwright against a
real backend, stacked/paired PR intelligence. The evidence report marks all of
these NOT_RUN — reviews cannot claim them.

## Layout

| Path | Purpose |
| --- | --- |
| `skills/pr-review/` | Coordinating skill + frontend/backend checklists + output contract |
| `.github/workflows/review-pr.yml` | The review workflow (workflow_dispatch) |
| `.github/workflows/ci.yaml` | Repo CI caller (emits the required `ci / ci` context) |
| `.github/workflows/ci-runner-checks.yaml` | Reusable CI: shellcheck, actionlint, schema check, bash tests |
| `scripts/` | Orchestration scripts (see headers in each) |
| `review/runner-config.yml` | Source-of-truth config record (mirrored by the workflow env) |
| `review/schemas/` | Input contract schema |
| `templates/` | Evidence / request / failure-summary templates |
| `tests/` | Bash test suites + fixtures (run in CI) |

## Required secrets

| Secret | Purpose |
| --- | --- |
| `SHOC_REVIEW_APP_ID` | GitHub App ID (read-only app, see below) |
| `SHOC_REVIEW_APP_PRIVATE_KEY` | The App's private key (PEM) |
| `FIREWORKS_API_KEY` | Fireworks inference API key |

`run-review-agent.sh` fails fast with a clear error when the Fireworks key is
missing or rejected.

## GitHub App

The App must have **exactly** these permissions and nothing else, installed on
**only** `shoc-frontend-new` and `shoc-backend`:

- Repository permissions: Contents **Read-only**, Pull requests **Read-only**,
  Metadata **Read-only**.

Verify after installing (and re-verify when the App changes):

```sh
APP_INSTALLS=$(gh api /orgs/Sea-Haven-Industries/installations --jq \
  '.installations[] | select(.app_slug=="<app-slug>")')
echo "$APP_INSTALLS" | jq '{permissions, repository_selection}'
# permissions must be exactly {contents: "read", pull_requests: "read", metadata: "read"}
gh api "/user/installations/$(echo "$APP_INSTALLS" | jq -r .id)/repositories" \
  --jq '.repositories[].full_name'
# must list exactly the two product repos
```

## Security architecture

Reviewing a PR means building it, and building it means executing code the PR
author wrote (npm lifecycle scripts, eslint/vite/vitest configs, MSBuild
targets). The workflow is therefore split into two jobs:

| Job | Executes PR code | Secrets present |
| --- | --- | --- |
| `gates` | yes | App token, revoked before the first build command runs |
| `review` | no | Fireworks API key only |

Step-level `env:` is not an isolation boundary inside a job — PR code can
append to `$GITHUB_ENV` to alter later steps, read a later step's process
environment, or overwrite the runner's own scripts. The job split is what makes
those attacks worthless: by the time any PR code runs, the gates job holds no
usable credential, and the review job re-checks out this repository fresh so
tampered scripts cannot follow it.

Further controls:

- Product repos are checked out under `$RUNNER_TEMP`, never beside this repo's
  `scripts/`.
- The App token is downscoped **at mint time** (`permission-contents: read` and
  friends), so it stays read-only even if the App installation is later granted
  broader permissions.
- Fork PR heads are refused outright.
- Gate results are recorded once per key outside the workspace, and every
  decision path calls `assert_gate_table_intact` first: a duplicate key means
  something other than the runner wrote the table, and the run fails closed
  rather than trusting a forged `PASS`.
- Changed-file contents are read from git objects, never the filesystem, so a
  symlink committed in a PR cannot pull host files into the prompt.
- The review prompt fences untrusted material with a per-run nonce, and
  `validate-review-output.sh` re-checks every claim against the recorded gate
  table rather than trusting the model.

**Residual risk, stated plainly:** someone with push access to a product repo
can make their own PR's gates report success by having the build fake it. The
integrity check turns the obvious forms of that into a hard failure, but a CI
system that builds untrusted code cannot fully certify its own results. The
review is a reviewing aid, not an authority — a human still reads the diff.

## Data flow to third parties

The PR diff and the contents of changed files are sent to **Fireworks AI**
(`api.fireworks.ai`) as the review prompt. This is private SHOC source leaving
the Sea Haven boundary to an external inference provider. Confirm the Fireworks
account has training and retention disabled before reviewing anything sensitive.

Artifacts are split so raw source is not retained as long as the review:

| Artifact | Contents | Retention |
| --- | --- | --- |
| `review-<run-id>` | review, evidence report, gate table, gate logs | 30 days |
| `review-context-<run-id>` | prompt, model responses, raw diffs | 2 days |

Anyone with read access to this repository can read those artifacts. Keep this
repository's read audience no broader than both product repositories'.

## Artifact hygiene

`scripts/redact-check.sh` scans every staged artifact for the run's secret
values before upload and blocks the upload on any hit. If sensitive content is
ever discovered in a published artifact, delete it immediately:

```sh
gh api repos/Sea-Haven-Industries/shoc-pr-review-runner/actions/artifacts \
  --jq '.artifacts[] | {id, name, created_at}'
gh api -X DELETE \
  repos/Sea-Haven-Industries/shoc-pr-review-runner/actions/artifacts/<id>
```

Artifact retention is 30 days.

## Provisioning notes

- Repo settings: `allow_auto_merge` + `delete_branch_on_merge` enabled; org
  Code Security Configuration "Sea Haven Standard" attached.
- **CodeQL exemption:** this repository contains only shell, YAML, Markdown,
  and JSON — no CodeQL-supported language — so CodeQL default setup is not
  enabled. Revisit if a supported language is ever added.
- Dependabot: `github-actions` ecosystem, weekly.

## Review instructions live here, not in the product repos

The frontend and backend review checklists are deliberately **not** committed
to `shoc-frontend-new` or `shoc-backend` (spec §27.1). They were ported from
the reviewers' local `.cursor/commands/pr-review.md` files on 2026-07-29; this
repository is now their source of truth.