docs: document live AWS prod deploy for Open SWE (#53)

Prod went live 2026-06-29 on self-hosted AWS EC2 behind the shared seahaven-com ALB, superseding the on-prem VM model the runbook described. - Rewrite deploy/seahaven/DEPLOYMENT.md as the canonical end-to-end runbook: infra CD (CDK stacks + OIDC roles + prod approval gate), config seeding (put-config.sh, the 13 boot-required prod vars, fetch-config fail-fast), app artifact deploy (S3 + SSM roll + is-active gate), promotion/rollback, live prod facts, and a RETAIN secret-shell troubleshooting entry that cross-references infra/README.md. - Correct retired *.seahavenind.com hosts to *.seahaven.com throughout and document the live GitHub/Slack/Linear webhook + OAuth endpoints. - Add a concise Deployment section to README pointing at the runbook. - Fix the stale host in the retired on-prem nginx/openswe.conf and mark it superseded by the AMI template.
2026-09-30 05:43:14 +00:00 · 2026-06-29 11:12:44 -04:00 · 2026-06-29 11:12:44 -04:00 · 9444fd7677
commit 9444fd7677
parent f379fbdaa9
3 changed files with 273 additions and 94 deletions
--- a/README.md
+++ b/README.md
@ -151,6 +151,21 @@ This is an area where you can extend Open SWE for your org: add deterministic CI
 - **[Installation Guide](INSTALLATION.md)** — local dev (backend + dashboard), GitHub App creation, LangSmith, Linear/Slack/GitHub triggers, and production deployment
 - **[Customization Guide](CUSTOMIZATION.md)** — swap the sandbox, model, tools, triggers, system prompt, and middleware for your org

+## Deployment (Sea Haven fork)
+
+This fork is **self-hosted on AWS** and live in production. Each env (`dev` /
+`prod`) runs the stock `langgraph dev` server (all three graphs + the FastAPI
+webapp) bound to loopback `127.0.0.1:2024` on a single ARM64 EC2 box, fronted by
+nginx (the sole ingress) behind the shared `seahaven-com` ALB. CDK (`infra/`)
+owns the per-env stacks; GitHub Actions handle CDK deploys (`cd-infra.yml`) and
+app-artifact releases to S3 rolled onto the box via an SSM document
+(`build-artifacts.yml`), with `dev` auto-deploying and `prod` gated behind a
+manual GitHub Environment approval.
+
+**[`deploy/seahaven/DEPLOYMENT.md`](deploy/seahaven/DEPLOYMENT.md) is the canonical
+deploy runbook** — full end-to-end pipeline, config seeding, promotion/rollback,
+and live prod facts. CDK specifics live in [`infra/README.md`](infra/README.md).
+
 ## License

 MIT
--- a/deploy/seahaven/DEPLOYMENT.md
+++ b/deploy/seahaven/DEPLOYMENT.md
@ -1,118 +1,278 @@
-# Sea Haven — Open SWE self-hosted deployment
+# Sea Haven — Open SWE deployment runbook

-How this fork is deployed at Sea Haven. The runtime is the **stock LangGraph dev
-server** (not the Aegra path — see [Aegra](#aegra-deferred)). Internal addresses,
-ARNs, and account IDs are shown as `<PLACEHOLDERS>`; the real values live in the
-private IT docs (Confluence "AWS Architecture Map") — **do not commit them to this
-public fork.**
+How this fork is deployed at Sea Haven. **PROD is LIVE as of 2026-06-29.** The
+runtime is the **stock LangGraph dev server** (not the Aegra path — see
+[Aegra](#aegra-deferred)), running on a self-hosted ARM64 EC2 box behind the
+shared `seahaven-com` ALB.

-## Topology
+Account `328440206208`, region `us-east-1`. Internal addresses, ARNs, snapshot
+ids, and account-scoped values that are sensitive are shown as `<PLACEHOLDERS>`;
+the real values live in the private IT docs (Confluence "AWS Architecture Map",
+id 1540098) and in AWS — **do not commit them to this public fork.**

-```
-GitHub / Slack ──▶ hooks.seahavenind.com ──┐
-                                            │  (public ALB :443, host+path rule)
-Browser ─────────▶ openswe.seahavenind.com ─┤
-                                            ▼
-                              AWS ALB  ──(Site-to-Site VPN)──▶ on-prem VM
-                                                                ├─ nginx :80  (dashboard SPA + /dashboard/api proxy)
-                                                                └─ langgraph dev :2024 (3+ graphs + FastAPI webapp)
-                                                                       └─▶ LangSmith cloud sandbox (build/git/PR)
-```
+This is the canonical deploy runbook. The CDK details live in
+[`infra/README.md`](../../infra/README.md); the rotation procedure in
+[`ROTATION.md`](ROTATION.md).

- The VM is **internet-closed**; all inbound rides the existing ALB over the VPN.
- **Webhooks** (`hooks.seahavenind.com`) → ALB listener rule scoped to `/webhooks/*`
-  only → VM `:2024`. The unauthenticated LangGraph API (`/threads`, `/runs`,
-  `/assistants`, `/store`) is never path-forwarded.
- **Dashboard** (`openswe.seahavenind.com`) → ALB → VM `:80` (nginx). nginx is the
-  security boundary: it serves the static SPA and proxies **only** `/dashboard/api/`
-  to `:2024`; the agent API is not reachable through it.
+## Live prod facts (2026-06-29)

-## VM components
-
-| Component | What |
+| | |
 |---|---|
-| `langgraph dev` | systemd `open-swe.service` — `--host 0.0.0.0 --port 2024 --no-browser --no-reload`. In-memory runtime. |
-| Store seeding | `seed_store.sh` as `ExecStartPost` (re-seeds team settings + user mappings, which the in-memory store loses on restart). |
-| nginx | `nginx/openswe.conf` — SPA from `/var/www/openswe`, proxy `/dashboard/api/` → `:2024`. |
-| Postgres | present (was for the Aegra path); unused by the stock in-memory runtime. |
-| swap | 8 GB swapfile — **required**: the dashboard (`ui/`) Nitro build OOMs on an 8 GB box without it. |
+| Dashboard | `https://openswe.seahaven.com` |
+| Webhooks | `https://hooks.seahaven.com/webhooks/*` |
+| Ingress | shared internet-facing ALB `app/seahaven-com` → target group `open-swe-prod-tg` → EC2 `i-08a729e50779c4b07` (`t4g.large`, ARM64) on nginx `:80` |
+| Backend | `langgraph dev` bound to `127.0.0.1:2024` (loopback only); nginx is the sole ingress |
+| CDK stacks | `open-swe-iam` (OIDC roles) · `open-swe-dev` · `open-swe-prod` |

-### Models
-Model selection is **store-driven**, not env. `LLM_MODEL_ID` is effectively dead for
-runtime selection; the `team_settings/default` store doc wins (then per-user profile,
-then per-thread). Defaults seeded by `seed_store.sh`:
- builder: `anthropic:claude-opus-4-8` (effort `high`)
- reviewer (cross-family): `openai:gpt-5.5` (effort `high`) — `openai:gpt-4.1` is **not**
-  in this fork's `SUPPORTED_MODELS` (`agent/dashboard/options.py`); a raw value is
-  silently rewritten to gpt-5.5. Add it to `SUPPORTED_MODELS` first if you need 4.1.
- `analyzer` graph is hardcoded to the code default and ignores team settings.
+Dev mirrors prod with `-dev` hosts (`openswe-dev.seahaven.com` /
+`hooks-dev.seahaven.com`), a `t4g.medium` box, and no GitHub-App/Slack/webhook
+integration (it is a deployment-validation env, not a live-triggered agent).

-## Build & deploy the dashboard (`ui/`)
+> The retired on-prem `*.seahavenind.com` ALB routing and DNS were removed on
+> 2026-06-29; prod is now live exclusively on `*.seahaven.com`.

-`ui/` is a **TanStack Start + Nitro** app (build with `bun`, not plain Vite):
+## Hosting model
+
+```
+GitHub / Slack ──▶ hooks.seahaven.com   ──┐
+                                          │  (shared ALB :443, host+path rules)
+Browser ─────────▶ openswe.seahaven.com ──┤
+                                          ▼
+                       ALB app/seahaven-com ──▶ open-swe-prod-tg ──▶ EC2 box :80 (nginx)
+                                                                       ├─ nginx — SPA + scoped proxy
+                                                                       │    /dashboard/api/* and /webhooks/*
+                                                                       └─ langgraph dev 127.0.0.1:2024
+                                                                            └─▶ LangSmith cloud sandbox (build/git/PR)
+```
+
+- A **single** VPC and a **single** internet-facing ALB (`app/seahaven-com`) are
+  shared with the on-prem `seahaven-site` stack. open-swe **imports** the VPC,
+  ALB SG, `:443` listener, and `seahaven.com` zone — it never owns/mutates them;
+  it only adds its own instance SG, a standalone ALB-egress rule, two listener
+  rules, a target group, and Route53 aliases.
+- The EC2 box is in a **private** subnet (us-east-1a, same AZ as the single NAT
+  for in-AZ egress). It is reachable **only** from the shared ALB SG on `:80`.
+- **nginx is the security boundary.** It serves the static dashboard SPA and
+  proxies exactly two prefixes to `:2024` — `/dashboard/api/*` and `/webhooks/*`.
+  The unauthenticated LangGraph API (`/threads`, `/runs`, `/assistants`,
+  `/store`) is never proxied; those paths return the SPA shell. `:2024` is
+  loopback-only and never network-reachable, even inside the SG.
+- **Webhooks** ride listener rules below the on-prem host-agnostic `/webhooks/*`
+  rule (priority 2 dev / 3 prod, host-scoped to the open-swe hosts) so they reach
+  the open-swe box and never steal an on-prem host's webhooks.
+
+The box holds **no durable state of its own**: secrets/config are materialized to
+a tmpfs `.env` at boot, the app artifact is pulled from S3, and the in-memory
+LangGraph store is re-seeded on every start. Replacement is tolerated; there is no
+RETAIN volume.
+
+---
+
+## Deploy pipeline (end to end)
+
+Two independent CD lanes, both OIDC-only (no static keys), both with a manual
+approval gate on prod via the GitHub **`prod` Environment** (required reviewer:
+Adam). The `environment: prod` declaration both fires the approval gate and makes
+the OIDC subject `…:environment:prod`, which is the only subject the prod deploy
+roles trust — so a dev-branch token can never reach prod.
+
+### (a) Infra CD — `cd-infra.yml`
+
+Deploys the CDK stacks. Path-filtered to `infra/**`.
+
+```
+push to dev  → Infra CI (tsc + jest + cdk synth) → cdk deploy OpenSweDevStack   (AUTO, CI-green-gated)
+push to main → Infra CI                          → cdk deploy OpenSweProdStack  (manual approval: env "prod")
+```
+
+- Roles: `githubdeploy-open-swe-infra-{dev,prod}` (in the `open-swe-iam` stack;
+  set as repo variables `AWS_DEPLOY_ROLE_INFRA_{DEV,PROD}`).
+- It targets **one stack explicitly per env** (`cdk deploy OpenSweDevStack` /
+  `OpenSweProdStack`), not `cdk deploy --all`, so a single-env push can never
+  deploy the other env or the shared IAM stack.
+- The shared `open-swe-iam` stack (owns both envs' OIDC deploy roles) is **not**
+  deployed by CD — it is a privileged, human-gated apply.
+
+**Stack order on a clean account:** `open-swe-iam` first (creates the OIDC roles;
+set the repo deploy-role variables and configure the `prod` Environment reviewer
+from its outputs), then `open-swe-dev`, then `open-swe-prod`.
+
+### (b) Seed the config store — `put-config.sh <env>`
+
+Run **after** `cdk deploy open-swe-<env>` and **before** the box first boots. CDK
+creates the value-less Secrets Manager shells (`open-swe-<env>/<VAR>`) and the
+IaC-managed SSM params (`/open-swe-<env>/<VAR>`); `put-config.sh` populates the
+secret values plus the out-of-band SSM params that cannot live in IaC.

 ```bash
-cd ui
-export PATH="$HOME/.bun/bin:$PATH"
-export NODE_OPTIONS=--max-old-space-size=6144   # + the 8 GB swapfile, or the build OOMs
-bun install
-bun run build                                   # -> .output/public (static SPA, _shell.html)
-sudo cp -r .output/public/. /var/www/openswe/   # served by nginx
+deploy/seahaven/put-config.sh <dev|prod>     # set each value inline, via OPENSWE_PUT_<VAR>, or from a vault
+deploy/seahaven/fetch-config.sh <dev|prod>   # (on the box) fail-fast verify before first start
 ```

-Served as a static SPA (per `ui/vercel.json`); the Nitro `.output/server` is unused.
+`put-config.sh` ships `<FILL>` placeholders only — **no real secret values are
+committed**. It does not touch the IaC-managed SSM params (CDK owns those).

-## Install / wire-up checklist
+**13 prod boot-required vars** — `fetch-config.sh` fail-fasts (refuses to write a
+partial `.env`, the unit does not start) if any are missing/empty:

-1. App config in `.env` (gitignored — never commit): LLM keys, GitHub App creds,
-   `LANGSMITH_API_KEY*` + `DEFAULT_SANDBOX_SNAPSHOT_ID` (`SANDBOX_TYPE=langsmith` — the
-   only sandbox provider with working in-sandbox git/gh auth), `LANGGRAPH_URL=http://127.0.0.1:2024`,
-   dashboard vars (`DASHBOARD_JWT_SECRET`, `CONFIGURED_ADMINS`, `DASHBOARD_*_URL=https://openswe.seahavenind.com`).
-2. `systemd/open-swe.service` → `/etc/systemd/system/`, `seed_store.sh` on the VM with
-   `OPENSWE_*` env exported (owner login/email, default repo, model ids).
-3. `nginx/openswe.conf` → `/etc/nginx/sites-available/openswe`, symlink into
-   `sites-enabled`, remove the default site, `nginx -t && systemctl reload nginx`.
-4. AWS (real IDs in Confluence): IP target groups → `<VM_LAN_IP>:2024` and `:80`;
-   ALB SG **egress** rules to those ports (the ALB SG is allow-listed — health checks
-   time out without them); `:443` listener rules for the two hostnames; Route53 ALIAS
-   records → ALB. Webhook rule must stay path-scoped to `/webhooks/*`.
-5. GitHub App: webhook URL `https://hooks.seahavenind.com/webhooks/github`; subscribe to
-   the events the install guide lists (Issue comment, PR review×2, check_run/suite,
-   workflow_run, status) — add **Issues** too if you want issue-title/body triggers.
-6. **GitHub App OAuth callback (manual, UI-only — not API-settable):**
-   `https://openswe.seahavenind.com/dashboard/api/auth/callback` — without it, dashboard
-   login fails with a `redirect_uri` mismatch.
+- **9 secrets** (Secrets Manager `open-swe-prod/<VAR>`): `DASHBOARD_JWT_SECRET`,
+  `TOKEN_ENCRYPTION_KEY`, `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`,
+  `LANGSMITH_API_KEY_PROD`, `GITHUB_APP_PRIVATE_KEY`, `GITHUB_APP_CLIENT_SECRET`,
+  `GITHUB_WEBHOOK_SECRET`, `SLACK_SIGNING_SECRET`.
+- **4 SSM params** (`/open-swe-prod/<VAR>`): `DEFAULT_SANDBOX_SNAPSHOT_ID`,
+  `GITHUB_APP_ID`, `GITHUB_APP_INSTALLATION_ID`, `GITHUB_APP_CLIENT_ID`.
+
+(`ANTHROPIC_API_KEY` + `OPENAI_API_KEY` are required because that is the seeded
+cross-family pair; the active set follows `REQUIRED_PROVIDER_KEYS`. The
+`LANGSMITH_API_KEY_PROD` + `DEFAULT_SANDBOX_SNAPSHOT_ID` pair is required because
+`SANDBOX_TYPE=langsmith`.) Dev boots without the GitHub-App / Slack / webhook
+secrets — it has no such integration.
+
+`fetch-config.sh` runs as an `ExecStartPre=+` hook (root, only long enough to
+write the `openswe`-owned `0600` tmpfs `.env`), reads all `/open-swe-<env>/*` SSM
+params + all `open-swe-<env>/*` secrets via the instance role, and forces
+`DEFAULT_REPO_OWNER` away from the upstream `langchain-ai` org.
+
+### (c) App artifact deploy — `build-artifacts.yml`
+
+Builds the release and rolls the box. Path-filtered to `agent/**`, `ui/**`,
+`deploy/**`, `langgraph.json`, `pyproject.toml`, `uv.lock`.
+
+```
+push to dev  → build SPA + package → open-swe-dev-assets/releases/  → SSM open-swe-dev-deploy  (AUTO)
+push to main → build SPA + package → open-swe-prod-assets/releases/ → SSM open-swe-prod-deploy (manual approval: env "prod")
+```
+
+1. The dashboard SPA is built **on the runner** (`bun run build` → vite →
+   `ui/.output/public`) — the box is small, so the memory-heavy build runs in CI.
+2. `package-artifacts.sh` produces two tarballs: `spa.tar.gz` (built SPA) and
+   `app.tar.gz` (Python source tree — no `ui/`, no `.venv`).
+3. Both are uploaded to S3 `open-swe-<env>-assets` under `releases/<sha>/`
+   (immutable, auditable) and mirrored to `releases/latest/` (what the box pulls).
+4. CI fires the `open-swe-<env>-deploy` SSM document (tag-scoped to
+   `project=open-swe,env=<env>`), which runs `/opt/open-swe/bin/deploy.sh` on the
+   box: pull the release from S3, build a native-ARM64 venv with
+   `uv sync --frozen --no-dev`, extract the SPA to the nginx web root,
+   `systemctl restart open-swe.service`, reload nginx, then gate on
+   `systemctl is-active --quiet open-swe.service` (a non-active unit exits the
+   deploy non-zero).
+
+Roles: `githubdeploy-open-swe-app-{dev,prod}` (repo variables
+`AWS_DEPLOY_ROLE_APP_{DEV,PROD}`) — tag-scoped `ssm:SendCommand` on the deploy
+document only (not the generic `AWS-RunShellScript`) + write to the env's S3
+bucket.
+
+Secrets/config are **not** fetched by `deploy.sh`; the `systemctl restart`'s
+`ExecStartPre=fetch-config.sh` re-materializes the `.env` on every restart, so a
+bad config surfaces as a failed unit.
+
+### (d) dev → main promotion + rollback
+
+**Promotion — `promote-dev-to-prod.yml`** (nightly cron `0 8 * * *` + manual
+dispatch): mints a GitHub App installation token (a bypass actor on the `main`
+ruleset), gates on **every check-run on the dev HEAD commit being completed and
+passing**, then **fast-forward-only** pushes `dev` → `main`. A diverged `main`
+fails loudly rather than force-updating. The push to `main` is what triggers the
+prod lanes of `cd-infra.yml` / `build-artifacts.yml` (each still behind the `prod`
+Environment approval). Re-gating via a PR on `main` would be redundant since the
+commit already passed every check on dev.
+
+**Rollback — `rollback.yml`** (manual dispatch, `env` + optional `sha`): re-points
+`releases/latest/` at a prior release and re-fires the `open-swe-<env>-deploy` SSM
+document — same fire/wait/gate path as a forward deploy, no rebuild.
+
+```
+env=dev,  sha blank → restore open-swe-dev-assets/releases/last-good/   (AUTO)
+env=prod, sha blank → restore open-swe-prod-assets/releases/last-good/  (manual approval: env "prod")
+sha=<commit>        → restore that exact releases/<sha>/ instead
+```
+
+It reuses the existing `githubdeploy-open-swe-app-<env>` role (no new IAM).
+
+---
+
+## On-box layout (reference)
+
+| Path | What |
+|---|---|
+| `open-swe.service` (systemd) | `langgraph dev --host 127.0.0.1 --port 2024 --no-browser --no-reload` as the unprivileged `openswe` user. In-memory runtime. |
+| `fetch-config.sh` | `ExecStartPre=+` — materializes the tmpfs `.env` from Secrets Manager + SSM, fail-fast. |
+| `seed_store.sh` | `ExecStartPost` — re-seeds `team_settings/default` + `user_mappings` (the in-memory store loses them on every restart). |
+| nginx | SPA from `/var/www/open-swe`, proxy `/dashboard/api/` + `/webhooks/` → `127.0.0.1:2024`, `/healthz` → 200. |
+| `deploy.sh` | the release procedure run on first boot (non-fatal) and by every SSM deploy. |
+| CloudWatch logs | `/open-swe/<env>/{app,user-data,nginx-access,nginx-error}` at 30-day retention. |
+
+The live systemd unit + nginx site are the **AMI templates**
+(`deploy/ami/templates/open-swe.service`, `open-swe.nginx.conf`), rendered at
+first boot by `deploy/ami/user-data.sh`. The AMI is the baked
+`open-swe-base-arm64` image (Ubuntu 24.04 + uv/py3.12 + nginx + CW agent), pinned
+by exact id in `infra/lib/constructs/ami-cache.ts`. There is intentionally **no**
+on-box swapfile — the OOM-prone SPA build now runs in CI, not on the box.
+
+> `deploy/seahaven/{nginx/openswe.conf,systemd/open-swe.service}` are the
+> **retired on-prem VM** variants (run as `adam` from a home dir, bound `0.0.0.0`,
+> Postgres-backed). They are kept only for on-prem-contrast reference and are not
+> used by the AWS deployment.
+
+### Models
+Model selection is **store-driven**, not env. The `team_settings/default` store
+doc wins (then per-user profile, then per-thread); `LLM_MODEL_ID` is only a
+seed-time fallback. Defaults seeded by `seed_store.sh`:
+- builder: `anthropic:claude-opus-4-8` (effort `high`)
+- reviewer (cross-family): `openai:gpt-5.5` (effort `high`) — `openai:gpt-4.1` is
+  **not** in this fork's `SUPPORTED_MODELS` (`agent/dashboard/options.py`); add it
+  there first if you need 4.1.
+- the `analyzer` graph is hardcoded to the code default and ignores team settings.

 ## Triggering

-Start a task by mentioning **`@openswe`** in a GitHub issue comment (the documented
-intake — the `open-swe` *label* path needs the `Issues` event subscription). The
-commenter must have a `user_mappings` entry or the run is skipped.
+Mention **`@openswe`** (or `@open-swe` / `@seahaven-openswe`) in a GitHub issue or
+PR comment, a Linear comment, or a Slack thread. The commenter must have a
+`user_mappings` entry (seeded by `seed_store.sh` from `CONFIGURED_ADMINS` /
+`SEED_USER_MAPPINGS`) or the run is skipped.

-## AWS variant — env-sourced config (.env from Secrets Manager + SSM)
+Live integration endpoints (set in each provider's app config):

-On the AWS lift-and-shift, the `.env` is **not** staged by hand. A boot hook pulls
-config from AWS via the EC2 instance role and materializes a root-only `.env` on a
-tmpfs before the service starts. Same stock runtime; only how `.env` is produced
-changes.
-
-| File | Role |
+| Integration | URL |
 |---|---|
-| `fetch-config.sh <dev\|prod>` | `ExecStartPre=+` hook. Reads all SSM params `/open-swe-<env>/*` + all secrets `open-swe-<env>/*`, FAIL-FAST on any missing required var, writes `/run/open-swe/.env` (tmpfs, root:root 0600), symlinks `<APP_DIR>/.env` → it. Forces `DEFAULT_REPO_OWNER`=`Sea-Haven-Industries` (never upstream `langchain-ai`). |
-| `seed_store.sh <dev\|prod>` | `ExecStartPost` hook (unchanged role). Now sources the materialized `.env`, so `default_repo`/models/user-mappings come from AWS config instead of hardcoded `OPENSWE_*`. Still re-seeds the in-memory store on **every** restart. |
-| `ROTATION.md` | How to rotate any secret/param (update AWS → `systemctl restart`) + per-secret notes (`TOKEN_ENCRYPTION_KEY` overlap list, GitHub App PEM, webhook signing secrets). |
+| GitHub webhook | `https://hooks.seahaven.com/webhooks/github` |
+| Slack events | `https://hooks.seahaven.com/webhooks/slack` (+ `/webhooks/slack/interactivity`) |
+| Linear webhook | `https://hooks.seahaven.com/webhooks/linear` |
+| GitHub OAuth callback | `https://openswe.seahaven.com/dashboard/api/auth/callback` |

-Because the `.env` is root-only, the AWS `open-swe.service` runs **as root** (the
-on-prem unit's `User=adam` cannot read it). See the header of `fetch-config.sh` for
-the exact unit snippet and the tmpfs / `RUN_DEDICATED_TMPFS` options. Naming/secret
-placement follows the T9 inventory: 29 secrets → Secrets Manager `open-swe-<env>/*`,
-53 config → SSM `/open-swe-<env>/*`, each keyed by the literal env-var name.
+---
+
+## Troubleshooting
+
+**RETAIN secret-shell orphan on stack re-create.** The Secrets Manager shells use
+`DeletionPolicy: Retain` + a fixed `open-swe-<env>/<VAR>` name. If a stack's first
+create rolls back (or on a teardown/rebuild, a secret logical-id refactor, or
+standing up a new env), the empty shells survive and keep their global names, so
+every later create fails `AlreadyExists` — and a plain `delete-secret` does not
+free the name (it stays reserved for the 7–30 day recovery window). Before
+re-creating the stack, **force-delete the empty orphans** (only shells with no
+value version — never a populated secret). Hit on prod 2026-06-29 (PR #51 deploy
+failure). Full recovery command + rationale:
+[`infra/README.md`](../../infra/README.md) (PR #52).
+
+**`langgraph dev` won't start after a deploy.** `fetch-config.sh` fail-fasts on a
+missing/empty required var and prints the offending variable **names** (never
+values) to the unit journal. Confirm the 13 prod boot-required vars are populated
+(`put-config.sh prod`), then `systemctl restart open-swe.service`.
+
+**ALB target unhealthy.** The TG health check is `GET /healthz` on nginx `:80`
+(static 200). nginx starts before the app on first boot, so an unhealthy target
+usually means the box can't reach the ALB SG on `:80` (the standalone ALB-egress
+rule) rather than an app fault.
+
+---

 ## Aegra (deferred)

-`aegra/aegra.json` + `aegra/aegra_entry.py` are the self-hosted-runtime alternative
-(Apache-2.0, avoids the LangGraph-Platform Elastic license). Not active on the stock
-deployment. To use: place both at the repo root, run `aegra serve` (:2026), and point
-`LANGGRAPH_URL` at `:2026`. Aegra gives a Postgres-backed durable store/checkpointer,
-which removes the need for `seed_store.sh` and survives restarts (paused HITL
-interrupts persist).
+`aegra/aegra.json` + `aegra/aegra_entry.py` are the self-hosted-runtime
+alternative (Apache-2.0, avoids the LangGraph-Platform Elastic license). Not
+active on the stock deployment. To use: place both at the repo root, run
+`aegra serve` (:2026), and point `LANGGRAPH_URL` at `:2026`. Aegra gives a
+Postgres-backed durable store/checkpointer, which removes the need for
+`seed_store.sh` and survives restarts (paused HITL interrupts persist).
+</content>
--- a/deploy/seahaven/nginx/openswe.conf
+++ b/deploy/seahaven/nginx/openswe.conf
@ -1,10 +1,14 @@
 # Open SWE dashboard frontend (TanStack Start SPA) + scoped API proxy.
+# RETIRED on-prem VM variant — kept for on-prem-contrast reference only. The LIVE
+# AWS nginx site is the AMI template deploy/ami/templates/open-swe.nginx.conf
+# (rendered from an @@SERVER_NAME@@ token at first boot). See DEPLOYMENT.md.
+#
 # nginx is the security boundary: ONLY /dashboard/api/* reaches the backend;
 # the unauthenticated LangGraph agent API (/threads,/runs,/assistants,/store) is NOT proxied.
 server {
    listen 80 default_server;
    listen [::]:80 default_server;
-    server_name openswe.seahavenind.com;
+    server_name openswe.seahaven.com;

    root /var/www/openswe;
    index _shell.html;