diff --git a/deploy/MIGRATION.md b/deploy/MIGRATION.md index ebccfced..44da7137 100644 --- a/deploy/MIGRATION.md +++ b/deploy/MIGRATION.md @@ -1,7 +1,9 @@ # Open SWE — Migration Plan: Self-Hosted AWS → Managed LangGraph Cloud + Vercel **Repo:** `Sea-Haven-Industries/open-swe` (private) · **AWS:** 328440206208 / us-east-1 -**Author:** Adam Moussa · **Date:** 2026-06-29 · **Status:** DRAFT — owes a `/sh-plan-review` before prod cutover (Phase C gate) +**Author:** Adam Moussa · **Date:** 2026-06-29 (final topology added 2026-06-30) · **Status:** EXECUTED — managed cutover live; §§3–11 below are the original (now-historical) phased plan, **superseded by §1a for all current-state facts (URLs, project layout, env)**. + +> **READ §1a FIRST.** The phased plan (§§2–11) and the Phase A/C spike notes capture how we got here and still hold for rationale, cost, and rollback. But the spike-era specifics they cite — the single `open-swe-dashboard` Vercel project, the `open-swe-dev-hosted-…`/`open-swe-v3-…` deployment URLs, the `ui/vercel.json` same-origin rewrite, the single GitHub App — are **stale**. §1a is the authoritative final topology and wins on every conflict. --- @@ -30,6 +32,75 @@ A self-hosted `langgraph up` + RDS plan (already `/sh-plan-review`'d to APPROVE- --- +## 1a. Final, verified topology (AUTHORITATIVE — supersedes spike-era values) + +This is the live managed deployment as of 2026-06-30. Where any later section disagrees (old URLs, a single Vercel project, a single GitHub App, the `ui/vercel.json` rewrite), **this section wins**. + +### Backend — managed LangGraph Cloud (two deployments, one LangSmith workspace) +Both deployments live in the **same LangSmith workspace**; the **same workspace API key authenticates both** (including the Store API — so per-deployment store writes use that one key with the per-deployment URL). + +| Deployment | URL | Git connection | +|---|---|---| +| **dev** | `https://open-swe-dev-fb737aa219605c8bbdb30ecbb33f30c0.us.langgraph.app` | branch `dev` | +| **prod** | `https://open-swe-prod-d6c7bb63aaa651b6a1d92f9492b1d983.us.langgraph.app` | branch `main` (auto-deploys on push to `main`) | + +> The dev deployment was **renamed `open-swe-dev`, deleted, and recreated** — which minted the **new URL hash** above. The spike-era `open-swe-v3-…` / `open-swe-dev-hosted-…` URLs are **dead/superseded**. Deleting + recreating a deployment is the one operation that changes the URL hash (otherwise stable across revisions) — when it happens, update **every** reference (Vercel env, GitHub App webhooks, OAuth callbacks, docs). + +### UI — Vercel (ONE project, two environments) +**One** Vercel project `open-swe-prod` (team `sea-haven`, id `prj_OOh6yjXMp4ah3Ws3Y7XRQxjmMmQU`). The old separate `open-swe-dashboard` project was **DELETED**. + +| Vercel environment | Branch | Backend | Custom domain | +|---|---|---|---| +| production | `main` | prod deployment URL | `openswe.seahaven.com` | +| custom **`dev`** (id `env_SMI23PULAJXk0GhwE0HLVhp5J3ZS`) | `dev` | dev deployment URL | `openswe-dev.seahaven.com` | + +- A **per-environment** env var `LANGGRAPH_BACKEND_URL` (prod env = prod URL, dev env = dev URL) drives the `/dashboard/api/*` proxy. +- Project settings: `framework=null`, `outputDirectory` cleared, root directory `ui`. +- **Proxy mechanism (current, after PR #76):** **Nitro `routeRules`** in `ui/vite.config.ts` read `process.env.LANGGRAPH_BACKEND_URL` and Nitro's Vercel preset compiles them into `.vercel/output/config.json` (Build Output API) at build time — a CDN-level proxy (not redirect, so the `osw_session` cookie stays first-party). PR #75's hand-rolled `ui/scripts/build-vercel-output.mjs` was the broken first attempt and is **gone** (`ui/scripts/` no longer exists). **Never** add a manual script that `rm`s `.vercel/output` — Nitro's Vercel preset auto-emits it. + +### DNS — Route 53 zone `seahaven.com` (`Z06652411XKH89KTZD3XA`) +- `openswe.seahaven.com` → CNAME `cname.vercel-dns.com` (prod env) +- `openswe-dev.seahaven.com` → CNAME to Vercel (dev env) + +### GitHub Apps — TWO (dev/prod isolated; each its own webhook URL) +| App | app_id | install | client_id | org | members scope | repos | +|---|---|---|---|---|---|---| +| **prod** `seahaven-openswe` | `4146115` | `142615168` | `Iv23lil96pKQNNDUn5yp` | `Sea-Haven-Industries` | members:**write** | all | +| **dev** `seahaven-openswe-dev` | `4162963` | `143023302` | `Iv23licQwJvGAPJj1HJe` | `seahaven-open-swe-dev` | members:**read** | all | + +- **Promotion App** `seahaven-promotion` (actor `4170147`) is the sole non-admin fast-forward-push bypass on the `main` ruleset `18238334` — its FF-push of `dev → main` is what triggers the managed prod build. + +### Env per deployment (set in LangGraph Cloud config + Vercel env — NOT Secrets Manager) +| Var | dev | prod | +|---|---|---| +| `LANGGRAPH_URL` | own (dev) deployment URL | own (prod) deployment URL | +| `DASHBOARD_BASE_URL` / `DASHBOARD_API_BASE_URL` | `https://openswe-dev.seahaven.com` | `https://openswe.seahaven.com` | +| `VITE_DASHBOARD_API_BASE_URL` | empty (same-origin via Vercel proxy) | empty | +| `ALLOWED_GITHUB_ORGS` | dev org (`seahaven-open-swe-dev`) | `Sea-Haven-Industries` | +| `CONFIGURED_ADMINS` | `amoussa1229,adam@seahavenind.com` | `amoussa1229,adam@seahavenind.com` | + +`DASHBOARD_BASE_URL` / `DASHBOARD_API_BASE_URL` **must include `https://`** (see gotcha 3). Secret **values** are still sourced from `open-swe-{dev,prod}/*` Secrets Manager + SSM (the remaining source of truth) and set into the LangGraph Cloud + Vercel env stores — the accepted `secrets-and-config.md` deviation. + +### Bedrock IAM (PR #74, still OPEN) +Two IAM users `open-swe-dev-bedrock` + `open-swe-prod-bedrock`, each attached to customer-managed policy `open-swe-bedrock-invoke` (least-privilege `bedrock:InvokeModel[WithResponseStream]` on the `us.anthropic.claude-opus-4-8` inference-profile ARN + its 3 routed foundation-model ARNs in us-east-1/us-east-2/us-west-2). Default model `bedrock_converse:us.anthropic.claude-opus-4-8` + 3 Fireworks models. Static access keys live only in the deployment env (dev key → dev, prod key → prod). + +### User store — per-deployment +**Each** managed deployment has its **own** Store. The GitHub→email mapping `amoussa1229 → adam@seahavenind.com` (namespace `["user_mappings"]`, key = lowercased login, record `{github_login, work_email, status:"active", source, created_at, updated_at}`) was written to **both** the dev and prod stores directly. New users need a mapping **per-deployment** (write each store directly, or use the dashboard admin User-mappings UI — the `work_email` field was added by PR #65 fix #4). + +### AWS decommission (PR #64) +Self-host CDK stacks destroyed. Residual: `CDKToolkit` (shared, **preserved**); ~50 `RETAIN`'d Secrets Manager shells + 3 S3 asset buckets (**pending cleanup**); AWS **Bedrock** (live dependency, kept). + +### Operational gotchas (hard-won — carry these into any runbook) +1. **Per-deployment store → seed user mappings per-deployment.** A missing mapping makes `process_github_issue` silently early-return ("No email mapping … skipping"): the webhook returns 200/accepted but produces **no reaction and no run**. Seed dev **and** prod. +2. **Org-login gate uses the App *installation* token**, so the App must be org-installed with **Members:read**. OAuth working ≠ membership check working — they use **separate creds** (CLIENT_ID/SECRET for OAuth vs APP_ID/INSTALLATION_ID/PRIVATE_KEY for the install token). A mangled multi-line `GITHUB_APP_PRIVATE_KEY` breaks the install token (and thus the gate) while OAuth still works. +3. **`DASHBOARD_API_BASE_URL` must be `https://`** — an `http://` value makes GitHub reject the OAuth callback with "redirect_uri not associated." +4. **`osw_oauth_state` cookie is host-only** — start login on the **same host** as `DASHBOARD_API_BASE_URL`, or you get "oauth state mismatch." +5. **Webhooks go DIRECT to the langgraph URL** (`/webhooks/*`). Vercel only proxies `/dashboard/api/*`. The app is **same-origin only** (no CORS). +6. **On Vercel CI, Nitro's Vercel preset auto-emits `.vercel/output`** — drive the proxy via Nitro `routeRules` from `LANGGRAPH_BACKEND_URL`; never a manual script that `rm`s `.vercel/output`. +7. **Deleting + recreating a LangGraph deployment mints a NEW URL hash** (otherwise stable across revisions) — update every reference (Vercel env, webhooks, OAuth, docs). + +--- + ## 2. Architecture: Before → After ### Before (self-hosted AWS — LIVE as of 2026-06-29) @@ -49,23 +120,29 @@ Browser (dashboard) ─────────────▶ openswe.seahaven. Sandbox: LangSmith cloud (DEFAULT_SANDBOX_SNAPSHOT_ID + GitHub proxy) ``` -### After (managed) +### After (managed — FINAL, see §1a for exact values) ``` -GitHub/Slack/Linear ──webhook──▶ *.langgraph.app (or hooks.seahaven.com CNAME → TODO §10) -Browser (dashboard) ─────────────▶ open-swe-dashboard.vercel.app (stable alias / custom domain) - │ same-origin rewrite /dashboard/api/* (ui/vercel.json) + ┌──────────────── DEV lane ────────────────┐ ┌──────────────── PROD lane ───────────────┐ +GitHub(dev org)/Slack/Linear │ webhook → open-swe-dev-….us.langgraph.app │ │ webhook → open-swe-prod-….us.langgraph.app│ GitHub(SHI org)/Slack/Linear + App seahaven-openswe-dev ───┘ (DIRECT to langgraph URL, /webhooks/*) │ │ (DIRECT to langgraph URL, /webhooks/*) └─── App seahaven-openswe + ▼ ▼ +Browser ▶ openswe-dev.seahaven.com ─┐ ┌─▶ openswe.seahaven.com ◀ Browser + │ ONE Vercel project `open-swe-prod` (team sea-haven) + │ ├─ env `dev` (branch dev) → proxies /dashboard/api/* → dev langgraph URL + │ └─ env production (branch main) → proxies /dashboard/api/* → prod langgraph URL + └─ proxy compiled by Nitro routeRules from per-env LANGGRAPH_BACKEND_URL (PR #76) ▼ - LangGraph Cloud "Deployment" (managed, git-connected to `main` for prod / `dev` for dev) - ├─ serves the 6 graphs (agent, reviewer, analyzer, chat, scheduler, ci_monitor) - ├─ serves the custom http.app (agent.webapp:app = webhooks + dashboard API + OAuth) - ├─ durable Postgres store + checkpointer (issue #9 SOLVED) — autoscaled 1→10 replicas - └─ env/secrets in the Deployment config (NOT Secrets Manager) — Adam accepted deviation + TWO LangGraph Cloud deployments (same LangSmith workspace; one workspace key auths both incl. Store) + ├─ dev ← branch `dev` · prod ← branch `main` (push-to-main auto-deploys prod) + ├─ each serves the graphs + the custom http.app (agent.webapp:app = webhooks + dashboard API + OAuth) + ├─ each has its OWN durable Postgres store + checkpointer (issue #9 SOLVED) — user mappings per-deployment + └─ env/secrets in the Deployment + Vercel config (NOT Secrets Manager) — Adam accepted deviation ▼ Sandbox: LangSmith cloud (UNCHANGED — DEFAULT_SANDBOX_SNAPSHOT_ID + GitHub-App proxy) - Auth: GitHub App seahaven-openswe (UNCHANGED — App 4146115 / Install 142615168) + Bedrock: IAM users open-swe-{dev,prod}-bedrock + policy open-swe-bedrock-invoke (static keys in deploy env) ``` -**What changes shape:** runtime host (EC2 → managed PaaS), durability (in-memory → managed Postgres), CD (bespoke S3/SSM/packer → git-connected auto-build), config home (Secrets Manager/SSM → Deployment+Vercel env). **What stays:** the GitHub App, the LangSmith sandbox plane, CI (lint/format/unit/Playwright), the app code itself. +**What changes shape:** runtime host (EC2 → managed PaaS), durability (in-memory → managed Postgres, one store **per deployment**), CD (bespoke S3/SSM/packer → git-connected auto-build), config home (Secrets Manager/SSM → Deployment+Vercel env), ingress topology (shared ALB + one App → two dev/prod-isolated GitHub Apps each hitting its own `*.langgraph.app` directly), UI proxy (`ui/vercel.json` rewrite → Nitro `routeRules`). **What stays:** the LangSmith sandbox plane, CI (lint/format/unit/Playwright), the app code itself, and Secrets Manager/SSM as the secret-**value** source of truth. --- @@ -74,11 +151,11 @@ Browser (dashboard) ─────────────▶ open-swe-dashboar ### Phase A — Dev spike (MOSTLY DONE) Goal: prove managed serves our custom app + durability, at $0, before committing prod $. -**Proven this session:** +**Proven this session** (spike-era specifics — superseded by §1a; URLs/project below are DEAD): - ✅ Dev backend deployed to LangGraph Cloud, connected to branch `dev`: - `https://open-swe-dev-hosted-e76c2b0e8a7955fe8ad3110a7a54e5d0.us.langgraph.app` + ~~`https://open-swe-dev-hosted-e76c2b0e8a7955fe8ad3110a7a54e5d0.us.langgraph.app`~~ → final dev URL in §1a (`open-swe-dev-fb737aa…`; the deployment was later deleted + recreated) (Bedrock + a Fireworks key set; AWS creds for Bedrock deferred — see §10 open decision). -- ✅ UI deployed to Vercel — team `sea-haven`, project `open-swe-dashboard`, `https://open-swe-dashboard.vercel.app`; `ui/vercel.json` rewrite repointed at the dev deployment. +- ✅ UI deployed to Vercel — team `sea-haven`, ~~project `open-swe-dashboard`, `https://open-swe-dashboard.vercel.app`~~ (DELETED); now the **single** project `open-swe-prod` with dev/prod environments (§1a); proxy is Nitro `routeRules`, not the `ui/vercel.json` rewrite. - ✅ Managed serves the custom `http.app` (dashboard API + webhooks) — **no platform auth gate** in front of our routes (webhook returns 401 sig-enforced, so signatures still govern). - ✅ Vercel same-origin rewrite → backend works. - ✅ GitHub OAuth dashboard login end-to-end. @@ -102,7 +179,7 @@ Land the 6 code fixes (§5), codify env/config, and resolve the Bedrock-auth dec ### Phase C — Prod deployment - [ ] C1. Create a **prod LangGraph Cloud deployment** tracking branch `main` (the durable autoscaled 1→10 tier, not the free Dev tier). Record its `*.langgraph.app` URL. -- [ ] C2. Create the **prod Vercel project/target** (or promote the existing `open-swe-dashboard` to production); set its env (same-origin mode: `VITE_DASHBOARD_API_BASE_URL` empty). +- [x] C2. ~~Create the **prod Vercel project/target** (or promote the existing `open-swe-dashboard` to production)~~ — **DONE differently:** one project `open-swe-prod` with a production env (`main`) + a custom `dev` env (`dev`), each with its own `LANGGRAPH_BACKEND_URL`. See §1a. - [ ] C3. Set the **prod env triad** (§4) on the prod deployment + Vercel: - `LANGGRAPH_URL` = the prod `*.langgraph.app` URL - `DASHBOARD_BASE_URL` + `DASHBOARD_API_BASE_URL` = the prod Vercel origin (with `https://` scheme — fix #2) @@ -139,15 +216,19 @@ Only after managed prod is proven + soaked. See §7 for the precise retire-vs-ke ## 4. Env / Config Reference -### The prod triad (per INSTALLATION.md §10, lines 630–658) -| Var | Prod value | Notes | +### The prod triad (per INSTALLATION.md §10, lines 630–658) — see §1a for the exact dev/prod values +| Var | Value (per deployment/env) | Notes | |---|---|---| -| `LANGGRAPH_URL` | the deployment URL (`https://...langgraph.app`) | **NOT** localhost. Drives `thread_ops.langgraph_url()` (fix #1). | -| `DASHBOARD_BASE_URL` | the Vercel origin | same-origin rewrite mode | -| `DASHBOARD_API_BASE_URL` | the Vercel origin, **with `https://` scheme** | scheme required or OAuth `redirect_uri` is schemeless and GitHub rejects (fix #2) | -| `VITE_DASHBOARD_API_BASE_URL` | **empty** | same-origin mode; UI calls relative `/dashboard/api/*`, Vercel rewrites to backend | +| `LANGGRAPH_URL` | the **own** deployment URL (`https://...langgraph.app`) | **NOT** localhost. Drives `thread_ops.langgraph_url()` (fix #1). dev→dev URL, prod→prod URL. | +| `DASHBOARD_BASE_URL` | the own dashboard origin, **with `https://`** (`https://openswe-dev.seahaven.com` / `https://openswe.seahaven.com`) | | +| `DASHBOARD_API_BASE_URL` | same as above, **with `https://` scheme** | scheme required or OAuth `redirect_uri` is schemeless and GitHub rejects (fix #2) | +| `VITE_DASHBOARD_API_BASE_URL` | **empty** | same-origin mode; UI calls relative `/dashboard/api/*`, the Vercel Nitro proxy rewrites to backend | +| `LANGGRAPH_BACKEND_URL` | **Vercel env var**, per Vercel environment (dev env = dev URL, prod env = prod URL) | drives the Nitro `routeRules` `/dashboard/api/*` proxy at build time (PR #76) — required on Vercel builds | +| `ALLOWED_GITHUB_ORGS` | `Sea-Haven-Industries` (prod) / `seahaven-open-swe-dev` (dev) | org-login gate; checked via the App **installation** token (gotcha 2) | -GitHub App dashboard OAuth callback = `/dashboard/api/auth/callback` (the Vercel prod origin). +GitHub App dashboard OAuth callback = `/dashboard/api/auth/callback` (the own dashboard origin — prod App on `openswe.seahaven.com`, dev App on `openswe-dev.seahaven.com`). + +**UI proxy mechanism (final, PR #76):** the `/dashboard/api/*` proxy is **Nitro `routeRules`** in `ui/vite.config.ts` reading `process.env.LANGGRAPH_BACKEND_URL`, compiled by Nitro's Vercel preset into `.vercel/output/config.json`. This **supersedes** the spike-era `ui/vercel.json` same-origin rewrite and PR #75's hand-rolled `ui/scripts/build-vercel-output.mjs` (deleted). `ui/vercel.json` now only carries `framework:null` + `buildCommand: bun run build`. ### Where secrets/env live now **LangGraph Cloud Deployment config + Vercel env** — NOT AWS Secrets Manager. This **deviates from the Sea Haven `secrets-and-config.md` "Secrets Manager for all sensitive" handbook rule** — **Adam ACCEPTED this deviation** (managed has no instance role / no fetch-config boot hook; the platform's own secret store is the mechanism).