open-swe/deploy/seahaven/ROTATION.md

Sea Haven — Open SWE secret & config rotation

How secrets and config reach the running app, and how to rotate either one.

How values flow at boot

AWS Secrets Manager  open-swe-<env>/*  ─┐
AWS SSM Param Store   /open-swe-<env>/* ─┤── fetch-config.sh ──▶ tmpfs /run/open-swe/.env (root:root 0600)
                                         │   (systemd ExecStartPre=+, EC2 role)        │
                                         ▼                                             ▼
                                    FAIL-FAST if a                         app symlink <APP_DIR>/.env
                                    required var is empty                  python-dotenv reads at import

The app reads .env once, at import. There is no hot-reload of secrets. Therefore the rotation contract is always the same two steps:

Rotation = (1) update the value in Secrets Manager / SSM, then (2) restart the service so fetch-config.sh re-materializes the .env.

# after updating a secret/param in AWS:
sudo systemctl restart open-swe.service
#   ExecStartPre=+  -> fetch-config.sh re-pulls + rewrites the tmpfs .env (fail-fast)
#   ExecStartPost   -> seed_store.sh re-seeds the in-memory store (team_settings + user_mappings)

There is no zero-downtime path for most secrets on the stock in-memory runtime — a restart is required and it also wipes the in-memory store (re-seeded by seed_store.sh automatically). The one secret built for zero-downtime overlap is TOKEN_ENCRYPTION_KEY (see below), but even it needs the restart to load the new key list.

Rotating a secret (Secrets Manager)

ENV=prod            # or dev
NAME=DASHBOARD_JWT_SECRET
aws secretsmanager put-secret-value \
  --secret-id "open-swe-${ENV}/${NAME}" \
  --secret-string 'NEW_VALUE' \
  --region us-east-1
sudo systemctl restart open-swe.service        # on the box

(update-secret/put-secret-value both create a new version; the boot hook always reads AWSCURRENT.)

Rotating a config param (SSM)

aws ssm put-parameter --overwrite \
  --name "/open-swe-${ENV}/DASHBOARD_BASE_URL" \
  --type String --value 'https://openswe.seahaven.com' \
  --region us-east-1
sudo systemctl restart open-swe.service

Per-secret rotation notes

Secret	Rotation notes
TOKEN_ENCRYPTION_KEY	Fernet key(s). Supports a comma/newline-separated list (agent/encryption.py) for zero-downtime key rotation: prepend the NEW key, keep the OLD key(s) in the list. New data is encrypted with the first key; old data still decrypts with the trailing keys. After all encrypted-at-rest tokens (per-user GitHub OAuth tokens in thread metadata) have been re-encrypted/expired, drop the old key. Store the list as one secret value; fetch-config.sh writes it verbatim. Never rotate to a single new key in one step or every existing encrypted token becomes undecryptable.
GITHUB_APP_PRIVATE_KEY	Multiline PEM. Generate a new private key in the GitHub App settings (you may have two active keys during overlap), put the new PEM into the secret, restart, verify install-token minting + a webhook delivery, then delete the old key in GitHub. fetch-config.sh writes the PEM as a double-quoted multiline value (python-dotenv-safe); paste the full -----BEGIN…-----END----- block including newlines.
GITHUB_WEBHOOK_SECRET	Webhook HMAC. GitHub allows only one webhook secret per App, so this is a brief-break rotation: update the secret in AWS and the GitHub App webhook config, restart. Deliveries signed with the old secret during the gap will 401 (GitHub auto-redelivers). Required in prod (fail-fast).
SLACK_SIGNING_SECRET	Slack request-signature secret. Rotate in the Slack app config and AWS together, restart. Required in prod (fail-fast). A stale value silently 401s url_verification/events until restart (known gotcha).
LINEAR_WEBHOOK_SECRET	Linear webhook signature. Required in prod only when the Linear integration is wired (LINEAR_API_KEY present). Rotate in Linear + AWS together, restart.
SLACK_CLIENT_SECRET / GITHUB_APP_CLIENT_SECRET	OAuth client secrets (dashboard login / Slack OAuth). Rotate in the provider console + AWS, restart. Existing dashboard sessions are JWT-signed by DASHBOARD_JWT_SECRET, not these, so they survive.
DASHBOARD_JWT_SECRET	Signs dashboard session cookies. Rotating invalidates all active sessions (users re-login). Hard-required (RuntimeError if empty). No overlap list — single value.
Model provider keys (ANTHROPIC_API_KEY, OPENAI_API_KEY, GOOGLE_API_KEY, GROQ_API_KEY, FIREWORKS_API_KEY)	Standard API-key rotation: issue new key, update AWS, restart, revoke old. The active provider keys (whatever team_settings/default seeds — currently ANTHROPIC_API_KEY + OPENAI_API_KEY) are fail-fast-required; keep REQUIRED_PROVIDER_KEYS in sync if you change the seeded models.
LANGSMITH_API_KEY_PROD	LangSmith key powering the langsmith sandbox (the only provider with working in-sandbox git/gh auth). Required when SANDBOX_TYPE=langsmith (fail-fast). Rotate in LangSmith + AWS, restart; existing sandboxes keep their already-injected proxy token until recycled.
SLACK_BOT_TOKEN / LINEAR_API_KEY / GITHUB_PAT / EXA_API_KEY / DAYTONA_API_KEY / RUNLOOP_API_KEY / CORRIDOR_ / USER_ID_API_KEY_MAP / X_SERVICE_AUTH_JWT_SECRET*	Plain API-token rotation: update AWS, restart, revoke old at the provider. None support an overlap list.

Fail-fast safety

fetch-config.sh refuses to write the .env (exit 1) if any required var is empty after a rotation — so a botched rotation (e.g. an empty put-secret-value) stops the service at ExecStartPre instead of starting it with a partial .env. The missing variable names are printed to the journal (values never are):

sudo journalctl -u open-swe.service -b | grep fetch-config

Required set enforced: DASHBOARD_JWT_SECRET, TOKEN_ENCRYPTION_KEY, GITHUB_APP_ID, GITHUB_APP_PRIVATE_KEY, GITHUB_APP_INSTALLATION_ID, GITHUB_APP_CLIENT_ID, GITHUB_APP_CLIENT_SECRET, the active provider keys (REQUIRED_PROVIDER_KEYS, default ANTHROPIC_API_KEY,OPENAI_API_KEY), the sandbox key for SANDBOX_TYPE (langsmith ⇒ LANGSMITH_API_KEY_PROD + DEFAULT_SANDBOX_SNAPSHOT_ID), and — in prod — GITHUB_WEBHOOK_SECRET, SLACK_SIGNING_SECRET (plus LINEAR_WEBHOOK_SECRET when Linear is wired).