shoc-backend/terraform/live/README.md

206 lines
10 KiB
Markdown
Raw Normal View History

# Backend environment adoption
These roots adopt environment-owned infrastructure without taking ownership of
shared or Elastic Beanstalk-generated infrastructure.
## Ownership
- `dev/` and `staging/` import the existing EB environment, runtime
role/profile/policies, deploy role/policy, app-config secret metadata, and API
record.
- `modules/environment-inventory/` reads and pins only shared resources.
- Org-baseline CloudFormation owns the narrowly scoped HCP Terraform plan/apply
roles.
The shared `shoc-backend` Elastic Beanstalk application and
`shoc-sqlserver-shared` RDS instance, VPC, subnets, EB service role, shared
certificate, shared RDS security group, and EB-generated SG/ALB/ASG/CloudFormation
resources must never enter an environment state.
Secret values are not Terraform resources, variables, outputs, or managed EB
settings. Terraform manages the app-config secret shell and maps approved JSON
keys through `aws:elasticbeanstalk:application:environmentsecrets` using
`secret-arn:json-key` references. Ordinary application environment settings are
limited to non-secret ASP.NET, webhook, and Sentry configuration. The Sentry
DSN is public ingestion configuration, delivered as the plain `SENTRY_DSN`
setting from the required `sentry_dsn` variable. `SENTRY_ENVIRONMENT` is
`development` for the dev root and the environment name (`staging`) otherwise.
No production root exists yet; production Sentry wiring will follow the same
variable when one is added. The pinned .NET 8
AL2023 platform 3.11.3 supports Secrets Manager JSON-key extraction.
## Mandatory live secret migration
Before importing dev or staging, perform a separately approved production
mutation from a trusted local session:
1. Create a temporary `OptionSettings` JSON file containing the exact
`environmentsecrets` ARN/key references configured in that root and the five
non-secret ordinary environment settings.
2. Create a temporary `OptionsToRemove` JSON file naming the old raw
secret-valued keys in `aws:elasticbeanstalk:application:environment`.
3. Run `aws elasticbeanstalk update-environment` for exactly
`shoc-backend-dev` or `shoc-backend-staging` with
`--option-settings file://...` and `--options-to-remove file://...`.
Include the provider-normalized sorted `Subnets` and `ELBSubnets` values in
this same approved update if live ordering differs.
4. Delete both files, wait for the replacement environment to become Ready and
healthy, and run `scripts/smoke-elastic-beanstalk.sh` against the exact API.
5. Verify the raw ordinary secret settings are absent before generating the
first Terraform plan.
This migration is not performed by Terraform. The first import plan remains
zero-change only after the migration succeeds.
## Mandatory role-boundary attachment
After the org baseline creates the dedicated boundary policies, perform a
separately approved production IAM mutation that attaches:
- `shoc-backend-dev-runtime-boundary` to `shoc-backend-dev`
- `shoc-backend-staging-runtime-boundary` to `shoc-backend-staging`
- `shoc-backend-dev-deploy-boundary` to `githubdeploy-shoc-backend-dev`
- `shoc-backend-staging-deploy-boundary` to
`githubdeploy-shoc-backend-staging`
Attach all four boundaries before the SCP and HCP `PutRolePolicy` exceptions
become effective. In the same approved pre-import phase, add the immutable
`HcpTerraformWorkspace` tag to each GitHub deploy role:
- `githubdeploy-shoc-backend-dev`: `shoc-backend-dev`
- `githubdeploy-shoc-backend-staging`: `shoc-backend-staging`
Verify each exact runtime and deploy boundary ARN and workspace tag from
`GetRole` before importing. Terraform requires the boundaries to be present
during `adoption_complete=false`, so the first import remains zero-change.
Terraform does not perform this pre-import mutation.
## Two-phase adoption
Each dev/staging root pins `adoption_complete=false` in reviewed code until its
initial import is proven. It is not an HCP workspace variable.
1. Create the HCP workspace and configure dynamic credentials.
2. Run the declarative imports.
3. Export the HCP plan as JSON and run:
```bash
python scripts/check-terraform-import-plan.py plan.json --environment dev
```
The first plan must be a no-op after import. The guard rejects updates,
creates, deletes, replacements, and managed resource types outside the
approved environment-owned boundary.
4. Apply the no-op import only after review.
5. Change the environment root to `adoption_complete=true` and
`manage_eb_settings=true` in a reviewed code change, then review the
controlled in-place role metadata, secret metadata, and Elastic Beanstalk
update:
```bash
# Dev example. Omit any address that is not updating.
python scripts/check-terraform-import-plan.py plan.json --environment dev \
--allow-update-address module.environment.aws_iam_instance_profile.runtime \
--allow-update-address module.environment.aws_iam_role.runtime \
--allow-update-address module.environment.aws_iam_role.github_deploy \
--allow-update-address module.environment.aws_secretsmanager_secret.app_config \
--allow-update-address module.environment.aws_elastic_beanstalk_environment.this
```
6. Apply only when every update address is named on the command line and the
plan contains no create, delete, or replacement action. The dev direct ALB
alias remains pinned during this phase and must not update.
The dev deploy role and generated inline policy completed their retained
CloudFormation-to-Terraform transfer before the legacy backend CDK source was
removed. Do not reintroduce that ownership path.
The reviewed `adoption_complete=true` change updates ownership tags on IAM
roles, instance profiles, and app-config secrets. Elastic Beanstalk
environment tags remain at their imported values. Terraform manages the
declared EB settings. Secret values remain out-of-band even after the secret
shell receives `ManagedBy=terraform`. Deploy-role descriptions and immutable
`HcpTerraformWorkspace` tags remain unchanged. Read-only AWS APIs retain
`Resource = "*"` only where AWS does not support resource-level permissions.
The measured self-contained .NET/EF bundle is approximately 199.5 MB and
separate builds are not byte-identical. Each deploy job therefore validates the
exact bundle it uploads; bundle bytes never enter Terraform plans or state.
## Application CD
GitHub Actions owns application versions. It compiles the bundle, uploads it,
creates the Elastic Beanstalk application version, and calls
`UpdateEnvironment`. Terraform ignores `version_label` so those deploys are not
drift. If health, smoke, or the webhook probe fails after that update, the job
restores the previous Elastic Beanstalk version label. Database migrations
already applied by the failed bundle are not reverted. Deploy parameters are read from `/shoc-backend/<env>/deploy/*` SSM
parameters this module writes.
Merge to `main` deploys **dev** unless the push is terraform-only. Staging is
cut from **Actions → Release** (`environment`, `bump`, `message`). That
workflow waits for CI, tags `vX.Y.Z-staging` from main HEAD with
`GITHUB_TOKEN`, then calls deploy. Do not cut prod yet; leave
`PROD_APP_CD_ENABLED` unset and do not create the `prod` GitHub Environment.
Staging import is proven after the first GitHub-owned zip
(`v0.0.1-staging`). Terraform now manages the declared Elastic Beanstalk
settings. The API CNAME stays pinned to the imported ALB target.
HCP workspaces stay VCS-driven with auto-apply on after cutover. Speculative
plans on every PR are the infra gate. Do not point `TFC_AWS_*` at
`hcptf-bootstrap`. Org-baseline CloudFormation owns the HCP plan/apply roles.
GitHub Actions does not create, wait on, or apply HCP runs. Application and
Terraform changes stay in separate PRs so a merge cannot race an HCP apply
against an app deploy. Terraform-only merges skip `deploy.yaml`. App-only
tags skip HCP when trigger patterns do not match.
### Credentials
Store `DEPLOY_ROLE_ARN` as a GitHub Environment **variable** (`dev`,
`staging`). OIDC trust is
`repo:Sea-Haven-Industries/shoc-backend:environment:<env>` plus
`job_workflow_ref` for `.github/workflows/deploy.yaml` at `refs/heads/main`
and `refs/tags/v*`. Adding another deploy workflow is a cross-family IAM
change. The new CD path does not use `TF_API_TOKEN`.
GitHub Environment deployment branch and tag policies are repository
settings, not this diff. The policy matches `GITHUB_REF` of the workflow run.
Branch patterns never match tag refs; adding `v*` as a branch pattern fails
the same way as an empty allowlist.
1. `dev` — allow branch `main`.
2. `staging` — **tag-type** policy matching `v*.*.*-staging` for
`deploy-tag.yaml`. Allow branch `main` because Actions → Release is
`workflow_dispatch` on `main` and then calls `deploy.yaml`
(`GITHUB_TOKEN` tag pushes do not start `deploy-tag.yaml`).
Do not create the `prod` environment yet. Leave `PROD_APP_CD_ENABLED`
unset. Until the `prod` environment exists with reviewers, do not run
Actions → Release with `environment=prod`, do not push a bare `vX.Y.Z`
tag, and do not `workflow_dispatch` deploy with `environment=prod`. Any
of those declares `environment: prod` and would auto-create an
unprotected environment.
## Pinned live identities
- Dev: workspace `shoc-backend-dev`; EB environment `shoc-backend-dev`
(`e-hehnrqjjrt`); .NET 8 AL2023 `3.11.3`; `api.dev.seahaven.com`.
- Staging: workspace `shoc-backend-staging`; EB environment
`shoc-backend-staging` (`e-6c9m4vb62z`); .NET 8 AL2023 `3.11.3`;
`api.staging.seahaven.com`.
The environment roots are intentionally not general-purpose modules. Exact
identifiers make accidental cross-environment reuse fail review and planning.
## Safety invariants
- After cutover, auto-apply is on. Speculative PR plans stay on.
- `shoc-backend-dev` is branch-based on `main` with trigger patterns
`terraform/live/dev/**` and `terraform/live/modules/**`.
- `shoc-backend-staging` is tag-based on `^v\d+\.\d+\.\d+-staging$` with
trigger patterns `terraform/live/staging/**` and `terraform/live/modules/**`.
- Org baseline owns HCP plan/apply permissions and manager tags. Never manage
`hcptf-*` in this repository.
- Every imported Terraform resource has `prevent_destroy`.
- Elastic Beanstalk `version_label` is ignored so GitHub deploys are not drift.