shoc-backend/terraform/live/README.md

205 lines
10 KiB
Markdown
Raw Normal View History

# Backend environment adoption
These roots adopt environment-owned infrastructure without taking ownership of
shared or Elastic Beanstalk-generated infrastructure.
## Ownership
- `dev/` and `staging/` import the existing EB environment, runtime
role/profile/policies, deploy role/policy, app-config secret metadata, and API
record.
- `modules/environment-inventory/` reads and pins only shared resources.
- Org-baseline CloudFormation owns the narrowly scoped HCP Terraform plan/apply
roles.
The shared `shoc-backend` Elastic Beanstalk application and
`shoc-sqlserver-shared` RDS instance, VPC, subnets, EB service role, shared
certificate, shared RDS security group, and EB-generated SG/ALB/ASG/CloudFormation
resources must never enter an environment state.
Secret values are not Terraform resources, variables, outputs, or managed EB
settings. Terraform manages the app-config secret shell and maps approved JSON
keys through `aws:elasticbeanstalk:application:environmentsecrets` using
`secret-arn:json-key` references. Ordinary application environment settings are
limited to non-secret ASP.NET, webhook, and Sentry configuration. The Sentry
DSN is public ingestion configuration, delivered as the plain `SENTRY_DSN`
setting from the required `sentry_dsn` variable. `SENTRY_ENVIRONMENT` is
`development` for the dev root and the environment name (`staging`) otherwise.
No production root exists yet; production Sentry wiring will follow the same
variable when one is added. The pinned .NET 8
AL2023 platform 3.11.3 supports Secrets Manager JSON-key extraction.
## Mandatory live secret migration
Before importing dev or staging, perform a separately approved production
mutation from a trusted local session:
1. Create a temporary `OptionSettings` JSON file containing the exact
`environmentsecrets` ARN/key references configured in that root and the five
non-secret ordinary environment settings.
2. Create a temporary `OptionsToRemove` JSON file naming the old raw
secret-valued keys in `aws:elasticbeanstalk:application:environment`.
3. Run `aws elasticbeanstalk update-environment` for exactly
`shoc-backend-dev` or `shoc-backend-staging` with
`--option-settings file://...` and `--options-to-remove file://...`.
Include the provider-normalized sorted `Subnets` and `ELBSubnets` values in
this same approved update if live ordering differs.
4. Delete both files, wait for the replacement environment to become Ready and
healthy, and run `scripts/smoke-elastic-beanstalk.sh` against the exact API.
5. Verify the raw ordinary secret settings are absent before generating the
first Terraform plan.
This migration is not performed by Terraform. The first import plan remains
zero-change only after the migration succeeds.
## Mandatory role-boundary attachment
After the org baseline creates the dedicated boundary policies, perform a
separately approved production IAM mutation that attaches:
- `shoc-backend-dev-runtime-boundary` to `shoc-backend-dev`
- `shoc-backend-staging-runtime-boundary` to `shoc-backend-staging`
- `shoc-backend-dev-deploy-boundary` to `githubdeploy-shoc-backend-dev`
- `shoc-backend-staging-deploy-boundary` to
`githubdeploy-shoc-backend-staging`
Attach all four boundaries before the SCP and HCP `PutRolePolicy` exceptions
become effective. In the same approved pre-import phase, add the immutable
`HcpTerraformWorkspace` tag to each GitHub deploy role:
- `githubdeploy-shoc-backend-dev`: `shoc-backend-dev`
- `githubdeploy-shoc-backend-staging`: `shoc-backend-staging`
Verify each exact runtime and deploy boundary ARN and workspace tag from
`GetRole` before importing. Terraform requires the boundaries to be present
during `adoption_complete=false`, so the first import remains zero-change.
Terraform does not perform this pre-import mutation.
## Two-phase adoption
Each dev/staging root pins `adoption_complete=false` in reviewed code until its
initial import is proven. It is not an HCP workspace variable.
1. Create the HCP workspace and configure dynamic credentials.
2. Run the declarative imports.
3. Export the HCP plan as JSON and run:
```bash
python scripts/check-terraform-import-plan.py plan.json --environment dev
```
The first plan must be a no-op after import. The guard rejects updates,
creates, deletes, replacements, and managed resource types outside the
approved environment-owned boundary.
4. Apply the no-op import only after review.
5. Change the environment root to `adoption_complete=true` and
`manage_eb_settings=true` in a reviewed code change, then review the
controlled in-place role metadata, secret metadata, and Elastic Beanstalk
update:
```bash
# Dev example. Omit any address that is not updating.
python scripts/check-terraform-import-plan.py plan.json --environment dev \
--allow-update-address module.environment.aws_iam_instance_profile.runtime \
--allow-update-address module.environment.aws_iam_role.runtime \
--allow-update-address module.environment.aws_iam_role.github_deploy \
--allow-update-address module.environment.aws_secretsmanager_secret.app_config \
--allow-update-address module.environment.aws_elastic_beanstalk_environment.this
```
6. Apply only when every update address is named on the command line and the
plan contains no create, delete, or replacement action. The dev direct ALB
alias remains pinned during this phase and must not update.
The dev deploy role and generated inline policy completed their retained
CloudFormation-to-Terraform transfer before the legacy backend CDK source was
removed. Do not reintroduce that ownership path.
The reviewed `adoption_complete=true` change updates ownership tags on IAM
roles, instance profiles, and app-config secrets. Elastic Beanstalk
environment tags remain at their imported values. Terraform manages the
declared EB settings. Secret values remain out-of-band even after the secret
shell receives `ManagedBy=terraform`. Deploy-role descriptions and immutable
`HcpTerraformWorkspace` tags remain unchanged. Read-only AWS APIs retain
`Resource = "*"` only where AWS does not support resource-level permissions.
The measured self-contained .NET/EF bundle is approximately 199.5 MB and
separate builds are not byte-identical. Each deploy job therefore validates the
exact bundle it uploads; bundle bytes never enter Terraform plans or state.
## Dev application CD
GitHub compiles, validates, and uploads the bundle, then creates the immutable
Elastic Beanstalk application version. HCP Terraform is the only caller of
`UpdateEnvironment`, by setting `version_label` on
`module.environment.aws_elastic_beanstalk_environment.this`. GitHub then
health-checks, smokes, and requests one guarded Terraform rollback. Terraform
does not manage `aws_elastic_beanstalk_application_version`; retained versions
are the rollback inventory.
`release_version_label` is a nullable root and module variable. Null VCS plans
leave the live version unchanged. Application-CD runs pass the immutable
`<full-sha>-<run-id>-<attempt>` label only as a run-specific
`TF_VAR_release_version_label` HCL string. Do not set this variable on the
workspace, in a variable set, or in `terraform.tfvars`. Do not upload a new
configuration version on application releases; `create-run` reuses the
workspace's last applied VCS config. Global auto-apply stays off. GitHub
`apply-run` treats an already-applied run as success so a mis-set auto-apply
cannot start a false-failure rollback. The workspace stays branch-based on
`dev` with Automatic Speculative Plans enabled and trigger patterns
`terraform/live/dev/**` and `terraform/live/modules/**`. GitHub discards a
leftover non-speculative VCS run before `create-run`, so a merge to `dev`
cannot lock the workspace out from under GitHub CD. GitHub applies only after
`plan-output` counts are `0/1/0` and
`scripts/check-terraform-release-plan.py` accepts a version-only plan JSON.
Staging application CD uses the same guarded lane against workspace
`shoc-backend-staging`. The workspace stays branch-based on `staging` with
trigger patterns `terraform/live/staging/**` and `terraform/live/modules/**`,
and GitHub deploys on pushes to `staging` and on manual `workflow_dispatch`.
### Credentials and enablement
Store dedicated HCP team tokens as the GitHub environment secret `TF_API_TOKEN`:
- `dev`: use a token scoped only to workspace `shoc-backend-dev`.
- `staging`: use a separate token scoped only to workspace
`shoc-backend-staging`.
Plan JSON download requires workspace admin on the corresponding workspace. Do
not grant project admin or workspace create/move/delete permissions. Do not rely
on a repository-level token or reuse the dev-scoped token for staging. Rotate
each token at least every 90 days.
Repository variable `TERRAFORM_APP_CD_ENABLED` starts unset/false so pushes to
`dev` do not deploy. `workflow_dispatch` on `dev` still runs a release for the
first manual proof. Set the variable to `true` only after that proof confirms
the exact version, a version-only plan, apply, `efbundle`, Ready/Green, smokes,
and a retained previous version.
This change is the allowed exception that mixes deployable application CD with
the Terraform variable that application CD needs. Later PRs must not mix
deployable application changes with Terraform or CDK changes.
## Pinned live identities
- Dev: workspace `shoc-backend-dev`; EB environment `shoc-backend-dev`
(`e-hehnrqjjrt`); .NET 8 AL2023 `3.11.3`; `api.dev.seahaven.com`.
- Staging: workspace `shoc-backend-staging`; EB environment
`shoc-backend-staging` (`e-6c9m4vb62z`); .NET 8 AL2023 `3.11.3`;
`api.staging.seahaven.com`.
The environment roots are intentionally not general-purpose modules. Exact
identifiers make accidental cross-environment reuse fail review and planning.
## Safety invariants
- Auto-apply remains off.
- VCS stays branch-based on `dev` for `shoc-backend-dev` and on `staging` for
`shoc-backend-staging`, with speculative PR plans enabled and trigger
patterns `terraform/live/dev/**` (dev) and `terraform/live/staging/**`
(staging), each alongside `terraform/live/modules/**`. Do not switch
Automatic Run Triggering to tag-based.
- Org baseline owns final HCP plan/apply permissions and manager tags.
- Every imported Terraform resource has `prevent_destroy`.