shoc-backend/terraform/live
2026-09-16 16:07:16 -03:00
..
dev feat: add API Sentry tracing (SH-298) (#105) 2026-09-04 14:11:32 -03:00
modules feat(deploy): rebuild protected staging lane 2026-09-16 15:52:02 -03:00
staging feat(deploy): rebuild protected staging lane 2026-09-16 15:52:02 -03:00
README.md docs(terraform): document staging token scope 2026-09-16 16:07:16 -03:00

Backend environment adoption

These roots adopt environment-owned infrastructure without taking ownership of shared or Elastic Beanstalk-generated infrastructure.

Ownership

  • dev/ and staging/ import the existing EB environment, runtime role/profile/policies, deploy role/policy, app-config secret metadata, and API record.
  • modules/environment-inventory/ reads and pins only shared resources.
  • Org-baseline CloudFormation owns the narrowly scoped HCP Terraform plan/apply roles.

The shared shoc-backend Elastic Beanstalk application and shoc-sqlserver-shared RDS instance, VPC, subnets, EB service role, shared certificate, shared RDS security group, and EB-generated SG/ALB/ASG/CloudFormation resources must never enter an environment state.

Secret values are not Terraform resources, variables, outputs, or managed EB settings. Terraform manages the app-config secret shell and maps approved JSON keys through aws:elasticbeanstalk:application:environmentsecrets using secret-arn:json-key references. Ordinary application environment settings are limited to non-secret ASP.NET, webhook, and Sentry configuration. The Sentry DSN is public ingestion configuration, delivered as the plain SENTRY_DSN setting from the required sentry_dsn variable. SENTRY_ENVIRONMENT is development for the dev root and the environment name (staging) otherwise. No production root exists yet; production Sentry wiring will follow the same variable when one is added. The pinned .NET 8 AL2023 platform 3.11.3 supports Secrets Manager JSON-key extraction.

Mandatory live secret migration

Before importing dev or staging, perform a separately approved production mutation from a trusted local session:

  1. Create a temporary OptionSettings JSON file containing the exact environmentsecrets ARN/key references configured in that root and the five non-secret ordinary environment settings.
  2. Create a temporary OptionsToRemove JSON file naming the old raw secret-valued keys in aws:elasticbeanstalk:application:environment.
  3. Run aws elasticbeanstalk update-environment for exactly shoc-backend-dev or shoc-backend-staging with --option-settings file://... and --options-to-remove file://.... Include the provider-normalized sorted Subnets and ELBSubnets values in this same approved update if live ordering differs.
  4. Delete both files, wait for the replacement environment to become Ready and healthy, and run scripts/smoke-elastic-beanstalk.sh against the exact API.
  5. Verify the raw ordinary secret settings are absent before generating the first Terraform plan.

This migration is not performed by Terraform. The first import plan remains zero-change only after the migration succeeds.

Mandatory role-boundary attachment

After the org baseline creates the dedicated boundary policies, perform a separately approved production IAM mutation that attaches:

  • shoc-backend-dev-runtime-boundary to shoc-backend-dev
  • shoc-backend-staging-runtime-boundary to shoc-backend-staging
  • shoc-backend-dev-deploy-boundary to githubdeploy-shoc-backend-dev
  • shoc-backend-staging-deploy-boundary to githubdeploy-shoc-backend-staging

Attach all four boundaries before the SCP and HCP PutRolePolicy exceptions become effective. In the same approved pre-import phase, add the immutable HcpTerraformWorkspace tag to each GitHub deploy role:

  • githubdeploy-shoc-backend-dev: shoc-backend-dev
  • githubdeploy-shoc-backend-staging: shoc-backend-staging

Verify each exact runtime and deploy boundary ARN and workspace tag from GetRole before importing. Terraform requires the boundaries to be present during adoption_complete=false, so the first import remains zero-change. Terraform does not perform this pre-import mutation.

Two-phase adoption

Each dev/staging root pins adoption_complete=false in reviewed code until its initial import is proven. It is not an HCP workspace variable.

  1. Create the HCP workspace and configure dynamic credentials.

  2. Run the declarative imports.

  3. Export the HCP plan as JSON and run:

    python scripts/check-terraform-import-plan.py plan.json --environment dev
    

    The first plan must be a no-op after import. The guard rejects updates, creates, deletes, replacements, and managed resource types outside the approved environment-owned boundary.

  4. Apply the no-op import only after review.

  5. Change the environment root to adoption_complete=true and manage_eb_settings=true in a reviewed code change, then review the controlled in-place role metadata, secret metadata, and Elastic Beanstalk update:

    # Dev example. Omit any address that is not updating.
    python scripts/check-terraform-import-plan.py plan.json --environment dev \
      --allow-update-address module.environment.aws_iam_instance_profile.runtime \
      --allow-update-address module.environment.aws_iam_role.runtime \
      --allow-update-address module.environment.aws_iam_role.github_deploy \
      --allow-update-address module.environment.aws_secretsmanager_secret.app_config \
      --allow-update-address module.environment.aws_elastic_beanstalk_environment.this
    
  6. Apply only when every update address is named on the command line and the plan contains no create, delete, or replacement action. The dev direct ALB alias remains pinned during this phase and must not update.

The dev deploy role and generated inline policy completed their retained CloudFormation-to-Terraform transfer before the legacy backend CDK source was removed. Do not reintroduce that ownership path.

The reviewed adoption_complete=true change updates ownership tags on IAM roles, instance profiles, and app-config secrets. Elastic Beanstalk environment tags remain at their imported values. Terraform manages the declared EB settings. Secret values remain out-of-band even after the secret shell receives ManagedBy=terraform. Deploy-role descriptions and immutable HcpTerraformWorkspace tags remain unchanged. Read-only AWS APIs retain Resource = "*" only where AWS does not support resource-level permissions.

The measured self-contained .NET/EF bundle is approximately 199.5 MB and separate builds are not byte-identical. Each deploy job therefore validates the exact bundle it uploads; bundle bytes never enter Terraform plans or state.

Dev application CD

GitHub compiles, validates, and uploads the bundle, then creates the immutable Elastic Beanstalk application version. HCP Terraform is the only caller of UpdateEnvironment, by setting version_label on module.environment.aws_elastic_beanstalk_environment.this. GitHub then health-checks, smokes, and requests one guarded Terraform rollback. Terraform does not manage aws_elastic_beanstalk_application_version; retained versions are the rollback inventory.

release_version_label is a nullable root and module variable. Null VCS plans leave the live version unchanged. Application-CD runs pass the immutable <full-sha>-<run-id>-<attempt> label only as a run-specific TF_VAR_release_version_label HCL string. Do not set this variable on the workspace, in a variable set, or in terraform.tfvars. Do not upload a new configuration version on application releases; create-run reuses the workspace's last applied VCS config. Global auto-apply stays off. GitHub apply-run treats an already-applied run as success so a mis-set auto-apply cannot start a false-failure rollback. The workspace stays branch-based on dev with Automatic Speculative Plans enabled and trigger patterns terraform/live/dev/** and terraform/live/modules/**. GitHub discards a leftover non-speculative VCS run before create-run, so a merge to dev cannot lock the workspace out from under GitHub CD. GitHub applies only after plan-output counts are 0/1/0 and scripts/check-terraform-release-plan.py accepts a version-only plan JSON.

Staging application CD uses the same guarded lane against workspace shoc-backend-staging. The workspace stays branch-based on staging with trigger patterns terraform/live/staging/** and terraform/live/modules/**, and GitHub deploys on pushes to staging and on manual workflow_dispatch.

Credentials and enablement

Store dedicated HCP team tokens as the GitHub environment secret TF_API_TOKEN:

  • dev: use a token scoped only to workspace shoc-backend-dev.
  • staging: use a separate token scoped only to workspace shoc-backend-staging.

Plan JSON download requires workspace admin on the corresponding workspace. Do not grant project admin or workspace create/move/delete permissions. Do not rely on a repository-level token or reuse the dev-scoped token for staging. Rotate each token at least every 90 days.

Repository variable TERRAFORM_APP_CD_ENABLED starts unset/false so pushes to dev do not deploy. workflow_dispatch on dev still runs a release for the first manual proof. Set the variable to true only after that proof confirms the exact version, a version-only plan, apply, efbundle, Ready/Green, smokes, and a retained previous version.

This change is the allowed exception that mixes deployable application CD with the Terraform variable that application CD needs. Later PRs must not mix deployable application changes with Terraform or CDK changes.

Pinned live identities

  • Dev: workspace shoc-backend-dev; EB environment shoc-backend-dev (e-hehnrqjjrt); .NET 8 AL2023 3.11.3; api.dev.seahaven.com.
  • Staging: workspace shoc-backend-staging; EB environment shoc-backend-staging (e-6c9m4vb62z); .NET 8 AL2023 3.11.3; api.staging.seahaven.com.

The environment roots are intentionally not general-purpose modules. Exact identifiers make accidental cross-environment reuse fail review and planning.

Safety invariants

  • Auto-apply remains off.
  • VCS stays branch-based on dev for shoc-backend-dev and on staging for shoc-backend-staging, with speculative PR plans enabled and trigger patterns terraform/live/dev/** (dev) and terraform/live/staging/** (staging), each alongside terraform/live/modules/**. Do not switch Automatic Run Triggering to tag-based.
  • Org baseline owns final HCP plan/apply permissions and manager tags.
  • Every imported Terraform resource has prevent_destroy.