shoc-backend/terraform/live/README.md

11 KiB

Backend environment adoption

These roots adopt environment-owned infrastructure without taking ownership of shared or Elastic Beanstalk-generated infrastructure.

Ownership

  • dev/ and staging/ import the existing EB environment, runtime role/profile/policies, deploy role/policy, app-config secret metadata, and API record.
  • modules/environment-inventory/ reads and pins only shared resources.
  • Org-baseline CloudFormation owns the narrowly scoped HCP Terraform plan/apply roles.

The shared shoc-backend Elastic Beanstalk application and shoc-sqlserver-shared RDS instance, VPC, subnets, EB service role, shared certificate, shared RDS security group, and EB-generated SG/ALB/ASG/CloudFormation resources must never enter an environment state.

Secret values are not Terraform resources, variables, outputs, or managed EB settings. Terraform manages the app-config secret shell and maps approved JSON keys through aws:elasticbeanstalk:application:environmentsecrets using secret-arn:json-key references. Ordinary application environment settings are limited to non-secret ASP.NET, webhook, and Sentry configuration. The Sentry DSN is public ingestion configuration, delivered as the plain SENTRY_DSN setting from the required sentry_dsn variable. SENTRY_ENVIRONMENT is development for the dev root and the environment name (staging) otherwise. No production root exists yet; production Sentry wiring will follow the same variable when one is added. The pinned .NET 8 AL2023 platform 3.11.3 supports Secrets Manager JSON-key extraction.

Mandatory live secret migration

Before importing dev or staging, perform a separately approved production mutation from a trusted local session:

  1. Create a temporary OptionSettings JSON file containing the exact environmentsecrets ARN/key references configured in that root and the five non-secret ordinary environment settings.
  2. Create a temporary OptionsToRemove JSON file naming the old raw secret-valued keys in aws:elasticbeanstalk:application:environment.
  3. Run aws elasticbeanstalk update-environment for exactly shoc-backend-dev or shoc-backend-staging with --option-settings file://... and --options-to-remove file://.... Include the provider-normalized sorted Subnets and ELBSubnets values in this same approved update if live ordering differs.
  4. Delete both files, wait for the replacement environment to become Ready and healthy, and run scripts/smoke-elastic-beanstalk.sh against the exact API.
  5. Verify the raw ordinary secret settings are absent before generating the first Terraform plan.

This migration is not performed by Terraform. The first import plan remains zero-change only after the migration succeeds.

Mandatory role-boundary attachment

After the org baseline creates the dedicated boundary policies, perform a separately approved production IAM mutation that attaches:

  • shoc-backend-dev-runtime-boundary to shoc-backend-dev
  • shoc-backend-staging-runtime-boundary to shoc-backend-staging
  • shoc-backend-dev-deploy-boundary to githubdeploy-shoc-backend-dev
  • shoc-backend-staging-deploy-boundary to githubdeploy-shoc-backend-staging

Attach all four boundaries before the SCP and HCP PutRolePolicy exceptions become effective. In the same approved pre-import phase, add the immutable HcpTerraformWorkspace tag to each GitHub deploy role:

  • githubdeploy-shoc-backend-dev: shoc-backend-dev
  • githubdeploy-shoc-backend-staging: shoc-backend-staging

Verify each exact runtime and deploy boundary ARN and workspace tag from GetRole before importing. Terraform requires the boundaries to be present during adoption_complete=false, so the first import remains zero-change. Terraform does not perform this pre-import mutation.

Two-phase adoption

Each dev/staging root pins adoption_complete=false in reviewed code until its initial import is proven. It is not an HCP workspace variable.

  1. Create the HCP workspace and configure dynamic credentials.

  2. Run the declarative imports.

  3. Export the HCP plan as JSON and run:

    python scripts/check-terraform-import-plan.py plan.json --environment dev
    

    The first plan must be a no-op after import. The guard rejects updates, creates, deletes, replacements, and managed resource types outside the approved environment-owned boundary.

  4. Apply the no-op import only after review.

  5. Change the environment root to adoption_complete=true and manage_eb_settings=true in a reviewed code change, then review the controlled in-place role metadata, secret metadata, and Elastic Beanstalk update:

    # Dev example. Omit any address that is not updating.
    python scripts/check-terraform-import-plan.py plan.json --environment dev \
      --allow-update-address module.environment.aws_iam_instance_profile.runtime \
      --allow-update-address module.environment.aws_iam_role.runtime \
      --allow-update-address module.environment.aws_iam_role.github_deploy \
      --allow-update-address module.environment.aws_secretsmanager_secret.app_config \
      --allow-update-address module.environment.aws_elastic_beanstalk_environment.this
    
  6. Apply only when every update address is named on the command line and the plan contains no create, delete, or replacement action. The dev direct ALB alias remains pinned during this phase and must not update.

The dev deploy role and generated inline policy completed their retained CloudFormation-to-Terraform transfer before the legacy backend CDK source was removed. Do not reintroduce that ownership path.

The reviewed adoption_complete=true change updates ownership tags on IAM roles, instance profiles, and app-config secrets. Elastic Beanstalk environment tags remain at their imported values. Terraform manages the declared EB settings. Secret values remain out-of-band even after the secret shell receives ManagedBy=terraform. Deploy-role descriptions and immutable HcpTerraformWorkspace tags remain unchanged. Read-only AWS APIs retain Resource = "*" only where AWS does not support resource-level permissions.

The measured self-contained .NET/EF bundle is approximately 199.5 MB and separate builds are not byte-identical. Each deploy job therefore validates the exact bundle it uploads; bundle bytes never enter Terraform plans or state.

Application CD

GitHub Actions owns application versions. It compiles the bundle, uploads it, creates the Elastic Beanstalk application version, and calls UpdateEnvironment. Terraform ignores version_label so those deploys are not drift. If health, smoke, or the webhook probe fails after that update, the job restores the previous Elastic Beanstalk version label. Database migrations already applied by the failed bundle are not reverted. Deploy parameters are read from /shoc-backend/<env>/deploy/* SSM parameters this module writes.

Merge to main deploys dev unless the push is terraform-only. Staging is cut from Actions → Release (environment, bump, message). That workflow waits for CI, tags vX.Y.Z-staging from main HEAD with GITHUB_TOKEN, then calls deploy. Do not cut prod yet; leave PROD_APP_CD_ENABLED unset and do not create the prod GitHub Environment. Staging remains adoption_complete=false with a pinned API CNAME until its import apply is proven.

HCP workspaces stay VCS-driven with auto-apply on after cutover. Speculative plans on every PR are the infra gate. Do not point TFC_AWS_* at hcptf-bootstrap. Org-baseline CloudFormation owns the HCP plan/apply roles. GitHub Actions does not create, wait on, or apply HCP runs. Application and Terraform changes stay in separate PRs so a merge cannot race an HCP apply against an app deploy. Terraform-only merges skip deploy.yaml. App-only tags skip HCP when trigger patterns do not match.

Until cutover, .github/workflows/deploy.yml still uses TF_API_TOKEN and TERRAFORM_APP_CD_ENABLED. Keep those secrets and the version-only plan guard on that leftover path only.

Credentials

Store DEPLOY_ROLE_ARN as a GitHub Environment variable (dev, staging). OIDC trust is repo:Sea-Haven-Industries/shoc-backend:environment:<env> plus job_workflow_ref for .github/workflows/deploy.yaml at refs/heads/main and refs/tags/v*. Adding another deploy workflow is a cross-family IAM change. After cutover, drop TF_API_TOKEN from GitHub Environments. The new CD path does not use it.

GitHub Environment deployment branch and tag policies are repository settings, not this diff. Update them before the first merge to main and the first staging cut. The policy matches GITHUB_REF of the workflow run. Branch patterns never match tag refs; adding v* as a branch pattern fails the same way as an empty allowlist.

  1. dev — allow branch main. Keep dev allowed while leftover .github/workflows/deploy.yml still deploys from that branch.
  2. staging — add a tag-type policy matching v*.*.*-staging for deploy-tag.yaml. Allow branch main because Actions → Release is workflow_dispatch on main and then calls deploy.yaml (GITHUB_TOKEN tag pushes do not start deploy-tag.yaml). Keep staging allowed while leftover deploy.yml still deploys from that branch.

Do not create the prod environment yet. Leave PROD_APP_CD_ENABLED unset. Until the prod environment exists with reviewers, do not run Actions → Release with environment=prod, do not push a bare vX.Y.Z tag, and do not workflow_dispatch deploy with environment=prod. Any of those declares environment: prod and would auto-create an unprotected environment.

Pinned live identities

  • Dev: workspace shoc-backend-dev; EB environment shoc-backend-dev (e-hehnrqjjrt); .NET 8 AL2023 3.11.3; api.dev.seahaven.com.
  • Staging: workspace shoc-backend-staging; EB environment shoc-backend-staging (e-6c9m4vb62z); .NET 8 AL2023 3.11.3; api.staging.seahaven.com.

The environment roots are intentionally not general-purpose modules. Exact identifiers make accidental cross-environment reuse fail review and planning.

Safety invariants

  • After cutover, auto-apply is on. Speculative PR plans stay on.
  • shoc-backend-dev is branch-based on main with trigger patterns terraform/live/dev/** and terraform/live/modules/**.
  • shoc-backend-staging is tag-based on ^v\d+\.\d+\.\d+-staging$ with trigger patterns terraform/live/staging/** and terraform/live/modules/**.
  • Org baseline owns HCP plan/apply permissions and manager tags. Never manage hcptf-* in this repository.
  • Every imported Terraform resource has prevent_destroy.
  • Elastic Beanstalk version_label is ignored so GitHub deploys are not drift.