shoc-backend/terraform/live/README.md
Adam Moussa 77c3016c9d
Some checks are pending
Validate and deploy / Validate deployable source bundle (push) Waiting to run
Validate and deploy / Deploy shoc-backend-dev through Terraform (push) Blocked by required conditions
Validate and deploy / Deploy shoc-backend-staging to Elastic Beanstalk (push) Blocked by required conditions
feat(deploy): move dev application CD through Terraform (#102)
* feat(deploy): move dev application CD through Terraform

GitHub creates the immutable Elastic Beanstalk version; HCP Terraform is the only UpdateEnvironment caller via a guarded version_label run.

* fix: add permissions block for dependency-review workflow

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix(terraform): stop pinning the generated dev instance SG

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-09-03 14:12:06 +00:00

9.8 KiB

Backend environment adoption

These roots adopt environment-owned infrastructure without taking ownership of shared or Elastic Beanstalk-generated infrastructure.

Ownership

  • dev/ and staging/ import the existing EB environment, runtime role/profile/policies, deploy role/policy, app-config secret metadata, and API record.
  • tf-poc/ manages the retained rehearsal environment after its completed CloudFormation-to-Terraform transfer, excluding the live-only webhook and Dynamo policies. It also owns the child zone and DNS-validated ACM certificate.
  • modules/environment-inventory/ reads and pins only shared resources.
  • Org-baseline CloudFormation owns the narrowly scoped HCP Terraform plan/apply roles.

The shared shoc-backend Elastic Beanstalk application and shoc-sqlserver-shared RDS instance, VPC, subnets, EB service role, shared certificate, shared RDS security group, and EB-generated SG/ALB/ASG/CloudFormation resources must never enter an environment state. The shoc_tf_poc SQL catalog is out of band.

Secret values are not Terraform resources, variables, outputs, or managed EB settings. Terraform manages the app-config secret shell and maps approved JSON keys through aws:elasticbeanstalk:application:environmentsecrets using secret-arn:json-key references. Ordinary application environment settings are limited to non-secret ASP.NET and webhook configuration. The pinned .NET 8 AL2023 platform 3.11.3 supports Secrets Manager JSON-key extraction.

Mandatory live secret migration

Before importing dev or staging, perform a separately approved production mutation from a trusted local session:

  1. Create a temporary OptionSettings JSON file containing the exact environmentsecrets ARN/key references configured in that root and the five non-secret ordinary environment settings.
  2. Create a temporary OptionsToRemove JSON file naming the old raw secret-valued keys in aws:elasticbeanstalk:application:environment.
  3. Run aws elasticbeanstalk update-environment for exactly shoc-backend-dev or shoc-backend-staging with --option-settings file://... and --options-to-remove file://.... Include the provider-normalized sorted Subnets and ELBSubnets values in this same approved update if live ordering differs.
  4. Delete both files, wait for the replacement environment to become Ready and healthy, and run scripts/smoke-elastic-beanstalk.sh against the exact API.
  5. Verify the raw ordinary secret settings are absent before generating the first Terraform plan.

This migration is not performed by Terraform. The first import plan remains zero-change only after the migration succeeds.

Mandatory role-boundary attachment

After the org baseline creates the dedicated boundary policies, perform a separately approved production IAM mutation that attaches:

  • shoc-backend-dev-runtime-boundary to shoc-backend-dev
  • shoc-backend-staging-runtime-boundary to shoc-backend-staging
  • shoc-backend-dev-deploy-boundary to githubdeploy-shoc-backend-dev
  • shoc-backend-staging-deploy-boundary to githubdeploy-shoc-backend-staging

Attach all four boundaries before the SCP and HCP PutRolePolicy exceptions become effective. In the same approved pre-import phase, add the immutable HcpTerraformWorkspace tag to each GitHub deploy role:

  • githubdeploy-shoc-backend-dev: shoc-backend-dev
  • githubdeploy-shoc-backend-staging: shoc-backend-staging

Verify each exact runtime and deploy boundary ARN and workspace tag from GetRole before importing. Terraform requires the boundaries to be present during adoption_complete=false, so the first import remains zero-change. Terraform does not perform this pre-import mutation.

Two-phase adoption

Each dev/staging root pins adoption_complete=false in reviewed code until its initial import is proven. It is not an HCP workspace variable. The retained tf-poc rehearsal has completed both phases and therefore pins adoption_complete=true.

  1. Create the HCP workspace and configure dynamic credentials.

  2. Run the declarative imports.

  3. Export the HCP plan as JSON and run:

    python scripts/check-terraform-import-plan.py plan.json --environment dev
    

    The first plan must be a no-op after import. The guard rejects updates, creates, deletes, replacements, and managed resource types outside the approved environment-owned boundary.

  4. Apply the no-op import only after review.

  5. Change the environment root to adoption_complete=true and manage_eb_settings=true in a reviewed code change, then review the controlled in-place role metadata, secret metadata, and Elastic Beanstalk update:

    # Dev example. Omit any address that is not updating.
    python scripts/check-terraform-import-plan.py plan.json --environment dev \
      --allow-update-address module.environment.aws_iam_instance_profile.runtime \
      --allow-update-address module.environment.aws_iam_role.runtime \
      --allow-update-address module.environment.aws_iam_role.github_deploy \
      --allow-update-address module.environment.aws_secretsmanager_secret.app_config \
      --allow-update-address module.environment.aws_elastic_beanstalk_environment.this
    
  6. Apply only when every update address is named on the command line and the plan contains no create, delete, or replacement action. The dev direct ALB alias remains pinned during this phase and must not update.

The dev deploy role and generated inline policy completed their retained CloudFormation-to-Terraform transfer before the legacy backend CDK source was removed. Do not reintroduce that ownership path.

The reviewed adoption_complete=true change updates ownership tags on IAM roles, instance profiles, and app-config secrets. Elastic Beanstalk environment tags remain at their imported values. Terraform manages the declared EB settings. Secret values remain out-of-band even after the secret shell receives ManagedBy=terraform. Deploy-role descriptions and immutable HcpTerraformWorkspace tags remain unchanged. Read-only AWS APIs retain Resource = "*" only where AWS does not support resource-level permissions.

The measured self-contained .NET/EF bundle is approximately 199.5 MB and separate builds are not byte-identical. Each deploy job therefore validates the exact bundle it uploads; bundle bytes never enter Terraform plans or state.

Dev application CD

GitHub compiles, validates, and uploads the bundle, then creates the immutable Elastic Beanstalk application version. HCP Terraform is the only caller of UpdateEnvironment, by setting version_label on module.environment.aws_elastic_beanstalk_environment.this. GitHub then health-checks, smokes, and requests one guarded Terraform rollback. Terraform does not manage aws_elastic_beanstalk_application_version; retained versions are the rollback inventory.

release_version_label is a nullable root and module variable. Null VCS plans leave the live version unchanged. Application-CD runs pass the immutable <full-sha>-<run-id>-<attempt> label only as a run-specific TF_VAR_release_version_label HCL string. Do not set this variable on the workspace, in a variable set, or in terraform.tfvars. Do not upload a new configuration version on application releases; create-run reuses the workspace's last applied VCS config. Global auto-apply stays off. GitHub applies only after plan-output counts are 0/1/0 and scripts/check-terraform-release-plan.py accepts a version-only plan JSON.

Staging keeps today's direct Elastic Beanstalk deploy path until staging adoption.

Credentials and enablement

Store a dedicated HCP team token only as the GitHub dev environment secret TF_API_TOKEN. Scope it to workspace shoc-backend-dev. Plan JSON download requires workspace admin on that one workspace. Do not grant project admin, workspace create/move/delete, or staging access. Rotate at least every 90 days.

Repository variable TERRAFORM_APP_CD_ENABLED starts unset/false so pushes to dev do not deploy. workflow_dispatch on dev still runs a release for the first manual proof. Set the variable to true only after that proof confirms the exact version, a version-only plan, apply, efbundle, Ready/Green, smokes, and a retained previous version.

This change is the allowed exception that mixes deployable application CD with the Terraform variable that application CD needs. Later PRs must not mix deployable application changes with Terraform or CDK changes.

POC retained identifiers

The tf-poc HCP workspace stores the exact retained environment ID, app-config secret ARN, child-zone ID, and certificate ARN declared in tf-poc/variables.tf. The declarative import blocks consumed those identifiers during the completed transfer. Do not guess or replace them, and do not put credentials or secret values in HCP variables.

ACM DNS validation remains part of the Terraform-owned certificate resource; its generated validation record is not a separate ownership target. The public delegation of tf-poc.seahaven.com from seahaven.com remains outside this Terraform state.

Pinned live identities

  • Dev: workspace shoc-backend-dev; EB environment shoc-backend-dev (e-hehnrqjjrt); .NET 8 AL2023 3.11.3; api.dev.seahaven.com.
  • Staging: workspace shoc-backend-staging; EB environment shoc-backend-staging (e-6c9m4vb62z); .NET 8 AL2023 3.11.3; api.staging.seahaven.com.

The environment roots are intentionally not general-purpose modules. Exact identifiers make accidental cross-environment reuse fail review and planning.

Safety invariants

  • Auto-apply remains off.
  • Org baseline owns final HCP plan/apply permissions and manager tags.
  • Every imported Terraform resource has prevent_destroy.
  • The tf-poc CloudFormation creator path was removed after its no-op import, controlled update, and retained-resource ownership transfer completed.