fix(infra): allow scoped apply to create the EC2 recover alarm (PLAT-78) (#40)

* fix(infra): allow scoped apply to create the EC2 recover alarm

PutMetricAlarm with the automate recover action needs RecoverInstances plus CreateServiceLinkedRole for AWSServiceRoleForCloudWatchEvents, which does not exist in seahaven-prod yet.

* docs: record prod EIP and bootstrap-then-scoped apply split

Bootstrap cannot CreateVpc; the live collector is 184.193.220.187 until UniFi is re-pointed.
This commit is contained in:
Adam Moussa 2026-09-16 21:48:35 +00:00 • committed by GitHub
parent e17b284370
commit dd61d34cd7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 24 additions and 5 deletions

View file

@ -30,7 +30,7 @@ office UniFi devices ──syslog/514──▶ EIP (prod) ──▶ EC2 (rsyslog
| HCP plan/apply roles | `hcptf-syslog-server-plan` / `hcptf-syslog-server` |
| Instance | `syslog-server`, t4g.nano, Amazon Linux 2023 (arm64), 30 GiB encrypted gp3 |
| VPC | dedicated `10.40.0.0/16`, public subnet `10.40.10.0/24` |
| Elastic IP | Terraform-managed (see HCP output `public_ip`) |
| Elastic IP | `184.193.220.187` (`eipalloc-07d82c1f79a22716a`) — UniFi still points at mgmt until INFRA-11 |
| Security group | `syslog-server` — 514 tcp/udp + 22 from office IPs + VPC/VPN CIDRs; 2055/2056 udp from office |
| Instance IAM | `/tf-managed/syslog-server-role` with `syslog-server-instance-boundary`; `AmazonSSMManagedInstanceCore` + `CloudWatchAgentServerPolicy` |
| Log group | `unifi-syslog` (90-day retention) |
@ -54,12 +54,12 @@ First apply uses the hcptf-bootstrap window (exact `StringEquals` trust, never
3. Point workspace `TFC_AWS_APPLY_ROLE_ARN` / `TFC_AWS_PLAN_ROLE_ARN` at
`hcptf-bootstrap` / `hcptf-bootstrap-plan`. Set `TFC_AWS_PROVIDER_AUTH=true`.
Never `TFC_AWS_RUN_ROLE_ARN`.
4. One manual apply. This creates the scoped `hcptf-*` roles, the instance
boundary, VPC, instance, EIP, log group, and alarms.
4. One manual apply as `hcptf-bootstrap` creates the scoped `hcptf-*` roles,
boundary, and instance role. Bootstrap cannot `ec2:CreateVpc`; the rest of
the stack applies as `hcptf-syslog-server`.
5. Retarget `TFC_AWS_*` to `hcptf-syslog-server` / `hcptf-syslog-server-plan`.
Re-run the create script with no `--allow-workspace`.
6. Second manual apply as the scoped role. After live-path proof, seal
auto-apply on.
6. Manual apply as the scoped role. After live-path proof, seal auto-apply on.
`Syslog-NoIncomingLogs` defaults `treat_missing_data` to `notBreaching` so the
empty prod log group does not page `site-alerts` before UniFi is re-pointed.

View file

@ -392,6 +392,7 @@ data "aws_iam_policy_document" "hcptf_apply_services" {
"ec2:DescribeImages",
"ec2:DescribeInstanceAttribute",
"ec2:DescribeInstanceCreditSpecifications",
"ec2:DescribeInstanceRecoveryAttribute",
"ec2:DescribeInstanceStatus",
"ec2:DescribeInstanceTypes",
"ec2:DescribeInstances",
@ -405,6 +406,7 @@ data "aws_iam_policy_document" "hcptf_apply_services" {
"ec2:ModifyVolume",
"ec2:MonitorInstances",
"ec2:RebootInstances",
"ec2:RecoverInstances",
"ec2:ReplaceIamInstanceProfileAssociation",
"ec2:RunInstances",
"ec2:StartInstances",
@ -414,6 +416,22 @@ data "aws_iam_policy_document" "hcptf_apply_services" {
]
resources = ["*"]
}
statement {
sid = "CloudWatchEc2RecoverServiceLinkedRole"
effect = "Allow"
actions = [
"iam:CreateServiceLinkedRole",
]
resources = [
"arn:aws:iam::${local.account_id}:role/aws-service-role/events.amazonaws.com/AWSServiceRoleForCloudWatchEvents",
]
condition {
test = "StringEquals"
variable = "iam:AWSServiceName"
values = ["events.amazonaws.com"]
}
}
}
data "aws_iam_policy_document" "hcptf_plan_refresh" {
@ -489,6 +507,7 @@ data "aws_iam_policy_document" "hcptf_plan_refresh" {
"ec2:DescribeImages",
"ec2:DescribeInstanceAttribute",
"ec2:DescribeInstanceCreditSpecifications",
"ec2:DescribeInstanceRecoveryAttribute",
"ec2:DescribeInstanceStatus",
"ec2:DescribeInstanceTypes",
"ec2:DescribeInstances",