From f0a2b7191d7e30d365c9b7fd2a453c4a0f8c533b Mon Sep 17 00:00:00 2001 From: Adam Moussa Date: Mon, 27 Jul 2026 17:44:58 -0400 Subject: [PATCH] fix(iam): close boundary-removal escalation in github-cfn-execution-role (Phase A) The shared CloudFormation execution role could remove the permissions boundary from the very roles that boundary was gating. Its iam-role-management-boundary-gated policy allows iam:DeleteRolePermissionsBoundary on role/* under a StringEquals condition on iam:PermissionsBoundary -- and for a delete that condition key reflects the boundary CURRENTLY attached to the target role, so it matches exactly the roles the gate protects. Create a boundary-gated role with an inline *:* policy, strip its boundary, PassRole it to Lambda, and the result is unbounded admin in the management account. Confirmed live with simulate-principal-policy, not inferred. Phase A adds an attached managed policy, seahaven-cfn-exec-iam-management, carrying the corrected statement set: the boundary-gated Allows without iam:DeleteRolePermissionsBoundary, plus three Deny backstops -- DenyBoundaryTampering (boundary removal), DenyBoundaryPolicyEdit (rewriting a seahaven-* policy document) and DenySelfMutation. DenySelfMutation exists because the first draft of this fix was not durable: the role holds iam:DetachRolePolicy, iam:DeleteRolePolicy and iam:DeleteRole on Resource "*" with no condition, so it could detach the Deny-carrying policy from itself in one call and reinstate the escalation. It now cannot modify its own role, any githubdeploy-* role, or any seahaven-* policy. Nothing legitimate needs that: the deploy substrate's own principals are owned by this stack, which is deployed manually with administrator credentials rather than through this role. The change is additive. The old inline policy stays in place, so CloudFormation removes nothing and there is no window in which the role lacks its IAM permissions -- an explicit Deny beats an Allow anywhere in the policy set, so the corrected version governs from the moment this lands. Phase B removes the redundant inline copy. The split is also required by size: inline sits at 10,006 of IAM's 10,240-byte per-role limit, and the Deny statements do not fit there. Also reconciles drift. The deployed role carries three logs:*MetricFilter actions added out-of-band on 2026-06-29 and never back-ported. afterhours-shift-manager creates an AWS::Logs::MetricFilter through this role, so they are load-bearing; the template now matches the live policy exactly, which keeps inline at 10,006 and stops a future write-back from silently stripping them. --- README.md | 16 +++ oidc-deploy-roles.yaml | 241 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 253 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 05f4f6a..d8657af 100644 --- a/README.md +++ b/README.md @@ -62,6 +62,11 @@ PR reviews are handled by the **official Claude Code GitHub App** (installed org - One **OIDC deploy role per repo** (`githubdeploy-`), assumed by that repo's `deploy.yaml` via OIDC and passed in as `AWS_DEPLOY_ROLE_ARN`. CDK repos use these to assume the `cdk-hnb659fds-*` bootstrap roles; SAM repos use these to run `sam deploy`. - The shared **SAM CloudFormation execution role** `github-cfn-execution-role` (`SamCfnExecutionRole`) — passed as `cfn-role-arn` by every SAM `deploy.yaml` (see §5). CloudFormation assumes it to provision the SAM stacks' resources. - The **`seahaven-lambda-execution-boundary`** managed policy. +- The **`seahaven-cfn-exec-iam-management`** managed policy (`SamCfnIamManagementPolicy`), attached to `github-cfn-execution-role`. It holds that role's boundary-gated IAM statements plus the Deny backstops that keep the permissions boundary from being detached, rewritten, or applied to the deploy substrate's own roles. It lives in a managed policy rather than inline because the role's inline policies sit at 10,006 of IAM's hard 10,240-byte per-role limit; attached managed policies have a separate 6,144-byte budget. + +> **Constraint for future maintainers.** `github-cfn-execution-role` is explicitly denied from mutating the deploy substrate's own principals — itself, any `githubdeploy-*` role, and any `seahaven-*` managed policy. Those are owned by this stack and deployed manually with administrator credentials, so nothing legitimate needs that path. If you ever add automation that manages one of them, it must not run through `github-cfn-execution-role` or it will fail with `AccessDenied`. + +> **Phase A / Phase B.** The boundary-gated statements are currently duplicated: the new managed policy carries the corrected set, and the older inline `iam-role-management-boundary-gated` policy is still present. That overlap is deliberate and temporary — an explicit Deny beats an Allow anywhere in the policy set, so the corrected version already governs, and keeping the inline copy meant CloudFormation removed nothing during the change. **Phase B deletes the inline copy** (inline usage 10,006 → 8,261). Do not delete it as "redundant" outside that planned change. > ⚠️ **This stack has no CD pipeline — it is deployed manually.** (It defines the very roles the pipelines use, so it can't deploy itself.) @@ -103,6 +108,17 @@ A function's effective permissions are the **intersection** of its own role poli Wrong order breaks every SAM deploy. (History: INFRA-103 established the boundary, INFRA-97 scoped the role.) CDK repos are unaffected — they deploy via `cdk-hnb659fds-*` roles, not this execution role. +This ordering rule is about changing the **boundary** or the conditions that gate it. It does not apply to changes that only add permissions to the exec role. + +#### Permissions boundaries can no longer be removed by CloudFormation + +`github-cfn-execution-role` is explicitly denied `iam:DeleteRolePermissionsBoundary`. It can *set* the boundary (that is what SAM needs) but never remove one. Two consequences worth knowing before debugging a stuck stack: + +- **Removing `PermissionsBoundary` from an existing role fails by design.** CloudFormation issues `DeleteRolePermissionsBoundary` for that edit, gets `AccessDenied`, and the stack update rolls back. Removing the boundary from a SAM function is a security regression, so failing loudly is intended. +- **Rollback of an update that *adds* a boundary to an existing role would also fail**, landing the stack in `UPDATE_ROLLBACK_FAILED`. This is currently unreachable — all 26 IAM roles across the five SAM stacks already carry the boundary (verified 2026-07-27), so no update can add one. It becomes reachable again only if a role is created without the boundary and given one later. + +Recovery in either case is an administrator action, not a pipeline retry: clear the wedged stack with `aws cloudformation continue-update-rollback --stack-name --resources-to-skip `, or replace the role by renaming its logical id. Note `cd-sam`'s pre-flight hard-fails on `*ROLLBACK_COMPLETE`, so that repo's deploys stay blocked until it is cleared. + ## Setup ### 1. Create a GitHub App diff --git a/oidc-deploy-roles.yaml b/oidc-deploy-roles.yaml index fe0430c..d027e55 100644 --- a/oidc-deploy-roles.yaml +++ b/oidc-deploy-roles.yaml @@ -241,6 +241,229 @@ Resources: Resource: - !Sub "arn:aws:kms:us-east-1:${AWS::AccountId}:key/*" + # --------------------------------------------------------------------------- + # IAM role lifecycle for the CFN execution role — BOUNDARY-GATED + # (attached managed policy) + # + # Ported verbatim from seahaven-org-baseline + # lib/deploy-substrate/deploy-substrate.template.yaml (stack + # seahaven-deploy-substrate, deployed and verified in seahaven-prod and + # seahaven-dev 2026-07-27). Keep the two copies in lockstep. + # + # Why a MANAGED policy and not inline on the role: this role's inline + # policies total ~10,006 bytes against IAM's hard 10,240-byte per-role + # inline limit — about 234 bytes of headroom. The Deny statements below + # do not fit inline (the equivalent attempt in prod/dev failed with + # ServiceLimitExceeded). Attached managed policies carry their own + # separate 6,144-byte budget. + # + # SECURITY: iam:DeleteRolePermissionsBoundary is deliberately ABSENT from + # the Allow below, and explicitly Denied further down. Granting it under + # the StringEquals iam:PermissionsBoundary condition is self-defeating: + # for a delete, that condition key reflects the boundary CURRENTLY + # attached to the target role, so it matches exactly the roles the gate + # protects — letting this role create a boundary-gated role with an + # inline *:* policy, strip the boundary, and pass the now-unbounded role + # to Lambda. Verified live on this very role 2026-07-27 via + # simulate-principal-policy (returned: allowed). + # + # DEPLOY NOTE: this resource is added while the inline + # iam-role-management-boundary-gated policy is left in place. The two + # overlap by design — an explicit Deny beats an Allow anywhere in the + # policy set, so the escalation closes the moment this lands, with no + # window in which the role lacks its IAM permissions. The redundant + # inline copy is removed in a separate follow-up change. + # --------------------------------------------------------------------------- + SamCfnIamManagementPolicy: + Type: AWS::IAM::ManagedPolicy + Properties: + # Fixed name: changing it makes CloudFormation create a replacement policy + # and detach this one, which briefly drops the role's IAM permissions + # mid-update. Treat a rename as a coordinated migration, not an edit. This + # is the role's FIRST attached managed policy (per-role quota is 10). + ManagedPolicyName: seahaven-cfn-exec-iam-management + Description: >- + Boundary-gated IAM role lifecycle for github-cfn-execution-role, plus the + explicit Deny backstops that keep the permissions boundary from being + detached or rewritten. Separated from the role's inline policies to stay + under IAM's 10,240-byte inline limit. + PolicyDocument: + Version: "2012-10-17" + Statement: + # Create role — MUST attach boundary + - Sid: IAMCreateRoleWithBoundary + Effect: Allow + Action: + - iam:CreateRole + Resource: + - !Sub "arn:aws:iam::${AWS::AccountId}:role/*" + Condition: + StringEquals: + "iam:PermissionsBoundary": !Sub "arn:aws:iam::${AWS::AccountId}:policy/seahaven-lambda-execution-boundary" + + # Attach managed policies — MUST have boundary already on role + - Sid: IAMAttachPolicyWithBoundary + Effect: Allow + Action: + - iam:AttachRolePolicy + Resource: + - !Sub "arn:aws:iam::${AWS::AccountId}:role/*" + Condition: + StringEquals: + "iam:PermissionsBoundary": !Sub "arn:aws:iam::${AWS::AccountId}:policy/seahaven-lambda-execution-boundary" + + # Put inline policy — MUST have boundary already on role + - Sid: IAMPutRolePolicyWithBoundary + Effect: Allow + Action: + - iam:PutRolePolicy + Resource: + - !Sub "arn:aws:iam::${AWS::AccountId}:role/*" + Condition: + StringEquals: + "iam:PermissionsBoundary": !Sub "arn:aws:iam::${AWS::AccountId}:policy/seahaven-lambda-execution-boundary" + + # Boundary management — SET the boundary only. DELETE is NOT + # granted: for a delete, the iam:PermissionsBoundary condition key + # reflects the boundary CURRENTLY attached to the target role, so + # a StringEquals condition on the boundary ARN MATCHES exactly the + # roles the gate protects. Granting delete under that condition + # lets this role create a boundary-gated role with an inline *:* + # policy, strip the boundary, and pass the now-unbounded role to + # Lambda — defeating the primary escalation control. Verified live + # against the mgmt copy 2026-07-27 (simulate-principal-policy: + # iam:DeleteRolePermissionsBoundary = allowed). + # + # OPERATIONAL CONSEQUENCE — read before debugging a stuck stack. + # SAM does not need the delete for the common paths: it SETS the + # boundary on roles it creates, and stack teardown calls DeleteRole. + # But there IS one path that now fails by design: updating an + # existing AWS::IAM::Role to REMOVE its PermissionsBoundary property + # makes CloudFormation call DeleteRolePermissionsBoundary, which is + # denied. The stack update fails and rolls back, and because cd-sam's + # pre-flight hard-fails on *ROLLBACK_COMPLETE, that repo's deploys + # stay blocked until it is cleared. Recovery is an out-of-band admin + # action (remove the boundary directly, or replace the role by + # renaming its logical id) — not a pipeline retry. Removing the + # boundary from a SAM function is a security regression anyway, so + # failing loudly here is the intent. + - Sid: IAMPutPermissionsBoundary + Effect: Allow + Action: + - iam:PutRolePermissionsBoundary + Resource: + - !Sub "arn:aws:iam::${AWS::AccountId}:role/*" + Condition: + StringEquals: + "iam:PermissionsBoundary": !Sub "arn:aws:iam::${AWS::AccountId}:policy/seahaven-lambda-execution-boundary" + + # Explicit Deny backstop (AWS's documented NoBoundaryPolicyEdit / + # NoBoundaryDelete delegation pattern). A Deny is required, not + # merely omitting the Allow: without it, any future Allow added to + # this role — or a broader managed policy attached to it — silently + # reopens the escalation. Covers both removing a boundary from a + # role and rewriting the boundary POLICY DOCUMENT itself (the + # latter is only implicitly denied today). + - Sid: DenyBoundaryTampering + Effect: Deny + Action: + - iam:DeleteRolePermissionsBoundary + - iam:DeleteUserPermissionsBoundary + Resource: + - !Sub "arn:aws:iam::${AWS::AccountId}:role/*" + - !Sub "arn:aws:iam::${AWS::AccountId}:user/*" + + # Scoped to the whole seahaven-* policy family, not just the boundary: + # this policy carries the Deny statements, so it is now a + # higher-value target than the boundary it protects. Safe to scope + # broadly — the role holds no iam:CreatePolicy anywhere and no SAM + # stack manages a managed policy through it (both verified + # 2026-07-27), so nothing legitimate writes policy versions here. + - Sid: DenyBoundaryPolicyEdit + Effect: Deny + Action: + - iam:CreatePolicyVersion + - iam:SetDefaultPolicyVersion + - iam:DeletePolicyVersion + - iam:DeletePolicy + Resource: + - !Sub "arn:aws:iam::${AWS::AccountId}:policy/seahaven-*" + + # Self-protection. Without this the whole control is one API call + # from being undone: IAMRoleReadAndDelete below grants + # iam:DetachRolePolicy on Resource "*" with no condition, so this + # role could detach the very policy carrying these Denies from + # itself and reinstate the escalation. Verified live 2026-07-27: + # simulate-principal-policy returned "allowed" for DetachRolePolicy, + # DeleteRolePolicy and DeleteRole against this role's own ARN and + # against githubdeploy-* roles. + # + # Also closes a denial-of-service and a self-elevation precondition: + # iam:PutRolePermissionsBoundary is condition-pinned to the Lambda + # boundary ARN but NOT scoped by target, so this role could apply + # that runtime boundary to itself or to a githubdeploy-* role — + # bricking the pipelines, unrecoverable without an admin because + # removing a boundary is denied above, and making the otherwise-inert + # AttachRolePolicy/PutRolePolicy self-elevation conditions start + # matching. + # + # Costs nothing operationally: the deploy substrate's own roles are + # managed by THIS stack, which is deployed manually with + # administrator credentials (no --role-arn), so CloudFormation never + # exercises these actions against them as this role. SAM-generated + # roles are named -Role- and are unaffected. + - Sid: DenySelfMutation + Effect: Deny + Action: + - iam:AttachRolePolicy + - iam:DeleteRole + - iam:DeleteRolePolicy + - iam:DeleteRolePermissionsBoundary + - iam:DetachRolePolicy + - iam:PutRolePolicy + - iam:PutRolePermissionsBoundary + - iam:UpdateAssumeRolePolicy + - iam:UpdateRole + - iam:UpdateRoleDescription + Resource: + - !Sub "arn:aws:iam::${AWS::AccountId}:role/github-cfn-execution-role" + - !Sub "arn:aws:iam::${AWS::AccountId}:role/githubdeploy-*" + + # Read / tag / delete role and policy — no boundary condition needed + - Sid: IAMRoleReadAndDelete + Effect: Allow + Action: + - iam:DeleteRole + - iam:DeleteRolePolicy + - iam:DetachRolePolicy + - iam:GetRole + - iam:GetRolePolicy + - iam:ListAttachedRolePolicies + - iam:ListRolePolicies + - iam:ListRoles + - iam:TagRole + - iam:UntagRole + - iam:UpdateRole + - iam:UpdateRoleDescription + - iam:UpdateAssumeRolePolicy + - iam:GetPolicy + - iam:GetPolicyVersion + - iam:ListPolicies + - iam:ListPolicyVersions + Resource: "*" + + # PassRole — CloudFormation passes the Lambda execution role + # to the Lambda service. Scoped to SAM-generated role pattern. + - Sid: IAMPassRole + Effect: Allow + Action: + - iam:PassRole + Resource: + - !Sub "arn:aws:iam::${AWS::AccountId}:role/*" + Condition: + StringEquals: + "iam:PassedToService": "lambda.amazonaws.com" + # --------------------------------------------------------------------------- # Shared CloudFormation execution role (SAM stacks) — INFRA-97 scoped # @@ -268,15 +491,17 @@ Resources: # where we can control the path, path scoping can be added in a follow-up. # # DEPLOY ORDER DEPENDENCY - # This role references the boundary ARN by literal value. The boundary - # managed policy (seahaven-lambda-execution-boundary, INFRA-103) MUST - # exist before this stack is deployed. See PR description for the - # mandatory three-step deploy sequence. + # This role references the boundary ARN by literal value, so the + # seahaven-lambda-execution-boundary managed policy must exist before this + # stack is deployed. It is created by this same stack above, and is not + # modified by the Phase A change. # --------------------------------------------------------------------------- SamCfnExecutionRole: Type: AWS::IAM::Role Properties: RoleName: github-cfn-execution-role + ManagedPolicyArns: + - !Ref SamCfnIamManagementPolicy AssumeRolePolicyDocument: Version: "2012-10-17" Statement: @@ -474,6 +699,14 @@ Resources: - logs:DescribeDestinations - logs:AssociateKmsKey - logs:DisassociateKmsKey + # Reconciles drift: these three exist on the DEPLOYED role + # (added out-of-band 2026-06-29) but were never back-ported + # here. afterhours-shift-manager creates an + # AWS::Logs::MetricFilter through this role, so omitting them + # risks a future write-back silently stripping them. + - logs:PutMetricFilter + - logs:DeleteMetricFilter + - logs:DescribeMetricFilters Resource: "*" # ── EventBridge / CloudWatch Events (scheduled Lambdas) ───────────