fix(iam): close boundary-removal escalation in github-cfn-execution-role (Phase A)

The shared CloudFormation execution role could remove the permissions
boundary from the very roles that boundary was gating. Its
iam-role-management-boundary-gated policy allows
iam:DeleteRolePermissionsBoundary on role/* under a StringEquals
condition on iam:PermissionsBoundary -- and for a delete that condition
key reflects the boundary CURRENTLY attached to the target role, so it
matches exactly the roles the gate protects. Create a boundary-gated
role with an inline *:* policy, strip its boundary, PassRole it to
Lambda, and the result is unbounded admin in the management account.
Confirmed live with simulate-principal-policy, not inferred.

Phase A adds an attached managed policy, seahaven-cfn-exec-iam-management,
carrying the corrected statement set: the boundary-gated Allows without
iam:DeleteRolePermissionsBoundary, plus three Deny backstops --
DenyBoundaryTampering (boundary removal), DenyBoundaryPolicyEdit
(rewriting a seahaven-* policy document) and DenySelfMutation.

DenySelfMutation exists because the first draft of this fix was not
durable: the role holds iam:DetachRolePolicy, iam:DeleteRolePolicy and
iam:DeleteRole on Resource "*" with no condition, so it could detach the
Deny-carrying policy from itself in one call and reinstate the
escalation. It now cannot modify its own role, any githubdeploy-* role,
or any seahaven-* policy. Nothing legitimate needs that: the deploy
substrate's own principals are owned by this stack, which is deployed
manually with administrator credentials rather than through this role.

The change is additive. The old inline policy stays in place, so
CloudFormation removes nothing and there is no window in which the role
lacks its IAM permissions -- an explicit Deny beats an Allow anywhere in
the policy set, so the corrected version governs from the moment this
lands. Phase B removes the redundant inline copy. The split is also
required by size: inline sits at 10,006 of IAM's 10,240-byte per-role
limit, and the Deny statements do not fit there.

Also reconciles drift. The deployed role carries three logs:*MetricFilter
actions added out-of-band on 2026-06-29 and never back-ported.
afterhours-shift-manager creates an AWS::Logs::MetricFilter through this
role, so they are load-bearing; the template now matches the live policy
exactly, which keeps inline at 10,006 and stops a future write-back from
silently stripping them.
This commit is contained in:
Adam Moussa 2026-07-27 17:44:58 -04:00
parent 0df5ee3955
commit f0a2b7191d
No known key found for this signature in database
2 changed files with 253 additions and 4 deletions

View file

@ -62,6 +62,11 @@ PR reviews are handled by the **official Claude Code GitHub App** (installed org
- One **OIDC deploy role per repo** (`githubdeploy-<repo>`), assumed by that repo's `deploy.yaml` via OIDC and passed in as `AWS_DEPLOY_ROLE_ARN`. CDK repos use these to assume the `cdk-hnb659fds-*` bootstrap roles; SAM repos use these to run `sam deploy`. - One **OIDC deploy role per repo** (`githubdeploy-<repo>`), assumed by that repo's `deploy.yaml` via OIDC and passed in as `AWS_DEPLOY_ROLE_ARN`. CDK repos use these to assume the `cdk-hnb659fds-*` bootstrap roles; SAM repos use these to run `sam deploy`.
- The shared **SAM CloudFormation execution role** `github-cfn-execution-role` (`SamCfnExecutionRole`) — passed as `cfn-role-arn` by every SAM `deploy.yaml` (see §5). CloudFormation assumes it to provision the SAM stacks' resources. - The shared **SAM CloudFormation execution role** `github-cfn-execution-role` (`SamCfnExecutionRole`) — passed as `cfn-role-arn` by every SAM `deploy.yaml` (see §5). CloudFormation assumes it to provision the SAM stacks' resources.
- The **`seahaven-lambda-execution-boundary`** managed policy. - The **`seahaven-lambda-execution-boundary`** managed policy.
- The **`seahaven-cfn-exec-iam-management`** managed policy (`SamCfnIamManagementPolicy`), attached to `github-cfn-execution-role`. It holds that role's boundary-gated IAM statements plus the Deny backstops that keep the permissions boundary from being detached, rewritten, or applied to the deploy substrate's own roles. It lives in a managed policy rather than inline because the role's inline policies sit at 10,006 of IAM's hard 10,240-byte per-role limit; attached managed policies have a separate 6,144-byte budget.
> **Constraint for future maintainers.** `github-cfn-execution-role` is explicitly denied from mutating the deploy substrate's own principals — itself, any `githubdeploy-*` role, and any `seahaven-*` managed policy. Those are owned by this stack and deployed manually with administrator credentials, so nothing legitimate needs that path. If you ever add automation that manages one of them, it must not run through `github-cfn-execution-role` or it will fail with `AccessDenied`.
> **Phase A / Phase B.** The boundary-gated statements are currently duplicated: the new managed policy carries the corrected set, and the older inline `iam-role-management-boundary-gated` policy is still present. That overlap is deliberate and temporary — an explicit Deny beats an Allow anywhere in the policy set, so the corrected version already governs, and keeping the inline copy meant CloudFormation removed nothing during the change. **Phase B deletes the inline copy** (inline usage 10,006 → 8,261). Do not delete it as "redundant" outside that planned change.
> ⚠️ **This stack has no CD pipeline — it is deployed manually.** (It defines the very roles the pipelines use, so it can't deploy itself.) > ⚠️ **This stack has no CD pipeline — it is deployed manually.** (It defines the very roles the pipelines use, so it can't deploy itself.)
@ -103,6 +108,17 @@ A function's effective permissions are the **intersection** of its own role poli
Wrong order breaks every SAM deploy. (History: INFRA-103 established the boundary, INFRA-97 scoped the role.) CDK repos are unaffected — they deploy via `cdk-hnb659fds-*` roles, not this execution role. Wrong order breaks every SAM deploy. (History: INFRA-103 established the boundary, INFRA-97 scoped the role.) CDK repos are unaffected — they deploy via `cdk-hnb659fds-*` roles, not this execution role.
This ordering rule is about changing the **boundary** or the conditions that gate it. It does not apply to changes that only add permissions to the exec role.
#### Permissions boundaries can no longer be removed by CloudFormation
`github-cfn-execution-role` is explicitly denied `iam:DeleteRolePermissionsBoundary`. It can *set* the boundary (that is what SAM needs) but never remove one. Two consequences worth knowing before debugging a stuck stack:
- **Removing `PermissionsBoundary` from an existing role fails by design.** CloudFormation issues `DeleteRolePermissionsBoundary` for that edit, gets `AccessDenied`, and the stack update rolls back. Removing the boundary from a SAM function is a security regression, so failing loudly is intended.
- **Rollback of an update that *adds* a boundary to an existing role would also fail**, landing the stack in `UPDATE_ROLLBACK_FAILED`. This is currently unreachable — all 26 IAM roles across the five SAM stacks already carry the boundary (verified 2026-07-27), so no update can add one. It becomes reachable again only if a role is created without the boundary and given one later.
Recovery in either case is an administrator action, not a pipeline retry: clear the wedged stack with `aws cloudformation continue-update-rollback --stack-name <stack> --resources-to-skip <RoleLogicalId>`, or replace the role by renaming its logical id. Note `cd-sam`'s pre-flight hard-fails on `*ROLLBACK_COMPLETE`, so that repo's deploys stay blocked until it is cleared.
## Setup ## Setup
### 1. Create a GitHub App ### 1. Create a GitHub App

View file

@ -241,6 +241,229 @@ Resources:
Resource: Resource:
- !Sub "arn:aws:kms:us-east-1:${AWS::AccountId}:key/*" - !Sub "arn:aws:kms:us-east-1:${AWS::AccountId}:key/*"
# ---------------------------------------------------------------------------
# IAM role lifecycle for the CFN execution role — BOUNDARY-GATED
# (attached managed policy)
#
# Ported verbatim from seahaven-org-baseline
# lib/deploy-substrate/deploy-substrate.template.yaml (stack
# seahaven-deploy-substrate, deployed and verified in seahaven-prod and
# seahaven-dev 2026-07-27). Keep the two copies in lockstep.
#
# Why a MANAGED policy and not inline on the role: this role's inline
# policies total ~10,006 bytes against IAM's hard 10,240-byte per-role
# inline limit — about 234 bytes of headroom. The Deny statements below
# do not fit inline (the equivalent attempt in prod/dev failed with
# ServiceLimitExceeded). Attached managed policies carry their own
# separate 6,144-byte budget.
#
# SECURITY: iam:DeleteRolePermissionsBoundary is deliberately ABSENT from
# the Allow below, and explicitly Denied further down. Granting it under
# the StringEquals iam:PermissionsBoundary condition is self-defeating:
# for a delete, that condition key reflects the boundary CURRENTLY
# attached to the target role, so it matches exactly the roles the gate
# protects — letting this role create a boundary-gated role with an
# inline *:* policy, strip the boundary, and pass the now-unbounded role
# to Lambda. Verified live on this very role 2026-07-27 via
# simulate-principal-policy (returned: allowed).
#
# DEPLOY NOTE: this resource is added while the inline
# iam-role-management-boundary-gated policy is left in place. The two
# overlap by design — an explicit Deny beats an Allow anywhere in the
# policy set, so the escalation closes the moment this lands, with no
# window in which the role lacks its IAM permissions. The redundant
# inline copy is removed in a separate follow-up change.
# ---------------------------------------------------------------------------
SamCfnIamManagementPolicy:
Type: AWS::IAM::ManagedPolicy
Properties:
# Fixed name: changing it makes CloudFormation create a replacement policy
# and detach this one, which briefly drops the role's IAM permissions
# mid-update. Treat a rename as a coordinated migration, not an edit. This
# is the role's FIRST attached managed policy (per-role quota is 10).
ManagedPolicyName: seahaven-cfn-exec-iam-management
Description: >-
Boundary-gated IAM role lifecycle for github-cfn-execution-role, plus the
explicit Deny backstops that keep the permissions boundary from being
detached or rewritten. Separated from the role's inline policies to stay
under IAM's 10,240-byte inline limit.
PolicyDocument:
Version: "2012-10-17"
Statement:
# Create role — MUST attach boundary
- Sid: IAMCreateRoleWithBoundary
Effect: Allow
Action:
- iam:CreateRole
Resource:
- !Sub "arn:aws:iam::${AWS::AccountId}:role/*"
Condition:
StringEquals:
"iam:PermissionsBoundary": !Sub "arn:aws:iam::${AWS::AccountId}:policy/seahaven-lambda-execution-boundary"
# Attach managed policies — MUST have boundary already on role
- Sid: IAMAttachPolicyWithBoundary
Effect: Allow
Action:
- iam:AttachRolePolicy
Resource:
- !Sub "arn:aws:iam::${AWS::AccountId}:role/*"
Condition:
StringEquals:
"iam:PermissionsBoundary": !Sub "arn:aws:iam::${AWS::AccountId}:policy/seahaven-lambda-execution-boundary"
# Put inline policy — MUST have boundary already on role
- Sid: IAMPutRolePolicyWithBoundary
Effect: Allow
Action:
- iam:PutRolePolicy
Resource:
- !Sub "arn:aws:iam::${AWS::AccountId}:role/*"
Condition:
StringEquals:
"iam:PermissionsBoundary": !Sub "arn:aws:iam::${AWS::AccountId}:policy/seahaven-lambda-execution-boundary"
# Boundary management — SET the boundary only. DELETE is NOT
# granted: for a delete, the iam:PermissionsBoundary condition key
# reflects the boundary CURRENTLY attached to the target role, so
# a StringEquals condition on the boundary ARN MATCHES exactly the
# roles the gate protects. Granting delete under that condition
# lets this role create a boundary-gated role with an inline *:*
# policy, strip the boundary, and pass the now-unbounded role to
# Lambda — defeating the primary escalation control. Verified live
# against the mgmt copy 2026-07-27 (simulate-principal-policy:
# iam:DeleteRolePermissionsBoundary = allowed).
#
# OPERATIONAL CONSEQUENCE — read before debugging a stuck stack.
# SAM does not need the delete for the common paths: it SETS the
# boundary on roles it creates, and stack teardown calls DeleteRole.
# But there IS one path that now fails by design: updating an
# existing AWS::IAM::Role to REMOVE its PermissionsBoundary property
# makes CloudFormation call DeleteRolePermissionsBoundary, which is
# denied. The stack update fails and rolls back, and because cd-sam's
# pre-flight hard-fails on *ROLLBACK_COMPLETE, that repo's deploys
# stay blocked until it is cleared. Recovery is an out-of-band admin
# action (remove the boundary directly, or replace the role by
# renaming its logical id) — not a pipeline retry. Removing the
# boundary from a SAM function is a security regression anyway, so
# failing loudly here is the intent.
- Sid: IAMPutPermissionsBoundary
Effect: Allow
Action:
- iam:PutRolePermissionsBoundary
Resource:
- !Sub "arn:aws:iam::${AWS::AccountId}:role/*"
Condition:
StringEquals:
"iam:PermissionsBoundary": !Sub "arn:aws:iam::${AWS::AccountId}:policy/seahaven-lambda-execution-boundary"
# Explicit Deny backstop (AWS's documented NoBoundaryPolicyEdit /
# NoBoundaryDelete delegation pattern). A Deny is required, not
# merely omitting the Allow: without it, any future Allow added to
# this role — or a broader managed policy attached to it — silently
# reopens the escalation. Covers both removing a boundary from a
# role and rewriting the boundary POLICY DOCUMENT itself (the
# latter is only implicitly denied today).
- Sid: DenyBoundaryTampering
Effect: Deny
Action:
- iam:DeleteRolePermissionsBoundary
- iam:DeleteUserPermissionsBoundary
Resource:
- !Sub "arn:aws:iam::${AWS::AccountId}:role/*"
- !Sub "arn:aws:iam::${AWS::AccountId}:user/*"
# Scoped to the whole seahaven-* policy family, not just the boundary:
# this policy carries the Deny statements, so it is now a
# higher-value target than the boundary it protects. Safe to scope
# broadly — the role holds no iam:CreatePolicy anywhere and no SAM
# stack manages a managed policy through it (both verified
# 2026-07-27), so nothing legitimate writes policy versions here.
- Sid: DenyBoundaryPolicyEdit
Effect: Deny
Action:
- iam:CreatePolicyVersion
- iam:SetDefaultPolicyVersion
- iam:DeletePolicyVersion
- iam:DeletePolicy
Resource:
- !Sub "arn:aws:iam::${AWS::AccountId}:policy/seahaven-*"
# Self-protection. Without this the whole control is one API call
# from being undone: IAMRoleReadAndDelete below grants
# iam:DetachRolePolicy on Resource "*" with no condition, so this
# role could detach the very policy carrying these Denies from
# itself and reinstate the escalation. Verified live 2026-07-27:
# simulate-principal-policy returned "allowed" for DetachRolePolicy,
# DeleteRolePolicy and DeleteRole against this role's own ARN and
# against githubdeploy-* roles.
#
# Also closes a denial-of-service and a self-elevation precondition:
# iam:PutRolePermissionsBoundary is condition-pinned to the Lambda
# boundary ARN but NOT scoped by target, so this role could apply
# that runtime boundary to itself or to a githubdeploy-* role —
# bricking the pipelines, unrecoverable without an admin because
# removing a boundary is denied above, and making the otherwise-inert
# AttachRolePolicy/PutRolePolicy self-elevation conditions start
# matching.
#
# Costs nothing operationally: the deploy substrate's own roles are
# managed by THIS stack, which is deployed manually with
# administrator credentials (no --role-arn), so CloudFormation never
# exercises these actions against them as this role. SAM-generated
# roles are named <stack>-<Function>Role-<hash> and are unaffected.
- Sid: DenySelfMutation
Effect: Deny
Action:
- iam:AttachRolePolicy
- iam:DeleteRole
- iam:DeleteRolePolicy
- iam:DeleteRolePermissionsBoundary
- iam:DetachRolePolicy
- iam:PutRolePolicy
- iam:PutRolePermissionsBoundary
- iam:UpdateAssumeRolePolicy
- iam:UpdateRole
- iam:UpdateRoleDescription
Resource:
- !Sub "arn:aws:iam::${AWS::AccountId}:role/github-cfn-execution-role"
- !Sub "arn:aws:iam::${AWS::AccountId}:role/githubdeploy-*"
# Read / tag / delete role and policy — no boundary condition needed
- Sid: IAMRoleReadAndDelete
Effect: Allow
Action:
- iam:DeleteRole
- iam:DeleteRolePolicy
- iam:DetachRolePolicy
- iam:GetRole
- iam:GetRolePolicy
- iam:ListAttachedRolePolicies
- iam:ListRolePolicies
- iam:ListRoles
- iam:TagRole
- iam:UntagRole
- iam:UpdateRole
- iam:UpdateRoleDescription
- iam:UpdateAssumeRolePolicy
- iam:GetPolicy
- iam:GetPolicyVersion
- iam:ListPolicies
- iam:ListPolicyVersions
Resource: "*"
# PassRole — CloudFormation passes the Lambda execution role
# to the Lambda service. Scoped to SAM-generated role pattern.
- Sid: IAMPassRole
Effect: Allow
Action:
- iam:PassRole
Resource:
- !Sub "arn:aws:iam::${AWS::AccountId}:role/*"
Condition:
StringEquals:
"iam:PassedToService": "lambda.amazonaws.com"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Shared CloudFormation execution role (SAM stacks) — INFRA-97 scoped # Shared CloudFormation execution role (SAM stacks) — INFRA-97 scoped
# #
@ -268,15 +491,17 @@ Resources:
# where we can control the path, path scoping can be added in a follow-up. # where we can control the path, path scoping can be added in a follow-up.
# #
# DEPLOY ORDER DEPENDENCY # DEPLOY ORDER DEPENDENCY
# This role references the boundary ARN by literal value. The boundary # This role references the boundary ARN by literal value, so the
# managed policy (seahaven-lambda-execution-boundary, INFRA-103) MUST # seahaven-lambda-execution-boundary managed policy must exist before this
# exist before this stack is deployed. See PR description for the # stack is deployed. It is created by this same stack above, and is not
# mandatory three-step deploy sequence. # modified by the Phase A change.
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
SamCfnExecutionRole: SamCfnExecutionRole:
Type: AWS::IAM::Role Type: AWS::IAM::Role
Properties: Properties:
RoleName: github-cfn-execution-role RoleName: github-cfn-execution-role
ManagedPolicyArns:
- !Ref SamCfnIamManagementPolicy
AssumeRolePolicyDocument: AssumeRolePolicyDocument:
Version: "2012-10-17" Version: "2012-10-17"
Statement: Statement:
@ -474,6 +699,14 @@ Resources:
- logs:DescribeDestinations - logs:DescribeDestinations
- logs:AssociateKmsKey - logs:AssociateKmsKey
- logs:DisassociateKmsKey - logs:DisassociateKmsKey
# Reconciles drift: these three exist on the DEPLOYED role
# (added out-of-band 2026-06-29) but were never back-ported
# here. afterhours-shift-manager creates an
# AWS::Logs::MetricFilter through this role, so omitting them
# risks a future write-back silently stripping them.
- logs:PutMetricFilter
- logs:DeleteMetricFilter
- logs:DescribeMetricFilters
Resource: "*" Resource: "*"
# ── EventBridge / CloudWatch Events (scheduled Lambdas) ─────────── # ── EventBridge / CloudWatch Events (scheduled Lambdas) ───────────