mirror of
https://github.com/Sea-Haven-Industries/.github.git
synced 2026-09-30 04:43:12 +00:00
fix(deploy): recover SAM stacks after update rollback
This commit is contained in:
parent
9a7171a855
commit
81cf168170
3 changed files with 25 additions and 25 deletions
2
.github/workflows/cd-sam.yaml
vendored
2
.github/workflows/cd-sam.yaml
vendored
|
|
@ -69,7 +69,7 @@ jobs:
|
||||||
--stack-name "${{ inputs.stack-name }}" \
|
--stack-name "${{ inputs.stack-name }}" \
|
||||||
--query 'Stacks[0].StackStatus' --output text 2>/dev/null || echo "NOT_FOUND")
|
--query 'Stacks[0].StackStatus' --output text 2>/dev/null || echo "NOT_FOUND")
|
||||||
case "$STATUS" in
|
case "$STATUS" in
|
||||||
*ROLLBACK_COMPLETE|*FAILED)
|
ROLLBACK_COMPLETE|*FAILED)
|
||||||
echo "::error::Stack ${{ inputs.stack-name }} is in $STATUS — manual intervention required."
|
echo "::error::Stack ${{ inputs.stack-name }} is in $STATUS — manual intervention required."
|
||||||
exit 1
|
exit 1
|
||||||
;;
|
;;
|
||||||
|
|
|
||||||
|
|
@ -121,7 +121,7 @@ This ordering rule is about changing the **boundary** or the conditions that gat
|
||||||
- **Removing `PermissionsBoundary` from an existing role fails by design.** CloudFormation issues `DeleteRolePermissionsBoundary` for that edit, gets `AccessDenied`, and the stack update rolls back. Removing the boundary from a SAM function is a security regression, so failing loudly is intended.
|
- **Removing `PermissionsBoundary` from an existing role fails by design.** CloudFormation issues `DeleteRolePermissionsBoundary` for that edit, gets `AccessDenied`, and the stack update rolls back. Removing the boundary from a SAM function is a security regression, so failing loudly is intended.
|
||||||
- **Rollback of an update that *adds* a boundary to an existing role would also fail**, landing the stack in `UPDATE_ROLLBACK_FAILED`. This is currently unreachable — all 26 IAM roles across the five SAM stacks already carry the boundary (verified 2026-07-27), so no update can add one. It becomes reachable again only if a role is created without the boundary and given one later.
|
- **Rollback of an update that *adds* a boundary to an existing role would also fail**, landing the stack in `UPDATE_ROLLBACK_FAILED`. This is currently unreachable — all 26 IAM roles across the five SAM stacks already carry the boundary (verified 2026-07-27), so no update can add one. It becomes reachable again only if a role is created without the boundary and given one later.
|
||||||
|
|
||||||
Recovery in either case is an administrator action, not a pipeline retry: clear the wedged stack with `aws cloudformation continue-update-rollback --stack-name <stack> --resources-to-skip <RoleLogicalId>`, or replace the role by renaming its logical id. Note `cd-sam`'s pre-flight hard-fails on `*ROLLBACK_COMPLETE`, so that repo's deploys stay blocked until it is cleared.
|
Recovery from `UPDATE_ROLLBACK_FAILED` is an administrator action, not a pipeline retry: clear the wedged stack with `aws cloudformation continue-update-rollback --stack-name <stack> --resources-to-skip <RoleLogicalId>`, or replace the role by renaming its logical id. A completed update rollback lands in `UPDATE_ROLLBACK_COMPLETE`, which is stable and can accept a corrective update; `cd-sam` blocks only first-create `ROLLBACK_COMPLETE` and failed or in-progress states.
|
||||||
|
|
||||||
## Setup
|
## Setup
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -338,30 +338,17 @@ Resources:
|
||||||
StringEquals:
|
StringEquals:
|
||||||
"iam:PermissionsBoundary": !Sub "arn:aws:iam::${AWS::AccountId}:policy/seahaven-lambda-execution-boundary"
|
"iam:PermissionsBoundary": !Sub "arn:aws:iam::${AWS::AccountId}:policy/seahaven-lambda-execution-boundary"
|
||||||
|
|
||||||
# Boundary management — SET the boundary only. DELETE is NOT
|
# Boundary management is SET-only. Granting delete would let this
|
||||||
# granted: for a delete, the iam:PermissionsBoundary condition key
|
# role create a boundary-gated role, strip the boundary, then pass an
|
||||||
# reflects the boundary CURRENTLY attached to the target role, so
|
# unconstrained role to Lambda. SAM creation and teardown need only
|
||||||
# a StringEquals condition on the boundary ARN MATCHES exactly the
|
# PutRolePermissionsBoundary and DeleteRole.
|
||||||
# roles the gate protects. Granting delete under that condition
|
|
||||||
# lets this role create a boundary-gated role with an inline *:*
|
|
||||||
# policy, strip the boundary, and pass the now-unbounded role to
|
|
||||||
# Lambda — defeating the primary escalation control. Verified live
|
|
||||||
# against the mgmt copy 2026-07-27 (simulate-principal-policy:
|
|
||||||
# iam:DeleteRolePermissionsBoundary = allowed).
|
|
||||||
#
|
#
|
||||||
# OPERATIONAL CONSEQUENCE — read before debugging a stuck stack.
|
# Removing a boundary therefore fails by design. A failed rollback
|
||||||
# SAM does not need the delete for the common paths: it SETS the
|
# reaches UPDATE_ROLLBACK_FAILED and needs admin recovery by skipping
|
||||||
# boundary on roles it creates, and stack teardown calls DeleteRole.
|
# or replacing the role. A successful rollback reaches the stable
|
||||||
# But there IS one path that now fails by design: updating an
|
# UPDATE_ROLLBACK_COMPLETE state and can accept a corrective update.
|
||||||
# existing AWS::IAM::Role to REMOVE its PermissionsBoundary property
|
# Removing a SAM function boundary is a security regression, so
|
||||||
# makes CloudFormation call DeleteRolePermissionsBoundary, which is
|
# failing loudly is intentional.
|
||||||
# denied. The stack update fails and rolls back, and because cd-sam's
|
|
||||||
# pre-flight hard-fails on *ROLLBACK_COMPLETE, that repo's deploys
|
|
||||||
# stay blocked until it is cleared. Recovery is an out-of-band admin
|
|
||||||
# action (remove the boundary directly, or replace the role by
|
|
||||||
# renaming its logical id) — not a pipeline retry. Removing the
|
|
||||||
# boundary from a SAM function is a security regression anyway, so
|
|
||||||
# failing loudly here is the intent.
|
|
||||||
- Sid: IAMPutPermissionsBoundary
|
- Sid: IAMPutPermissionsBoundary
|
||||||
Effect: Allow
|
Effect: Allow
|
||||||
Action:
|
Action:
|
||||||
|
|
@ -479,6 +466,19 @@ Resources:
|
||||||
StringEquals:
|
StringEquals:
|
||||||
"iam:PassedToService": "lambda.amazonaws.com"
|
"iam:PassedToService": "lambda.amazonaws.com"
|
||||||
|
|
||||||
|
# API Gateway assumes SAM authorizer invocation roles. Keep this
|
||||||
|
# separate from Lambda PassRole so each target service and role
|
||||||
|
# pattern remains independently constrained.
|
||||||
|
- Sid: IAMPassAuthorizerRole
|
||||||
|
Effect: Allow
|
||||||
|
Action:
|
||||||
|
- iam:PassRole
|
||||||
|
Resource:
|
||||||
|
- !Sub "arn:aws:iam::${AWS::AccountId}:role/cfn-managed/*AuthorizerInvokeRole-*"
|
||||||
|
Condition:
|
||||||
|
StringEquals:
|
||||||
|
"iam:PassedToService": "apigateway.amazonaws.com"
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Shared CloudFormation execution role (SAM stacks) — INFRA-97 scoped
|
# Shared CloudFormation execution role (SAM stacks) — INFRA-97 scoped
|
||||||
#
|
#
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue