mirror of
https://github.com/Sea-Haven-Industries/afterhours-shift-manager.git
synced 2026-09-30 05:33:12 +00:00
The weekly-post pay-summary email to payroll failed with SES AccessDenied every Monday since v1.10.1: the role granted ses:SendEmail on identity/noreply@seahaven.com, but that address is not a verified SES identity — it is covered by the verified domain identity seahaven.com, which is what SES authorizes against. Grant the domain ARN instead. Pin the grant with a ses:FromAddress condition (= noreply@seahaven.com, the existing SES_SENDER) so the domain-wide identity can't be used to send-as any other @seahaven.com mailbox (BEC blast radius). Surfaced by /sh-security-review; matches the existing single-sender intent. Add a CloudWatch metric-filter alarm on the swallowed "Failed to send pay summary" log line -> site-alerts. The email send is wrapped in try/except so a delivery failure never increments the Lambda Errors metric; this is the only signal that surfaces a silent payroll failure. Closes #142
This commit is contained in:
parent
254f6b989f
commit
f7c44778b5
3 changed files with 71 additions and 3 deletions
|
|
@ -10,6 +10,15 @@ fine and still supported.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## v1.13.1 — June 29, 2026
|
||||||
|
|
||||||
|
**Fixed: the weekly pay summary email to payroll is sending again.** A
|
||||||
|
permissions change had quietly been blocking the Monday pay-summary email to
|
||||||
|
payroll since mid-June. The Slack pay report and the schedule post were never
|
||||||
|
affected — only the email to payroll. That's now fixed, the two missed weeks
|
||||||
|
were re-sent to payroll, and we've added an alert so a future email failure
|
||||||
|
can't slip by unnoticed.
|
||||||
|
|
||||||
## v1.13.0 — June 27, 2026
|
## v1.13.0 — June 27, 2026
|
||||||
|
|
||||||
**The two-week schedule post now sticks to the bottom of the channel.** It used
|
**The two-week schedule post now sticks to the bottom of the channel.** It used
|
||||||
|
|
|
||||||
|
|
@ -10,6 +10,15 @@ fine and still supported.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## v1.13.1 — June 29, 2026
|
||||||
|
|
||||||
|
**Fixed: the weekly pay summary email to payroll is sending again.** A
|
||||||
|
permissions change had quietly been blocking the Monday pay-summary email to
|
||||||
|
payroll since mid-June. The Slack pay report and the schedule post were never
|
||||||
|
affected — only the email to payroll. That's now fixed, the two missed weeks
|
||||||
|
were re-sent to payroll, and we've added an alert so a future email failure
|
||||||
|
can't slip by unnoticed.
|
||||||
|
|
||||||
## v1.13.0 — June 27, 2026
|
## v1.13.0 — June 27, 2026
|
||||||
|
|
||||||
**The two-week schedule post now sticks to the bottom of the channel.** It used
|
**The two-week schedule post now sticks to the bottom of the channel.** It used
|
||||||
|
|
|
||||||
|
|
@ -166,14 +166,26 @@ Resources:
|
||||||
Action:
|
Action:
|
||||||
- ses:SendEmail
|
- ses:SendEmail
|
||||||
Resource:
|
Resource:
|
||||||
# Only the single sending identity (noreply@seahaven.com), not
|
# The From address (noreply@seahaven.com) is NOT a standalone
|
||||||
# every identity in the account.
|
# verified SES identity — it is covered by the verified DOMAIN
|
||||||
- !Sub "arn:aws:ses:${AWS::Region}:${AWS::AccountId}:identity/noreply@seahaven.com"
|
# identity seahaven.com, which is the identity SES authorizes
|
||||||
|
# SendEmail against. Granting identity/noreply@seahaven.com (a
|
||||||
|
# non-existent identity) caused AccessDenied; the domain ARN is
|
||||||
|
# the correct least-privilege resource. Scoped to this one domain,
|
||||||
|
# not identity/* (every identity in the account).
|
||||||
|
- !Sub "arn:aws:ses:${AWS::Region}:${AWS::AccountId}:identity/seahaven.com"
|
||||||
# The sending identity has a default configuration set
|
# The sending identity has a default configuration set
|
||||||
# (seahaven-email-events); SES authorizes SendEmail against the
|
# (seahaven-email-events); SES authorizes SendEmail against the
|
||||||
# config-set resource too, so it must be granted alongside the
|
# config-set resource too, so it must be granted alongside the
|
||||||
# identity or the send is denied. Scoped to the known set name.
|
# identity or the send is denied. Scoped to the known set name.
|
||||||
- !Sub "arn:aws:ses:${AWS::Region}:${AWS::AccountId}:configuration-set/seahaven-email-events"
|
- !Sub "arn:aws:ses:${AWS::Region}:${AWS::AccountId}:configuration-set/seahaven-email-events"
|
||||||
|
# The domain identity covers every address under seahaven.com, so
|
||||||
|
# without this the role could send-as any @seahaven.com mailbox.
|
||||||
|
# Pin the From to the one legitimate sender (matches SES_SENDER) so
|
||||||
|
# a compromised function can't spoof internal senders (BEC).
|
||||||
|
Condition:
|
||||||
|
StringEquals:
|
||||||
|
ses:FromAddress: noreply@seahaven.com
|
||||||
Events:
|
Events:
|
||||||
# EST: 7am ET = 12:00 UTC (Nov-Mar)
|
# EST: 7am ET = 12:00 UTC (Nov-Mar)
|
||||||
WeeklyPostEST:
|
WeeklyPostEST:
|
||||||
|
|
@ -551,6 +563,44 @@ Resources:
|
||||||
AlarmActions:
|
AlarmActions:
|
||||||
- !Sub "arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts"
|
- !Sub "arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts"
|
||||||
|
|
||||||
|
# --- Payroll email delivery failure (log-metric alarm) ---
|
||||||
|
# The pay-summary email send is wrapped in try/except so a delivery failure
|
||||||
|
# never aborts the schedule post — which means it does NOT surface on the
|
||||||
|
# Lambda Errors metric above (the handler exits "success"). This filter turns
|
||||||
|
# the "Failed to send pay summary" log line into a metric so a silent payroll
|
||||||
|
# delivery failure still pages site-alerts. DefaultValue 0 keeps the metric
|
||||||
|
# reporting on every run so the alarm sits in OK, not INSUFFICIENT_DATA.
|
||||||
|
PayrollEmailFailureMetricFilter:
|
||||||
|
Type: AWS::Logs::MetricFilter
|
||||||
|
Properties:
|
||||||
|
# !Ref the stack-managed log group (not a literal name) so CloudFormation
|
||||||
|
# orders this filter after the group exists.
|
||||||
|
LogGroupName: !Ref WeeklyPostLogGroup
|
||||||
|
FilterPattern: '"Failed to send pay summary"'
|
||||||
|
MetricTransformations:
|
||||||
|
- MetricName: PayrollEmailFailures
|
||||||
|
MetricNamespace: AfterHours/WeeklyPost
|
||||||
|
MetricValue: "1"
|
||||||
|
# DefaultValue 0 means non-matching runs emit 0, so the metric stays
|
||||||
|
# populated and the alarm rests in OK rather than INSUFFICIENT_DATA.
|
||||||
|
DefaultValue: 0
|
||||||
|
|
||||||
|
PayrollEmailFailureAlarm:
|
||||||
|
Type: AWS::CloudWatch::Alarm
|
||||||
|
Properties:
|
||||||
|
AlarmName: PayrollEmailFailure-afterhours-weekly-post
|
||||||
|
AlarmDescription: "Weekly-post failed to email the pay summary to payroll (SES send error)"
|
||||||
|
Namespace: AfterHours/WeeklyPost
|
||||||
|
MetricName: PayrollEmailFailures
|
||||||
|
Statistic: Sum
|
||||||
|
Period: 300
|
||||||
|
EvaluationPeriods: 1
|
||||||
|
Threshold: 1
|
||||||
|
ComparisonOperator: GreaterThanOrEqualToThreshold
|
||||||
|
TreatMissingData: notBreaching
|
||||||
|
AlarmActions:
|
||||||
|
- !Sub "arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts"
|
||||||
|
|
||||||
# --- Lambda Duration alarms (Maximum, ms; 2-of-3 evaluation) ---
|
# --- Lambda Duration alarms (Maximum, ms; 2-of-3 evaluation) ---
|
||||||
# Thresholds set to ~80% of each function's timeout, with 2-of-3 evaluation.
|
# Thresholds set to ~80% of each function's timeout, with 2-of-3 evaluation.
|
||||||
# Timeouts: slack-bot/weekly-post/release-notifier = 30s (global default);
|
# Timeouts: slack-bot/weekly-post/release-notifier = 30s (global default);
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue