diff --git a/README.md b/README.md index b1d95c6..d4392a5 100644 --- a/README.md +++ b/README.md @@ -5,11 +5,10 @@ ![Slack](https://img.shields.io/badge/Slack-integration-4A154B?logo=slack&logoColor=white) ![CI](https://github.com/Sea-Haven-Industries/payments-dashboard/actions/workflows/ci.yaml/badge.svg) -AWS SAM application that ingests payment CSVs, syncs check data with Bank of America CashPro APIs, processes Gusto payroll confirmation emails into Slack notifications, surfaces an outstanding-payments dashboard in Slack, and routes expense approvals through a Slack reaction-driven workflow. +AWS SAM application that ingests payment CSVs, syncs check data with Bank of America CashPro APIs, surfaces an outstanding-payments dashboard in Slack, and routes expense approvals through a Slack reaction-driven workflow. ## Architecture -- **ProcessPayrollEmail** — Lambda triggered by S3 (inbound email) and SQS (batch timer). SES receives Gusto payroll emails at `payroll@int.seahaven.com`, stores them to S3, and this Lambda parses the email body, extracts financial data, and posts a combined Slack notification (employee payroll + contractor payments) after a 10-minute batching window. Runs outside VPC. - **ProcessPaymentCsv** — Lambda triggered by S3 CSV upload. Parses Stampli payment exports, upserts to DynamoDB, and submits new/cancelled checks to the CashPro Check Management API. - **FetchBoaTransactions** — Scheduled Lambda. Weekdays 9am ET it calls the CashPro **previous-day** Transaction Inquiry (authoritative sweep, trailing 7 days); weekdays at 16:00/19:00/22:00 UTC (~12/3/6pm ET, fixed-UTC so it drifts an hour in winter) it calls the **current-day** inquiry for same-day visibility (EventBridge `Input: {"endpoint":"current-day"}`, today-only, staleness sweep skipped). Every run archives the exact raw response to the `seahaven-payments-boa-raw-*` bucket (`raw//_/.json`, SSE-S3, 730-day lifecycle, PutObject-only grant; Retain-protected — decommission goes through the CFN decommission runbook) and upserts per-date `boa_balance##` snapshots (latest-wins on `run_at`, no TTL) from the Summary rows. Classifies each transaction and reconciles onto DynamoDB payment records. Event payload: `{fromDate?, toDate?, endpoint?}` (endpoint allowlisted and validated; unknown fields ignored). Intraday runs are disable-able as a unit via the `IntradaySchedule` rule. See [Bank reconciliation](#bank-reconciliation-fetchboatransactions). - **SlackAppHome** — Lambda behind API Gateway (`POST /slack/events`). Verifies the Slack signing secret (HMAC-SHA256, 5-minute replay window) before processing, then renders the payments dashboard on the Slack App Home tab with outstanding aging buckets, drill-down modals, and an always-visible "Returned — Needs Action" queue (bank-returned payments awaiting a reissue/void decision, sorted oldest return first). Returned records are excluded from Outstanding totals; terminal voided-and-bounced records appear in neither (audit trail only). @@ -17,7 +16,7 @@ AWS SAM application that ingests payment CSVs, syncs check data with Bank of Ame - **ExpenseReceiver** — Lambda behind API Gateway (`POST /slack/expense-events`). Verifies the Slack signing secret (HMAC-SHA256), handles URL verification challenges, and async-invokes ExpenseProcessor. Runs outside VPC. - **ExpenseProcessor** — Async Lambda invoked by ExpenseReceiver. Processes `:white_check_mark:` reactions to advance expense messages through a four-stage Slack channel pipeline: Submitted → Processed → Authorized → Matched. Runs outside VPC. -ProcessPaymentCsv, FetchBoaTransactions, and SlackAppHome run inside a VPC with a NAT Gateway for a static outbound IP (required by BoA IP whitelisting). ProcessPayrollEmail, ExpenseReceiver, and ExpenseProcessor run outside the VPC. +ProcessPaymentCsv, FetchBoaTransactions, and SlackAppHome run inside a VPC with a NAT Gateway for a static outbound IP (required by BoA IP whitelisting). ExpenseReceiver, and ExpenseProcessor run outside the VPC. ## Expense Approval Bot @@ -41,14 +40,6 @@ Reaction-driven workflow that routes expense submissions through four Slack chan | `payments-dashboard/expense-slack-token` | Slack Bot token for the Expense Approval Bot app | | `payments-dashboard/expense-slack-signing-secret` | Slack signing secret for request verification | -## Payroll Email Pipeline - -Gusto sends payroll confirmation emails when payroll is run. A Gmail filter on adam@seahavenind.com auto-forwards emails from `automated@gusto.com` and `gustonoreply@gusto.com` to `payroll@int.seahaven.com`. - -**Flow:** Gmail forward → SES receipt rule → S3 bucket → Lambda parses email → DynamoDB (pending) → SQS delay queue (10 min) → Lambda batches all pending items for that date → single Slack message → DynamoDB (notified) - -**Deduplication:** Each email is deduplicated by DynamoDB key (`PAYROLL_EMAIL#employee#` or `PAYROLL_EMAIL#contractor##`). The batch post is deduplicated by `PAYROLL_BATCH#`. All items have a 90-day TTL. - ## BoA CashPro API Integration Two separate CashPro APIs are used, each with its own OAuth credentials: @@ -117,7 +108,7 @@ All BoA and Slack credentials are stored in AWS Secrets Manager (per `engineerin | Secret | Type | Contents | |--------|------|----------| -| `payments-dashboard/slack-bot-token` | plaintext | Slack Bot OAuth token (used by `processPayrollEmail`, `slackAppHome`) | +| `payments-dashboard/slack-bot-token` | plaintext | Slack Bot OAuth token (used by `slackAppHome`) | | `payments-dashboard/slack-signing-secret` | plaintext | Slack signing secret for `slackAppHome` request verification | | `payments-dashboard/boa-check-mgmt` | JSON | `appId`, `clientId`, `token`, `accountNumber`, `companyId` — Check Management API (`processPaymentCsv`) | | `payments-dashboard/boa-reporting` | JSON | `appId`, `clientId`, `token`, `accountNumber`, `bankId` — Reporting API (`fetchBoaTransactions`) | @@ -136,7 +127,7 @@ The `PaymentsDashboard` DynamoDB table (`AWS::DynamoDB::Table`, `TableName: Paym The table is imported by name, so there is no compile-time link between the stacks: any change to the table name, `pk` format, these attribute names, the encryption key, or the table's lifecycle policy will silently break the Bedrock agent at runtime. Coordinate such changes with `seahaven-slack-bot` before shipping (INFRA-138). -**Key prefixes in this table** (all owned by this stack): `payment#` (payment records), `metadata` (ingest metadata), `boa_txn##` (BoA submission journal, 90d TTL), `boa_recon#_#` (reconciliation run summaries, 90d TTL), `boa_balance##` (daily balance snapshots, latest-wins, no TTL), `PAYROLL_*` (payroll pipeline, 90d TTL). New prefixes are invisible to `seahaven-slack-bot`'s `begins_with(pk, "payment#")` scan — no consumer coordination needed when adding one. +**Key prefixes in this table** (all owned by this stack): `payment#` (payment records), `metadata` (ingest metadata), `boa_txn##` (BoA submission journal, 90d TTL), `boa_recon#_#` (reconciliation run summaries, 90d TTL), `boa_balance##` (daily balance snapshots, latest-wins, no TTL). New prefixes are invisible to `seahaven-slack-bot`'s `begins_with(pk, "payment#")` scan — no consumer coordination needed when adding one. ## Monitoring & Alarms @@ -146,19 +137,17 @@ All CloudWatch alarms publish to the shared `site-alerts` SNS topic (`arn:aws:sn | Alarm | Source | |-------|--------| -| `payments-payroll-batch-dlq-messages` | `payments-payroll-batch-dlq` | | `payments-processPaymentCsv-async-dlq-messages` | async-invoke OnFailure DLQ | -| `payments-processPayrollEmail-async-dlq-messages` | async-invoke OnFailure DLQ | **Lambda** (per function — `payments--...`): -| Type | Metric / Statistic | Threshold | -|------|--------------------|-----------| -| `-errors` (all 6) | `Errors` / Sum | > 0 | -| `-throttles` (all 6) | `Throttles` / Sum | > 0 | -| `-duration` (all 6) | `Duration` / Maximum | ~80% of each function's timeout | +| Type | Metric / Statistic | Threshold | +|----------------------|--------------------|-----------| +| `-errors` (all 5) | `Errors` / Sum | > 0 | +| `-throttles` (all 5) | `Throttles` / Sum | > 0 | +| `-duration` (all 5) | `Duration` / Maximum | ~80% of each function's timeout | -Duration thresholds (ms): processPaymentCsv 96000, processPayrollEmail 48000, fetchBoaTransactions 48000, slackAppHome 24000, expenseProcessor 12000, expenseReceiver 4000. +Duration thresholds (ms): processPaymentCsv 96000, fetchBoaTransactions 48000, slackAppHome 24000, expenseProcessor 12000, expenseReceiver 4000. **DynamoDB** (`PaymentsDashboard` table, `TableName` dimension, Sum > 0): `payments-dashboard-table-read-throttle` (`ReadThrottleEvents`), `payments-dashboard-table-write-throttle` (`WriteThrottleEvents`). The table is PAY_PER_REQUEST; these metrics emit only when a throttle occurs. `SystemErrors` is intentionally not alarmed because it does not emit at the `TableName`-only dimension. diff --git a/package-lock.json b/package-lock.json index 7ce1a4b..46d1f64 100644 --- a/package-lock.json +++ b/package-lock.json @@ -12,10 +12,8 @@ "@aws-sdk/client-lambda": "^3.1121.0", "@aws-sdk/client-s3": "^3.1121.0", "@aws-sdk/client-secrets-manager": "^3.1121.0", - "@aws-sdk/client-sqs": "^3.1121.0", "@aws-sdk/lib-dynamodb": "^3.1121.0", - "csv-parse": "^7.0.2", - "mailparser": "^3.9.17" + "csv-parse": "^7.0.2" } }, "node_modules/@aws-sdk/checksums": { @@ -115,26 +113,6 @@ "node": ">=20.0.0" } }, - "node_modules/@aws-sdk/client-sqs": { - "version": "3.1122.0", - "resolved": "https://registry.npmjs.org/@aws-sdk/client-sqs/-/client-sqs-3.1122.0.tgz", - "integrity": "sha512-DjZmwC+W5CrqTzHGBjSWOPsYqClmHYvPeipG+LA4BPQKUdYExM53j2MSNgUqffp7KYu08wiY9Y+xRPTIvLz5JQ==", - "license": "Apache-2.0", - "dependencies": { - "@aws-sdk/core": "^3.977.9", - "@aws-sdk/credential-provider-node": "^3.972.81", - "@aws-sdk/middleware-sdk-sqs": "^3.972.42", - "@aws-sdk/types": "^3.974.5", - "@smithy/core": "^3.33.3", - "@smithy/fetch-http-handler": "^5.7.2", - "@smithy/node-http-handler": "^4.11.3", - "@smithy/types": "^4.17.2", - "tslib": "^2.6.2" - }, - "engines": { - "node": ">=20.0.0" - } - }, "node_modules/@aws-sdk/core": { "version": "3.977.9", "resolved": "https://registry.npmjs.org/@aws-sdk/core/-/core-3.977.9.tgz", @@ -382,21 +360,6 @@ "node": ">=20.0.0" } }, - "node_modules/@aws-sdk/middleware-sdk-sqs": { - "version": "3.972.42", - "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-sdk-sqs/-/middleware-sdk-sqs-3.972.42.tgz", - "integrity": "sha512-D/O6iHAqlm0b430vIGewanSsr5RzaWbSDFlHYdKLWlZb4kaMKBmb44ReNHAFEKj5tNRY8pysw0qfciosB/VcJg==", - "license": "Apache-2.0", - "dependencies": { - "@aws-sdk/types": "^3.974.5", - "@smithy/core": "^3.33.3", - "@smithy/types": "^4.17.2", - "tslib": "^2.6.2" - }, - "engines": { - "node": ">=20.0.0" - } - }, "node_modules/@aws-sdk/nested-clients": { "version": "3.997.44", "resolved": "https://registry.npmjs.org/@aws-sdk/nested-clients/-/nested-clients-3.997.44.tgz", @@ -498,22 +461,6 @@ "node": ">=18.0.0" } }, - "node_modules/@selderee/plugin-htmlparser2": { - "version": "0.12.0", - "resolved": "https://registry.npmjs.org/@selderee/plugin-htmlparser2/-/plugin-htmlparser2-0.12.0.tgz", - "integrity": "sha512-oELmoyA6ML9jDRMV3kgcMQFKxUfBU0yFVn6yTctVaLT5ygXnxH52I3TZEgV9EhXJC68/uFvE5Daj1/25c0Xa/A==", - "license": "MIT", - "dependencies": { - "domelementtype": "~2.3.0", - "domhandler": "~5.0.3" - }, - "funding": { - "url": "https://github.com/sponsors/KillyMXI" - }, - "peerDependencies": { - "selderee": "~0.12.0" - } - }, "node_modules/@smithy/core": { "version": "3.33.3", "resolved": "https://registry.npmjs.org/@smithy/core/-/core-3.33.3.tgz", @@ -595,17 +542,6 @@ "node": ">=18.0.0" } }, - "node_modules/@zone-eu/mailsplit": { - "version": "5.4.16", - "resolved": "https://registry.npmjs.org/@zone-eu/mailsplit/-/mailsplit-5.4.16.tgz", - "integrity": "sha512-zQ9iXvlT3Wi/hazeC1MdI4rQc1UJwJ6IQ6QzSZ5KDxLZZWQSazWLOzImLFluXadKShJ9WJvI1xH+AyVS8b9azg==", - "license": "(MIT OR EUPL-1.1+)", - "dependencies": { - "libbase64": "1.3.0", - "libmime": "5.4.3", - "libqp": "2.1.1" - } - }, "node_modules/bowser": { "version": "2.14.1", "resolved": "https://registry.npmjs.org/bowser/-/bowser-2.14.1.tgz", @@ -618,246 +554,6 @@ "integrity": "sha512-uKZghv9UmPkMVLYy//KZ9HFAIJsl7wkhoEdIL0+rhuSY9pZQlhaeGEDPIe+/w7eh81MOql8Q/9+inAGWG6ZHYA==", "license": "MIT" }, - "node_modules/deepmerge-ts": { - "version": "8.0.2", - "resolved": "https://registry.npmjs.org/deepmerge-ts/-/deepmerge-ts-8.0.2.tgz", - "integrity": "sha512-uqbvqLUMrc6p0MO+WBRtTxY55hmyh94WRwI5a++PZe54X+bfVh59FSN7uWCBCW1CCVjzjnrwzfI8zidE2obMMw==", - "funding": [ - { - "type": "ko-fi", - "url": "https://ko-fi.com/rebeccastevens" - }, - { - "type": "tidelift", - "url": "https://tidelift.com/funding/github/npm/deepmerge-ts" - } - ], - "license": "BSD-3-Clause", - "engines": { - "node": ">=16.9.0" - } - }, - "node_modules/dom-serializer": { - "version": "2.0.0", - "resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz", - "integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==", - "license": "MIT", - "dependencies": { - "domelementtype": "^2.3.0", - "domhandler": "^5.0.2", - "entities": "^4.2.0" - }, - "funding": { - "url": "https://github.com/cheeriojs/dom-serializer?sponsor=1" - } - }, - "node_modules/domelementtype": { - "version": "2.3.0", - "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz", - "integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==", - "funding": [ - { - "type": "github", - "url": "https://github.com/sponsors/fb55" - } - ], - "license": "BSD-2-Clause" - }, - "node_modules/domhandler": { - "version": "5.0.3", - "resolved": "https://registry.npmjs.org/domhandler/-/domhandler-5.0.3.tgz", - "integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==", - "license": "BSD-2-Clause", - "dependencies": { - "domelementtype": "^2.3.0" - }, - "engines": { - "node": ">= 4" - }, - "funding": { - "url": "https://github.com/fb55/domhandler?sponsor=1" - } - }, - "node_modules/domutils": { - "version": "3.2.2", - "resolved": "https://registry.npmjs.org/domutils/-/domutils-3.2.2.tgz", - "integrity": "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==", - "license": "BSD-2-Clause", - "dependencies": { - "dom-serializer": "^2.0.0", - "domelementtype": "^2.3.0", - "domhandler": "^5.0.3" - }, - "funding": { - "url": "https://github.com/fb55/domutils?sponsor=1" - } - }, - "node_modules/encoding-japanese": { - "version": "2.3.0", - "resolved": "https://registry.npmjs.org/encoding-japanese/-/encoding-japanese-2.3.0.tgz", - "integrity": "sha512-eQyh1vzHz13DUkZcJO+0IOAoKXRQwKV5IBffeuYsWZyRLGiSzfzXObCqWvqFXdX0UU8qOk+lBXbkUhMCpdJe4Q==", - "license": "MIT", - "engines": { - "node": ">=18.0.0" - } - }, - "node_modules/entities": { - "version": "4.5.0", - "resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz", - "integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==", - "license": "BSD-2-Clause", - "engines": { - "node": ">=0.12" - }, - "funding": { - "url": "https://github.com/fb55/entities?sponsor=1" - } - }, - "node_modules/he": { - "version": "1.2.0", - "resolved": "https://registry.npmjs.org/he/-/he-1.2.0.tgz", - "integrity": "sha512-F/1DnUGPopORZi0ni+CvrCgHQ5FyEAHRLSApuYWMmrbSwoN2Mn/7k+Gl38gJnR7yyDZk6WLXwiGod1JOWNDKGw==", - "license": "MIT", - "bin": { - "he": "bin/he" - } - }, - "node_modules/html-to-text": { - "version": "10.0.1", - "resolved": "https://registry.npmjs.org/html-to-text/-/html-to-text-10.0.1.tgz", - "integrity": "sha512-GiVhRI1BatGARSCmlXWNCjDT0cWrwBWoeduLoV0WSKAgaV/wa+hUWy5LiQLUs4UwiUrE52ZCMfBGiKD87TDPrg==", - "license": "MIT", - "dependencies": { - "@selderee/plugin-htmlparser2": "~0.12.0", - "deepmerge-ts": "^8.0.1", - "dom-serializer": "^2.0.0", - "htmlparser2": "^10.1.0", - "selderee": "~0.12.0" - }, - "engines": { - "node": ">=20.19.0" - }, - "funding": { - "url": "https://github.com/sponsors/KillyMXI" - } - }, - "node_modules/htmlparser2": { - "version": "10.1.0", - "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-10.1.0.tgz", - "integrity": "sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ==", - "funding": [ - "https://github.com/fb55/htmlparser2?sponsor=1", - { - "type": "github", - "url": "https://github.com/sponsors/fb55" - } - ], - "license": "MIT", - "dependencies": { - "domelementtype": "^2.3.0", - "domhandler": "^5.0.3", - "domutils": "^3.2.2", - "entities": "^7.0.1" - } - }, - "node_modules/htmlparser2/node_modules/entities": { - "version": "7.0.1", - "resolved": "https://registry.npmjs.org/entities/-/entities-7.0.1.tgz", - "integrity": "sha512-TWrgLOFUQTH994YUyl1yT4uyavY5nNB5muff+RtWaqNVCAK408b5ZnnbNAUEWLTCpum9w6arT70i1XdQ4UeOPA==", - "license": "BSD-2-Clause", - "engines": { - "node": ">=0.12" - }, - "funding": { - "url": "https://github.com/fb55/entities?sponsor=1" - } - }, - "node_modules/iconv-lite": { - "version": "0.7.3", - "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.7.3.tgz", - "integrity": "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ==", - "license": "MIT", - "dependencies": { - "safer-buffer": ">= 2.1.2 < 3.0.0" - }, - "engines": { - "node": ">=0.10.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/express" - } - }, - "node_modules/leac": { - "version": "0.7.0", - "resolved": "https://registry.npmjs.org/leac/-/leac-0.7.0.tgz", - "integrity": "sha512-qMrZeyEekgdRQ9o6a4NAB2EQZrv827GJdn1vnapwSJ90hWRB4TzUSunvacPkxQ2TnNqHNI1/zSt0hlo0crG8Jw==", - "license": "MIT", - "funding": { - "url": "https://github.com/sponsors/KillyMXI" - } - }, - "node_modules/libbase64": { - "version": "1.3.0", - "resolved": "https://registry.npmjs.org/libbase64/-/libbase64-1.3.0.tgz", - "integrity": "sha512-GgOXd0Eo6phYgh0DJtjQ2tO8dc0IVINtZJeARPeiIJqge+HdsWSuaDTe8ztQ7j/cONByDZ3zeB325AHiv5O0dg==", - "license": "MIT" - }, - "node_modules/libmime": { - "version": "5.4.3", - "resolved": "https://registry.npmjs.org/libmime/-/libmime-5.4.3.tgz", - "integrity": "sha512-di9BoDabBUMqjeD/wGj+hHpSgdqAph5ui7w6OdY6NpzU6O6VFLQsMOg9tqCjm/zf9OHzAM9EZxSOF7uIb8O8Hw==", - "license": "MIT", - "dependencies": { - "encoding-japanese": "2.3.0", - "iconv-lite": "0.7.3", - "libbase64": "1.3.0", - "libqp": "2.1.1" - } - }, - "node_modules/libqp": { - "version": "2.1.1", - "resolved": "https://registry.npmjs.org/libqp/-/libqp-2.1.1.tgz", - "integrity": "sha512-0Wd+GPz1O134cP62YU2GTOPNA7Qgl09XwCqM5zpBv87ERCXdfDtyKXvV7c9U22yWJh44QZqBocFnXN11K96qow==", - "license": "MIT" - }, - "node_modules/linkify-it": { - "version": "5.0.2", - "resolved": "https://registry.npmjs.org/linkify-it/-/linkify-it-5.0.2.tgz", - "integrity": "sha512-ONTm2jCMAVZjgQa/Fy1kScXsuOoF5NPTsoFBdE1KVIZ2vAh/r9+Bqo+0jINCBYnavTPQZz38QzFTme79ENoN3Q==", - "funding": [ - { - "type": "github", - "url": "https://github.com/sponsors/puzrin" - }, - { - "type": "github", - "url": "https://github.com/sponsors/markdown-it" - } - ], - "license": "MIT", - "dependencies": { - "uc.micro": "^2.0.0" - } - }, - "node_modules/mailparser": { - "version": "3.9.19", - "resolved": "https://registry.npmjs.org/mailparser/-/mailparser-3.9.19.tgz", - "integrity": "sha512-ik490D4yTo2B9aTdI8au8fOt6xIiC1EbBizi51ZjxX2zBd7k0qvatfvXVH3snXe4NNEovQ7rlP56dChp10pl9A==", - "license": "MIT", - "dependencies": { - "@zone-eu/mailsplit": "5.4.16", - "encoding-japanese": "2.3.0", - "he": "1.2.0", - "html-to-text": "10.0.1", - "iconv-lite": "0.7.3", - "libmime": "5.4.3", - "linkify-it": "5.0.2", - "nodemailer": "9.1.0", - "punycode.js": "2.3.1", - "tlds": "1.261.0" - } - }, "node_modules/mnemonist": { "version": "0.38.3", "resolved": "https://registry.npmjs.org/mnemonist/-/mnemonist-0.38.3.tgz", @@ -867,90 +563,17 @@ "obliterator": "^1.6.1" } }, - "node_modules/nodemailer": { - "version": "9.1.0", - "resolved": "https://registry.npmjs.org/nodemailer/-/nodemailer-9.1.0.tgz", - "integrity": "sha512-xj1Ri5Sau3qpPffHJwi2bY0oWVVWYq62Ph17l+2v1xXpxjqTls3YPc3L8dub9mcJpxj+1ucNnuYVqLlgh4pmDA==", - "license": "MIT-0", - "engines": { - "node": ">=6.0.0" - } - }, "node_modules/obliterator": { "version": "1.6.1", "resolved": "https://registry.npmjs.org/obliterator/-/obliterator-1.6.1.tgz", "integrity": "sha512-9WXswnqINnnhOG/5SLimUlzuU1hFJUc8zkwyD59Sd+dPOMf05PmnYG/d6Q7HZ+KmgkZJa1PxRso6QdM3sTNHig==", "license": "MIT" }, - "node_modules/parseley": { - "version": "0.13.1", - "resolved": "https://registry.npmjs.org/parseley/-/parseley-0.13.1.tgz", - "integrity": "sha512-uNBJZzmb60l6p6VWLTmevizNAGnE0xoSf1n0B4q3ntegDNzcS68NRCcBDZTcyXHxt2XhBChsCuqj4M+nChvE/A==", - "license": "MIT", - "dependencies": { - "leac": "^0.7.0", - "peberminta": "^0.10.0" - }, - "funding": { - "url": "https://github.com/sponsors/KillyMXI" - } - }, - "node_modules/peberminta": { - "version": "0.10.0", - "resolved": "https://registry.npmjs.org/peberminta/-/peberminta-0.10.0.tgz", - "integrity": "sha512-80B2AsU+I4Qdb0ZAPSfe9UwvGzwkM37IKIFEvdS3D/3Ndgv2bsuJ0bfG1+iEYO+l7Gfd4EUJmuRyq7efLgRMzQ==", - "license": "MIT", - "funding": { - "url": "https://github.com/sponsors/KillyMXI" - } - }, - "node_modules/punycode.js": { - "version": "2.3.1", - "resolved": "https://registry.npmjs.org/punycode.js/-/punycode.js-2.3.1.tgz", - "integrity": "sha512-uxFIHU0YlHYhDQtV4R9J6a52SLx28BCjT+4ieh7IGbgwVJWO+km431c4yRlREUAsAmt/uMjQUyQHNEPf0M39CA==", - "license": "MIT", - "engines": { - "node": ">=6" - } - }, - "node_modules/safer-buffer": { - "version": "2.1.2", - "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz", - "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==", - "license": "MIT" - }, - "node_modules/selderee": { - "version": "0.12.0", - "resolved": "https://registry.npmjs.org/selderee/-/selderee-0.12.0.tgz", - "integrity": "sha512-b1YMh3+DHZp59DLna3qVwQ5iOla/nrI6mLBNW02XxU77M3046Df6VLkoaJyFz20VsGIG5kkp+FK0kg4K4HnUFw==", - "license": "MIT", - "dependencies": { - "parseley": "~0.13.1" - }, - "funding": { - "url": "https://github.com/sponsors/KillyMXI" - } - }, - "node_modules/tlds": { - "version": "1.261.0", - "resolved": "https://registry.npmjs.org/tlds/-/tlds-1.261.0.tgz", - "integrity": "sha512-QXqwfEl9ddlGBaRFXIvNKK6OhipSiLXuRuLJX5DErz0o0Q0rYxulWLdFryTkV5PkdZct5iMInwYEGe/eR++1AA==", - "license": "MIT", - "bin": { - "tlds": "bin.js" - } - }, "node_modules/tslib": { "version": "2.8.1", "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz", "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==", "license": "0BSD" - }, - "node_modules/uc.micro": { - "version": "2.1.0", - "resolved": "https://registry.npmjs.org/uc.micro/-/uc.micro-2.1.0.tgz", - "integrity": "sha512-ARDJmphmdvUk6Glw7y9DQ2bFkKBHwQHLi2lsaH6PPmz/Ka9sFOBsBluozhDltWmnv9u/cF6Rt87znRTPV+yp/A==", - "license": "MIT" } } } diff --git a/package.json b/package.json index 127c2ca..1901d9d 100644 --- a/package.json +++ b/package.json @@ -14,9 +14,7 @@ "@aws-sdk/client-lambda": "^3.1121.0", "@aws-sdk/client-s3": "^3.1121.0", "@aws-sdk/client-secrets-manager": "^3.1121.0", - "@aws-sdk/client-sqs": "^3.1121.0", "@aws-sdk/lib-dynamodb": "^3.1121.0", - "csv-parse": "^7.0.2", - "mailparser": "^3.9.17" + "csv-parse": "^7.0.2" } } diff --git a/src/processPayrollEmail.js b/src/processPayrollEmail.js deleted file mode 100644 index 3979b06..0000000 --- a/src/processPayrollEmail.js +++ /dev/null @@ -1,478 +0,0 @@ -import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3"; -import { DynamoDBClient } from "@aws-sdk/client-dynamodb"; -import { - DynamoDBDocumentClient, - GetCommand, - PutCommand, - ScanCommand, - BatchWriteCommand, -} from "@aws-sdk/lib-dynamodb"; -import { SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs"; -import { SecretsManagerClient, GetSecretValueCommand } from "@aws-sdk/client-secrets-manager"; -import { simpleParser } from "mailparser"; - -const s3 = new S3Client(); -const ddb = DynamoDBDocumentClient.from(new DynamoDBClient()); -const sqs = new SQSClient(); -const secrets = new SecretsManagerClient(); - -const TABLE_NAME = process.env.TABLE_NAME; -const CHANNEL_ID = process.env.PAYROLL_CHANNEL_ID; -const QUEUE_URL = process.env.PAYROLL_BATCH_QUEUE_URL; - -let cachedToken; -async function getSlackToken() { - if (cachedToken) return cachedToken; - const { SecretString } = await secrets.send( - new GetSecretValueCommand({ SecretId: process.env.SLACK_BOT_TOKEN_SECRET_NAME }) - ); - cachedToken = SecretString; - return cachedToken; -} - -const formatCurrency = (v) => - new Intl.NumberFormat("en-US", { - style: "currency", - currency: "USD", - }).format(Number(v || 0)); - -function isAllowedGustoHost(hostname) { - const h = (hostname || "").toLowerCase(); - return h === "gusto.com" || h.endsWith(".gusto.com"); -} - -function bodyMentionsAllowedGustoUrl(text) { - const urlMatches = (text || "").match(/\bhttps?:\/\/[^\s<>"')]+/gi) || []; - for (const rawUrl of urlMatches) { - try { - const parsed = new URL(rawUrl); - if (isAllowedGustoHost(parsed.hostname)) return true; - } catch { - // Ignore malformed URLs in email text. - } - } - return false; -} - -function classifyEmail(from, subject, text) { - const fromAddr = (from?.text || from || "").toLowerCase(); - const subj = (subject || "").toLowerCase(); - - if ( - fromAddr.includes("automated@gusto.com") && - subj.includes("payroll confirmation") - ) { - return "employee"; - } - if ( - fromAddr.includes("gustonoreply@gusto.com") && - subj.includes("payment confirmation") - ) { - return "contractor"; - } - - const strippedSubj = subj.replace(/^fwd?:\s*/i, ""); - const bodyMentionsGusto = bodyMentionsAllowedGustoUrl(text || ""); - - if (bodyMentionsGusto && strippedSubj.includes("payroll confirmation")) { - return "employee"; - } - if (bodyMentionsGusto && strippedSubj.includes("payment confirmation")) { - return "contractor"; - } - - return null; -} - -function parseDateFromSubject(subject) { - const m = subject.match(/:\s*(\w+,\s*\w+\s+\d+)\s+(?:payroll|payment)\s+confirmation/i); - if (!m) return null; - const raw = m[1].trim(); - const year = new Date().getFullYear(); - const d = new Date(`${raw}, ${year}`); - if (isNaN(d)) return null; - return { - display: raw, - iso: d.toISOString().slice(0, 10), - }; -} - -function parseEmployeePayroll(subject, text) { - const checkDate = parseDateFromSubject(subject); - if (!checkDate) return null; - - const norm = text.replace(/\n/g, " ").replace(/\s+/g, " "); - - const debitMatch = norm.match( - /we.ll debit \$([\d,]+\.\d{2}) from the bank account ending in (\d+)/i - ); - if (!debitMatch) return null; - - const totalDebit = parseFloat(debitMatch[1].replace(/,/g, "")); - const bankSuffix = debitMatch[2]; - - const payPeriodMatch = norm.match(/payroll for the (.+?)\s+pay period/i); - const netPayMatch = norm.match( - /\$([\d,]+\.\d{2}) will be for your employee net pay/i - ); - const taxesMatch = norm.match(/\$([\d,]+\.\d{2}) will be for taxes/i); - const reimbursementsMatch = norm.match( - /\$([\d,]+\.\d{2}) will be for reimbursements/i - ); - - return { - checkDate: checkDate.display, - dateKey: checkDate.iso, - totalDebit, - bankSuffix, - payPeriod: payPeriodMatch ? payPeriodMatch[1].trim() : null, - netPay: netPayMatch ? parseFloat(netPayMatch[1].replace(/,/g, "")) : null, - taxes: taxesMatch ? parseFloat(taxesMatch[1].replace(/,/g, "")) : null, - reimbursements: reimbursementsMatch - ? parseFloat(reimbursementsMatch[1].replace(/,/g, "")) - : null, - }; -} - -function parseContractorPayment(subject, text) { - const checkDate = parseDateFromSubject(subject); - if (!checkDate) return null; - - const norm = text.replace(/\n/g, " ").replace(/\s+/g, " "); - - const debitMatch = norm.match( - /we.ll debit \$([\d,]+\.\d{2}) from the bank account ending in (\d+)/i - ); - if (!debitMatch) return null; - - const totalDebit = parseFloat(debitMatch[1].replace(/,/g, "")); - const bankSuffix = debitMatch[2]; - - const nameMatch = norm.match(/at that time for (.+?) to be paid on/i); - const contractorName = nameMatch ? nameMatch[1].trim() : "Unknown contractor"; - - return { - checkDate: checkDate.display, - dateKey: checkDate.iso, - totalDebit, - bankSuffix, - contractorName, - }; -} - -function buildCombinedBlocks(dateDisplay, employee, contractors) { - const blocks = [ - { - type: "header", - text: { - type: "plain_text", - text: `Payroll Processed — ${dateDisplay}`, - }, - }, - ]; - - if (employee) { - const topFields = [ - { type: "mrkdwn", text: `*Check Date*\n${employee.checkDate}` }, - ]; - if (employee.payPeriod) { - topFields.unshift({ - type: "mrkdwn", - text: `*Pay Period*\n${employee.payPeriod}`, - }); - } - blocks.push({ type: "section", fields: topFields }); - blocks.push({ type: "divider" }); - - const detailFields = []; - if (employee.netPay != null) { - detailFields.push({ - type: "mrkdwn", - text: `*Employee Net Pay*\n${formatCurrency(employee.netPay)}`, - }); - } - if (employee.taxes != null) { - detailFields.push({ - type: "mrkdwn", - text: `*Taxes*\n${formatCurrency(employee.taxes)}`, - }); - } - if (detailFields.length) blocks.push({ type: "section", fields: detailFields }); - - const extraFields = []; - if (employee.reimbursements != null) { - extraFields.push({ - type: "mrkdwn", - text: `*Reimbursements*\n${formatCurrency(employee.reimbursements)}`, - }); - } - extraFields.push({ - type: "mrkdwn", - text: `*Bank Account*\n...${employee.bankSuffix}`, - }); - blocks.push({ type: "section", fields: extraFields }); - } - - if (contractors.length > 0) { - blocks.push({ type: "divider" }); - blocks.push({ - type: "section", - text: { - type: "mrkdwn", - text: `*Contractor Payments*`, - }, - }); - for (const c of contractors) { - blocks.push({ - type: "section", - fields: [ - { type: "mrkdwn", text: `*Contractor*\n${c.contractorName}` }, - { type: "mrkdwn", text: `*Amount*\n${formatCurrency(c.totalDebit)}` }, - ], - }); - } - } - - const grandTotal = - (employee ? employee.totalDebit : 0) + - contractors.reduce((sum, c) => sum + c.totalDebit, 0); - - blocks.push({ type: "divider" }); - blocks.push({ - type: "section", - text: { - type: "mrkdwn", - text: `:moneybag: *Total Bank Debit: ${formatCurrency(grandTotal)}*`, - }, - }); - blocks.push({ - type: "context", - elements: [ - { type: "mrkdwn", text: "Source: Gusto payroll confirmation emails" }, - ], - }); - - return { blocks, grandTotal }; -} - -async function postSlack(token, blocks, text) { - const res = await fetch("https://slack.com/api/chat.postMessage", { - method: "POST", - headers: { - Authorization: `Bearer ${token}`, - "Content-Type": "application/json", - }, - body: JSON.stringify({ channel: CHANNEL_ID, blocks, text }), - }); - const data = await res.json(); - if (!data.ok) { - console.error("Slack post failed:", data.error); - throw new Error(`Slack API error: ${data.error}`); - } - return data.ts; -} - -const ttl90Days = () => Math.floor(Date.now() / 1000) + 90 * 24 * 60 * 60; - -async function handleS3Event(record) { - const bucket = record.s3.bucket.name; - const key = decodeURIComponent(record.s3.object.key.replace(/\+/g, " ")); - - const { Body } = await s3.send( - new GetObjectCommand({ Bucket: bucket, Key: key }) - ); - const rawEmail = await Body.transformToByteArray(); - const parsed = await simpleParser(Buffer.from(rawEmail)); - - const type = classifyEmail(parsed.from, parsed.subject, parsed.text); - if (!type) { - console.log("Unrecognized email, skipping:", parsed.subject); - return; - } - - if (type === "employee") { - const data = parseEmployeePayroll(parsed.subject, parsed.text); - if (!data) { - console.error("Failed to parse employee payroll:", parsed.subject); - return; - } - - const pk = `PAYROLL_EMAIL#employee#${data.dateKey}`; - const existing = await ddb.send( - new GetCommand({ TableName: TABLE_NAME, Key: { pk } }) - ); - if (existing.Item) { - console.log(`Already recorded: employee ${data.dateKey}`); - return; - } - - await ddb.send( - new PutCommand({ - TableName: TABLE_NAME, - Item: { - pk, - type: "employee", - status: "pending", - checkDate: data.checkDate, - dateKey: data.dateKey, - totalDebit: data.totalDebit, - bankSuffix: data.bankSuffix, - payPeriod: data.payPeriod, - netPay: data.netPay, - taxes: data.taxes, - reimbursements: data.reimbursements, - processedAt: new Date().toISOString(), - ttl: ttl90Days(), - }, - }) - ); - - await sqs.send( - new SendMessageCommand({ - QueueUrl: QUEUE_URL, - MessageBody: JSON.stringify({ dateKey: data.dateKey }), - }) - ); - console.log(`Queued employee payroll for ${data.dateKey}`); - } - - if (type === "contractor") { - const data = parseContractorPayment(parsed.subject, parsed.text); - if (!data) { - console.error("Failed to parse contractor payment:", parsed.subject); - return; - } - - const pk = `PAYROLL_EMAIL#contractor#${data.dateKey}#${data.bankSuffix}`; - const existing = await ddb.send( - new GetCommand({ TableName: TABLE_NAME, Key: { pk } }) - ); - if (existing.Item) { - console.log( - `Already recorded: contractor ${data.contractorName} ${data.dateKey}` - ); - return; - } - - await ddb.send( - new PutCommand({ - TableName: TABLE_NAME, - Item: { - pk, - type: "contractor", - status: "pending", - checkDate: data.checkDate, - dateKey: data.dateKey, - totalDebit: data.totalDebit, - bankSuffix: data.bankSuffix, - contractorName: data.contractorName, - processedAt: new Date().toISOString(), - ttl: ttl90Days(), - }, - }) - ); - - await sqs.send( - new SendMessageCommand({ - QueueUrl: QUEUE_URL, - MessageBody: JSON.stringify({ dateKey: data.dateKey }), - }) - ); - console.log( - `Queued contractor: ${data.contractorName} for ${data.dateKey}` - ); - } -} - -async function handleSqsEvent(sqsRecord) { - const { dateKey } = JSON.parse(sqsRecord.body); - - const batchPk = `PAYROLL_BATCH#${dateKey}`; - const batchCheck = await ddb.send( - new GetCommand({ TableName: TABLE_NAME, Key: { pk: batchPk } }) - ); - if (batchCheck.Item) { - console.log(`Batch already posted for ${dateKey}`); - return; - } - - const { Items } = await ddb.send( - new ScanCommand({ - TableName: TABLE_NAME, - FilterExpression: "begins_with(pk, :prefix) AND #s = :pending", - ExpressionAttributeNames: { "#s": "status" }, - ExpressionAttributeValues: { - ":prefix": `PAYROLL_EMAIL#`, - ":pending": "pending", - }, - }) - ); - - const pending = (Items || []).filter((i) => i.dateKey === dateKey); - if (pending.length === 0) { - console.log(`No pending items for ${dateKey}`); - return; - } - - const employee = pending.find((i) => i.type === "employee") || null; - const contractors = pending.filter((i) => i.type === "contractor"); - const dateDisplay = pending[0].checkDate; - - const token = await getSlackToken(); - const { blocks, grandTotal } = buildCombinedBlocks(dateDisplay, employee, contractors); - const slackTs = await postSlack( - token, - blocks, - `Payroll processed for ${dateDisplay}: ${formatCurrency(grandTotal)}` - ); - - await ddb.send( - new PutCommand({ - TableName: TABLE_NAME, - Item: { - pk: batchPk, - dateKey, - checkDate: dateDisplay, - employeeCount: employee ? 1 : 0, - contractorCount: contractors.length, - totalDebit: grandTotal, - slackTs, - processedAt: new Date().toISOString(), - ttl: ttl90Days(), - }, - }) - ); - - const updateBatch = pending.map((item) => ({ - PutRequest: { - Item: { ...item, status: "notified" }, - }, - })); - for (let i = 0; i < updateBatch.length; i += 25) { - await ddb.send( - new BatchWriteCommand({ - RequestItems: { [TABLE_NAME]: updateBatch.slice(i, i + 25) }, - }) - ); - } - - console.log( - `Posted batch: ${employee ? 1 : 0} employee + ${contractors.length} contractor(s) for ${dateKey}` - ); -} - -export const handler = async (event) => { - if (event.Records?.[0]?.eventSource === "aws:s3") { - for (const record of event.Records) { - await handleS3Event(record); - } - } else if (event.Records?.[0]?.eventSource === "aws:sqs") { - for (const record of event.Records) { - await handleSqsEvent(record); - } - } else { - console.log("Unknown event source:", JSON.stringify(event).slice(0, 200)); - } - - return { statusCode: 200 }; -}; diff --git a/template.yaml b/template.yaml index d027bd8..70b7cfa 100644 --- a/template.yaml +++ b/template.yaml @@ -197,11 +197,6 @@ Resources: Status: Enabled ExpirationInDays: 730 - # Defense-in-depth for bank data: deny any non-TLS access. No Allow - # statements — access is IAM-only (the Lambda's PutObject grant); unlike - # PayrollEmailBucket there is no cross-service principal here. The Deny - # covers bucket-level actions too, which is safe because nothing is - # granted List/Get — revisit if read access is ever added. BoaRawBucketPolicy: Type: AWS::S3::BucketPolicy Properties: @@ -243,112 +238,11 @@ Resources: SSEType: KMS KMSMasterKeyId: !Ref DynamoDbCmkArn - PayrollEmailBucket: - Type: AWS::S3::Bucket - Properties: - BucketName: !Sub seahaven-payroll-emails-${AWS::AccountId} - PublicAccessBlockConfiguration: - BlockPublicAcls: true - IgnorePublicAcls: true - BlockPublicPolicy: true - RestrictPublicBuckets: true - LifecycleConfiguration: - Rules: - - Id: ExpireEmails - Status: Enabled - ExpirationInDays: 30 - - PayrollEmailBucketPolicy: - Type: AWS::S3::BucketPolicy - Properties: - Bucket: !Ref PayrollEmailBucket - PolicyDocument: - Version: "2012-10-17" - Statement: - - Sid: AllowSESPut - Effect: Allow - Principal: - Service: ses.amazonaws.com - Action: s3:PutObject - Resource: !Sub arn:aws:s3:::seahaven-payroll-emails-${AWS::AccountId}/* - Condition: - StringEquals: - AWS:SourceAccount: !Ref AWS::AccountId - - PayrollEmailRule: - Type: AWS::SES::ReceiptRule - DependsOn: PayrollEmailBucketPolicy - Properties: - RuleSetName: INBOUND_MAIL - After: ExistingRuleSetWorkorderEmailRuleEA29F845-okepxcVarTfu - Rule: - Name: store-payroll-emails - Enabled: true - ScanEnabled: true - Recipients: - - payroll@int.seahaven.com - Actions: - - S3Action: - BucketName: !Ref PayrollEmailBucket - ObjectKeyPrefix: inbound/ - - PayrollBatchQueue: - Type: AWS::SQS::Queue - Properties: - QueueName: payments-payroll-batch - DelaySeconds: 600 - MessageRetentionPeriod: 86400 - VisibilityTimeout: 60 - RedrivePolicy: - deadLetterTargetArn: !GetAtt PayrollBatchDLQ.Arn - maxReceiveCount: 3 - - # Audit L-15: payroll-batch messages were lost after max receives. 14-day - # retention so a failure on Friday survives the weekend. - PayrollBatchDLQ: - Type: AWS::SQS::Queue - Properties: - QueueName: payments-payroll-batch-dlq - MessageRetentionPeriod: 1209600 - - PayrollBatchDLQAlarm: - Type: AWS::CloudWatch::Alarm - Properties: - AlarmName: payments-payroll-batch-dlq-messages - AlarmDescription: Failed payroll-batch messages landed in the DLQ - Namespace: AWS/SQS - MetricName: ApproximateNumberOfMessagesVisible - Dimensions: - - Name: QueueName - Value: !GetAtt PayrollBatchDLQ.QueueName - Statistic: Maximum - Period: 300 - EvaluationPeriods: 1 - Threshold: 0 - ComparisonOperator: GreaterThanThreshold - TreatMissingData: notBreaching - # ALARM-only notification by convention — no OK/recovery action - AlarmActions: - - !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts - - # Async-invoke OnFailure DLQs (INFRA-41 / H-8). Reconciles the interim - # CLI-created queues into CloudFormation. Names are CFN-generated to avoid - # colliding with the live interim payments--dlq queues (deleted after - # this deploy). 14-day retention mirrors the payments-payroll-batch DLQ so a - # Friday failure survives the weekend. - # Distinct -async-dlq name (not the live interim payments--dlq) so this - # CFN queue does not collide with the queue being deleted post-deploy. ProcessPaymentCsvDLQ: Type: AWS::SQS::Queue Properties: QueueName: payments-processPaymentCsv-async-dlq - MessageRetentionPeriod: 1209600 # 14d, matches payments-payroll-batch-dlq - - ProcessPayrollEmailDLQ: - Type: AWS::SQS::Queue - Properties: - QueueName: payments-processPayrollEmail-async-dlq - MessageRetentionPeriod: 1209600 # 14d, matches payments-payroll-batch-dlq + MessageRetentionPeriod: 1209600 # 14d # ALARM-only Lambda Errors alarms (INFRA-41 / H-8). Threshold > 0 on the # Errors Sum, no OK/recovery action by convention. @@ -371,25 +265,6 @@ Resources: AlarmActions: - !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts - ProcessPayrollEmailErrorsAlarm: - Type: AWS::CloudWatch::Alarm - Properties: - AlarmName: payments-processPayrollEmail-errors - AlarmDescription: payments-processPayrollEmail invocation errors - Namespace: AWS/Lambda - MetricName: Errors - Dimensions: - - Name: FunctionName - Value: !Ref ProcessPayrollEmailFunction - Statistic: Sum - Period: 300 - EvaluationPeriods: 1 - Threshold: 0 - ComparisonOperator: GreaterThanThreshold - TreatMissingData: notBreaching - AlarmActions: - - !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts - # ── Lambda Errors alarms (Wave 1) ────────────────────────────────────────── # Clone of ProcessPaymentCsvErrorsAlarm for the remaining functions. AWS/Lambda # Errors, Sum over 5m, threshold > 0, ALARM-only by convention. @@ -491,25 +366,6 @@ Resources: AlarmActions: - !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts - ProcessPayrollEmailThrottlesAlarm: - Type: AWS::CloudWatch::Alarm - Properties: - AlarmName: payments-processPayrollEmail-throttles - AlarmDescription: payments-processPayrollEmail invocations throttled - Namespace: AWS/Lambda - MetricName: Throttles - Dimensions: - - Name: FunctionName - Value: !Ref ProcessPayrollEmailFunction - Statistic: Sum - Period: 300 - EvaluationPeriods: 1 - Threshold: 0 - ComparisonOperator: GreaterThanThreshold - TreatMissingData: notBreaching - AlarmActions: - - !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts - FetchBoaTransactionsThrottlesAlarm: Type: AWS::CloudWatch::Alarm Properties: @@ -608,25 +464,6 @@ Resources: AlarmActions: - !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts - ProcessPayrollEmailDurationAlarm: - Type: AWS::CloudWatch::Alarm - Properties: - AlarmName: payments-processPayrollEmail-duration - AlarmDescription: payments-processPayrollEmail approaching timeout (~80% of 60s) - Namespace: AWS/Lambda - MetricName: Duration - Dimensions: - - Name: FunctionName - Value: !Ref ProcessPayrollEmailFunction - Statistic: Maximum - Period: 300 - EvaluationPeriods: 1 - Threshold: 48000 - ComparisonOperator: GreaterThanThreshold - TreatMissingData: notBreaching - AlarmActions: - - !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts - FetchBoaTransactionsDurationAlarm: Type: AWS::CloudWatch::Alarm Properties: @@ -811,8 +648,8 @@ Resources: AlarmActions: - !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts - # Messages-present alarms on the async-invoke OnFailure DLQs, mirroring - # PayrollBatchDLQAlarm. Threshold > 0 on the visible-message count, ALARM-only. + # Messages-present alarms on the async-invoke OnFailure DLQs, + # Threshold > 0 on the visible-message count, ALARM-only. ProcessPaymentCsvDLQAlarm: Type: AWS::CloudWatch::Alarm Properties: @@ -833,32 +670,6 @@ Resources: AlarmActions: - !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts - ProcessPayrollEmailDLQAlarm: - Type: AWS::CloudWatch::Alarm - Properties: - AlarmName: payments-processPayrollEmail-async-dlq-messages - AlarmDescription: Failed processPayrollEmail async invocations landed in the DLQ - Namespace: AWS/SQS - MetricName: ApproximateNumberOfMessagesVisible - Dimensions: - - Name: QueueName - Value: !GetAtt ProcessPayrollEmailDLQ.QueueName - Statistic: Maximum - Period: 300 - EvaluationPeriods: 1 - Threshold: 0 - ComparisonOperator: GreaterThanThreshold - TreatMissingData: notBreaching - # ALARM-only notification by convention — no OK/recovery action - AlarmActions: - - !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts - - ProcessPayrollEmailLogGroup: - Type: AWS::Logs::LogGroup - Properties: - LogGroupName: /aws/lambda/payments-processPayrollEmail - RetentionInDays: 60 - ProcessPaymentCsvLogGroup: Type: AWS::Logs::LogGroup Properties: @@ -889,63 +700,6 @@ Resources: LogGroupName: /aws/lambda/payments-expenseProcessor RetentionInDays: 60 - ProcessPayrollEmailFunction: - Type: AWS::Serverless::Function - Properties: - FunctionName: payments-processPayrollEmail - Handler: src/processPayrollEmail.handler - Timeout: 60 - EventInvokeConfig: - MaximumRetryAttempts: 2 - MaximumEventAgeInSeconds: 21600 - DestinationConfig: - OnFailure: - Type: SQS - Destination: !GetAtt ProcessPayrollEmailDLQ.Arn - Environment: - Variables: - SLACK_BOT_TOKEN_SECRET_NAME: payments-dashboard/slack-bot-token - PAYROLL_CHANNEL_ID: C0AV5RBMYKU - PAYROLL_BATCH_QUEUE_URL: !Ref PayrollBatchQueue - Events: - EmailReceived: - Type: S3 - Properties: - Bucket: !Ref PayrollEmailBucket - Events: s3:ObjectCreated:* - Filter: - S3Key: - Rules: - - Name: prefix - Value: inbound/ - PayrollBatch: - Type: SQS - Properties: - Queue: !GetAtt PayrollBatchQueue.Arn - BatchSize: 1 - Policies: - - S3ReadPolicy: - BucketName: !Sub seahaven-payroll-emails-${AWS::AccountId} - - DynamoDBCrudPolicy: - TableName: !Ref DashboardTable - - Version: "2012-10-17" - Statement: - - Effect: Allow - Action: - - kms:Decrypt - - kms:GenerateDataKey - - kms:DescribeKey - Resource: !Ref DynamoDbCmkArn - - Version: "2012-10-17" - Statement: - - Effect: Allow - Action: secretsmanager:GetSecretValue - Resource: !Sub arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:payments-dashboard/slack-bot-token-* - - SQSSendMessagePolicy: - QueueName: !GetAtt PayrollBatchQueue.QueueName - - SQSPollerPolicy: - QueueName: !GetAtt PayrollBatchQueue.QueueName - ProcessPaymentCsvFunction: Type: AWS::Serverless::Function Properties: @@ -1171,9 +925,6 @@ Outputs: StaticOutboundIp: Description: Static IP for BoA API whitelist Value: !Ref NatEip - PayrollEmailBucket: - Description: S3 bucket for inbound payroll emails from SES - Value: !Ref PayrollEmailBucket ExpenseSlackEventsUrl: Description: URL for Expense Approval Bot Slack Event Subscriptions Value: !Sub https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/slack/expense-events