mirror of
https://github.com/Sea-Haven-Industries/payments-dashboard.git
synced 2026-09-30 01:53:13 +00:00
chore(payroll): remove deprecated Gusto email pipeline (#105)
Some checks failed
Deploy / deploy (push) Has been cancelled
Some checks failed
Deploy / deploy (push) Has been cancelled
This commit is contained in:
Adam Moussa
GitHub
parent
a56d83c9a0
commit
3f2aa692c3
5 changed files with 15 additions and 1132 deletions
31
README.md
31
README.md
|
|
@ -5,11 +5,10 @@
|
|||
|
||||
|
||||
|
||||
AWS SAM application that ingests payment CSVs, syncs check data with Bank of America CashPro APIs, processes Gusto payroll confirmation emails into Slack notifications, surfaces an outstanding-payments dashboard in Slack, and routes expense approvals through a Slack reaction-driven workflow.
|
||||
AWS SAM application that ingests payment CSVs, syncs check data with Bank of America CashPro APIs, surfaces an outstanding-payments dashboard in Slack, and routes expense approvals through a Slack reaction-driven workflow.
|
||||
|
||||
## Architecture
|
||||
|
||||
- **ProcessPayrollEmail** — Lambda triggered by S3 (inbound email) and SQS (batch timer). SES receives Gusto payroll emails at `payroll@int.seahaven.com`, stores them to S3, and this Lambda parses the email body, extracts financial data, and posts a combined Slack notification (employee payroll + contractor payments) after a 10-minute batching window. Runs outside VPC.
|
||||
- **ProcessPaymentCsv** — Lambda triggered by S3 CSV upload. Parses Stampli payment exports, upserts to DynamoDB, and submits new/cancelled checks to the CashPro Check Management API.
|
||||
- **FetchBoaTransactions** — Scheduled Lambda. Weekdays 9am ET it calls the CashPro **previous-day** Transaction Inquiry (authoritative sweep, trailing 7 days); weekdays at 16:00/19:00/22:00 UTC (~12/3/6pm ET, fixed-UTC so it drifts an hour in winter) it calls the **current-day** inquiry for same-day visibility (EventBridge `Input: {"endpoint":"current-day"}`, today-only, staleness sweep skipped). Every run archives the exact raw response to the `seahaven-payments-boa-raw-*` bucket (`raw/<endpoint>/<fromDate>_<toDate>/<runAt>.json`, SSE-S3, 730-day lifecycle, PutObject-only grant; Retain-protected — decommission goes through the CFN decommission runbook) and upserts per-date `boa_balance#<asOfDate>#<endpoint>` snapshots (latest-wins on `run_at`, no TTL) from the Summary rows. Classifies each transaction and reconciles onto DynamoDB payment records. Event payload: `{fromDate?, toDate?, endpoint?}` (endpoint allowlisted and validated; unknown fields ignored). Intraday runs are disable-able as a unit via the `IntradaySchedule` rule. See [Bank reconciliation](#bank-reconciliation-fetchboatransactions).
|
||||
- **SlackAppHome** — Lambda behind API Gateway (`POST /slack/events`). Verifies the Slack signing secret (HMAC-SHA256, 5-minute replay window) before processing, then renders the payments dashboard on the Slack App Home tab with outstanding aging buckets, drill-down modals, and an always-visible "Returned — Needs Action" queue (bank-returned payments awaiting a reissue/void decision, sorted oldest return first). Returned records are excluded from Outstanding totals; terminal voided-and-bounced records appear in neither (audit trail only).
|
||||
|
|
@ -17,7 +16,7 @@ AWS SAM application that ingests payment CSVs, syncs check data with Bank of Ame
|
|||
- **ExpenseReceiver** — Lambda behind API Gateway (`POST /slack/expense-events`). Verifies the Slack signing secret (HMAC-SHA256), handles URL verification challenges, and async-invokes ExpenseProcessor. Runs outside VPC.
|
||||
- **ExpenseProcessor** — Async Lambda invoked by ExpenseReceiver. Processes `:white_check_mark:` reactions to advance expense messages through a four-stage Slack channel pipeline: Submitted → Processed → Authorized → Matched. Runs outside VPC.
|
||||
|
||||
ProcessPaymentCsv, FetchBoaTransactions, and SlackAppHome run inside a VPC with a NAT Gateway for a static outbound IP (required by BoA IP whitelisting). ProcessPayrollEmail, ExpenseReceiver, and ExpenseProcessor run outside the VPC.
|
||||
ProcessPaymentCsv, FetchBoaTransactions, and SlackAppHome run inside a VPC with a NAT Gateway for a static outbound IP (required by BoA IP whitelisting). ExpenseReceiver, and ExpenseProcessor run outside the VPC.
|
||||
|
||||
## Expense Approval Bot
|
||||
|
||||
|
|
@ -41,14 +40,6 @@ Reaction-driven workflow that routes expense submissions through four Slack chan
|
|||
| `payments-dashboard/expense-slack-token` | Slack Bot token for the Expense Approval Bot app |
|
||||
| `payments-dashboard/expense-slack-signing-secret` | Slack signing secret for request verification |
|
||||
|
||||
## Payroll Email Pipeline
|
||||
|
||||
Gusto sends payroll confirmation emails when payroll is run. A Gmail filter on adam@seahavenind.com auto-forwards emails from `automated@gusto.com` and `gustonoreply@gusto.com` to `payroll@int.seahaven.com`.
|
||||
|
||||
**Flow:** Gmail forward → SES receipt rule → S3 bucket → Lambda parses email → DynamoDB (pending) → SQS delay queue (10 min) → Lambda batches all pending items for that date → single Slack message → DynamoDB (notified)
|
||||
|
||||
**Deduplication:** Each email is deduplicated by DynamoDB key (`PAYROLL_EMAIL#employee#<date>` or `PAYROLL_EMAIL#contractor#<date>#<bank-suffix>`). The batch post is deduplicated by `PAYROLL_BATCH#<date>`. All items have a 90-day TTL.
|
||||
|
||||
## BoA CashPro API Integration
|
||||
|
||||
Two separate CashPro APIs are used, each with its own OAuth credentials:
|
||||
|
|
@ -117,7 +108,7 @@ All BoA and Slack credentials are stored in AWS Secrets Manager (per `engineerin
|
|||
|
||||
| Secret | Type | Contents |
|
||||
|--------|------|----------|
|
||||
| `payments-dashboard/slack-bot-token` | plaintext | Slack Bot OAuth token (used by `processPayrollEmail`, `slackAppHome`) |
|
||||
| `payments-dashboard/slack-bot-token` | plaintext | Slack Bot OAuth token (used by `slackAppHome`) |
|
||||
| `payments-dashboard/slack-signing-secret` | plaintext | Slack signing secret for `slackAppHome` request verification |
|
||||
| `payments-dashboard/boa-check-mgmt` | JSON | `appId`, `clientId`, `token`, `accountNumber`, `companyId` — Check Management API (`processPaymentCsv`) |
|
||||
| `payments-dashboard/boa-reporting` | JSON | `appId`, `clientId`, `token`, `accountNumber`, `bankId` — Reporting API (`fetchBoaTransactions`) |
|
||||
|
|
@ -136,7 +127,7 @@ The `PaymentsDashboard` DynamoDB table (`AWS::DynamoDB::Table`, `TableName: Paym
|
|||
|
||||
The table is imported by name, so there is no compile-time link between the stacks: any change to the table name, `pk` format, these attribute names, the encryption key, or the table's lifecycle policy will silently break the Bedrock agent at runtime. Coordinate such changes with `seahaven-slack-bot` before shipping (INFRA-138).
|
||||
|
||||
**Key prefixes in this table** (all owned by this stack): `payment#<check_number>` (payment records), `metadata` (ingest metadata), `boa_txn#<ts>#<action>` (BoA submission journal, 90d TTL), `boa_recon#<from>_<to>#<runAt>` (reconciliation run summaries, 90d TTL), `boa_balance#<asOfDate>#<endpoint>` (daily balance snapshots, latest-wins, no TTL), `PAYROLL_*` (payroll pipeline, 90d TTL). New prefixes are invisible to `seahaven-slack-bot`'s `begins_with(pk, "payment#")` scan — no consumer coordination needed when adding one.
|
||||
**Key prefixes in this table** (all owned by this stack): `payment#<check_number>` (payment records), `metadata` (ingest metadata), `boa_txn#<ts>#<action>` (BoA submission journal, 90d TTL), `boa_recon#<from>_<to>#<runAt>` (reconciliation run summaries, 90d TTL), `boa_balance#<asOfDate>#<endpoint>` (daily balance snapshots, latest-wins, no TTL). New prefixes are invisible to `seahaven-slack-bot`'s `begins_with(pk, "payment#")` scan — no consumer coordination needed when adding one.
|
||||
|
||||
## Monitoring & Alarms
|
||||
|
||||
|
|
@ -146,19 +137,17 @@ All CloudWatch alarms publish to the shared `site-alerts` SNS topic (`arn:aws:sn
|
|||
|
||||
| Alarm | Source |
|
||||
|-------|--------|
|
||||
| `payments-payroll-batch-dlq-messages` | `payments-payroll-batch-dlq` |
|
||||
| `payments-processPaymentCsv-async-dlq-messages` | async-invoke OnFailure DLQ |
|
||||
| `payments-processPayrollEmail-async-dlq-messages` | async-invoke OnFailure DLQ |
|
||||
|
||||
**Lambda** (per function — `payments-<fn>-...`):
|
||||
|
||||
| Type | Metric / Statistic | Threshold |
|
||||
|------|--------------------|-----------|
|
||||
| `-errors` (all 6) | `Errors` / Sum | > 0 |
|
||||
| `-throttles` (all 6) | `Throttles` / Sum | > 0 |
|
||||
| `-duration` (all 6) | `Duration` / Maximum | ~80% of each function's timeout |
|
||||
| Type | Metric / Statistic | Threshold |
|
||||
|----------------------|--------------------|-----------|
|
||||
| `-errors` (all 5) | `Errors` / Sum | > 0 |
|
||||
| `-throttles` (all 5) | `Throttles` / Sum | > 0 |
|
||||
| `-duration` (all 5) | `Duration` / Maximum | ~80% of each function's timeout |
|
||||
|
||||
Duration thresholds (ms): processPaymentCsv 96000, processPayrollEmail 48000, fetchBoaTransactions 48000, slackAppHome 24000, expenseProcessor 12000, expenseReceiver 4000.
|
||||
Duration thresholds (ms): processPaymentCsv 96000, fetchBoaTransactions 48000, slackAppHome 24000, expenseProcessor 12000, expenseReceiver 4000.
|
||||
|
||||
**DynamoDB** (`PaymentsDashboard` table, `TableName` dimension, Sum > 0): `payments-dashboard-table-read-throttle` (`ReadThrottleEvents`), `payments-dashboard-table-write-throttle` (`WriteThrottleEvents`). The table is PAY_PER_REQUEST; these metrics emit only when a throttle occurs. `SystemErrors` is intentionally not alarmed because it does not emit at the `TableName`-only dimension.
|
||||
|
||||
|
|
|
|||
379
package-lock.json
generated
379
package-lock.json
generated
|
|
@ -12,10 +12,8 @@
|
|||
"@aws-sdk/client-lambda": "^3.1121.0",
|
||||
"@aws-sdk/client-s3": "^3.1121.0",
|
||||
"@aws-sdk/client-secrets-manager": "^3.1121.0",
|
||||
"@aws-sdk/client-sqs": "^3.1121.0",
|
||||
"@aws-sdk/lib-dynamodb": "^3.1121.0",
|
||||
"csv-parse": "^7.0.2",
|
||||
"mailparser": "^3.9.17"
|
||||
"csv-parse": "^7.0.2"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/checksums": {
|
||||
|
|
@ -115,26 +113,6 @@
|
|||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/client-sqs": {
|
||||
"version": "3.1122.0",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/client-sqs/-/client-sqs-3.1122.0.tgz",
|
||||
"integrity": "sha512-DjZmwC+W5CrqTzHGBjSWOPsYqClmHYvPeipG+LA4BPQKUdYExM53j2MSNgUqffp7KYu08wiY9Y+xRPTIvLz5JQ==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@aws-sdk/core": "^3.977.9",
|
||||
"@aws-sdk/credential-provider-node": "^3.972.81",
|
||||
"@aws-sdk/middleware-sdk-sqs": "^3.972.42",
|
||||
"@aws-sdk/types": "^3.974.5",
|
||||
"@smithy/core": "^3.33.3",
|
||||
"@smithy/fetch-http-handler": "^5.7.2",
|
||||
"@smithy/node-http-handler": "^4.11.3",
|
||||
"@smithy/types": "^4.17.2",
|
||||
"tslib": "^2.6.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/core": {
|
||||
"version": "3.977.9",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/core/-/core-3.977.9.tgz",
|
||||
|
|
@ -382,21 +360,6 @@
|
|||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/middleware-sdk-sqs": {
|
||||
"version": "3.972.42",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/middleware-sdk-sqs/-/middleware-sdk-sqs-3.972.42.tgz",
|
||||
"integrity": "sha512-D/O6iHAqlm0b430vIGewanSsr5RzaWbSDFlHYdKLWlZb4kaMKBmb44ReNHAFEKj5tNRY8pysw0qfciosB/VcJg==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@aws-sdk/types": "^3.974.5",
|
||||
"@smithy/core": "^3.33.3",
|
||||
"@smithy/types": "^4.17.2",
|
||||
"tslib": "^2.6.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/nested-clients": {
|
||||
"version": "3.997.44",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/nested-clients/-/nested-clients-3.997.44.tgz",
|
||||
|
|
@ -498,22 +461,6 @@
|
|||
"node": ">=18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@selderee/plugin-htmlparser2": {
|
||||
"version": "0.12.0",
|
||||
"resolved": "https://registry.npmjs.org/@selderee/plugin-htmlparser2/-/plugin-htmlparser2-0.12.0.tgz",
|
||||
"integrity": "sha512-oELmoyA6ML9jDRMV3kgcMQFKxUfBU0yFVn6yTctVaLT5ygXnxH52I3TZEgV9EhXJC68/uFvE5Daj1/25c0Xa/A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"domelementtype": "~2.3.0",
|
||||
"domhandler": "~5.0.3"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/KillyMXI"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"selderee": "~0.12.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@smithy/core": {
|
||||
"version": "3.33.3",
|
||||
"resolved": "https://registry.npmjs.org/@smithy/core/-/core-3.33.3.tgz",
|
||||
|
|
@ -595,17 +542,6 @@
|
|||
"node": ">=18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@zone-eu/mailsplit": {
|
||||
"version": "5.4.16",
|
||||
"resolved": "https://registry.npmjs.org/@zone-eu/mailsplit/-/mailsplit-5.4.16.tgz",
|
||||
"integrity": "sha512-zQ9iXvlT3Wi/hazeC1MdI4rQc1UJwJ6IQ6QzSZ5KDxLZZWQSazWLOzImLFluXadKShJ9WJvI1xH+AyVS8b9azg==",
|
||||
"license": "(MIT OR EUPL-1.1+)",
|
||||
"dependencies": {
|
||||
"libbase64": "1.3.0",
|
||||
"libmime": "5.4.3",
|
||||
"libqp": "2.1.1"
|
||||
}
|
||||
},
|
||||
"node_modules/bowser": {
|
||||
"version": "2.14.1",
|
||||
"resolved": "https://registry.npmjs.org/bowser/-/bowser-2.14.1.tgz",
|
||||
|
|
@ -618,246 +554,6 @@
|
|||
"integrity": "sha512-uKZghv9UmPkMVLYy//KZ9HFAIJsl7wkhoEdIL0+rhuSY9pZQlhaeGEDPIe+/w7eh81MOql8Q/9+inAGWG6ZHYA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/deepmerge-ts": {
|
||||
"version": "8.0.2",
|
||||
"resolved": "https://registry.npmjs.org/deepmerge-ts/-/deepmerge-ts-8.0.2.tgz",
|
||||
"integrity": "sha512-uqbvqLUMrc6p0MO+WBRtTxY55hmyh94WRwI5a++PZe54X+bfVh59FSN7uWCBCW1CCVjzjnrwzfI8zidE2obMMw==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "ko-fi",
|
||||
"url": "https://ko-fi.com/rebeccastevens"
|
||||
},
|
||||
{
|
||||
"type": "tidelift",
|
||||
"url": "https://tidelift.com/funding/github/npm/deepmerge-ts"
|
||||
}
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"engines": {
|
||||
"node": ">=16.9.0"
|
||||
}
|
||||
},
|
||||
"node_modules/dom-serializer": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz",
|
||||
"integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"domelementtype": "^2.3.0",
|
||||
"domhandler": "^5.0.2",
|
||||
"entities": "^4.2.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/cheeriojs/dom-serializer?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/domelementtype": {
|
||||
"version": "2.3.0",
|
||||
"resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz",
|
||||
"integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/fb55"
|
||||
}
|
||||
],
|
||||
"license": "BSD-2-Clause"
|
||||
},
|
||||
"node_modules/domhandler": {
|
||||
"version": "5.0.3",
|
||||
"resolved": "https://registry.npmjs.org/domhandler/-/domhandler-5.0.3.tgz",
|
||||
"integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==",
|
||||
"license": "BSD-2-Clause",
|
||||
"dependencies": {
|
||||
"domelementtype": "^2.3.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/domhandler?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/domutils": {
|
||||
"version": "3.2.2",
|
||||
"resolved": "https://registry.npmjs.org/domutils/-/domutils-3.2.2.tgz",
|
||||
"integrity": "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==",
|
||||
"license": "BSD-2-Clause",
|
||||
"dependencies": {
|
||||
"dom-serializer": "^2.0.0",
|
||||
"domelementtype": "^2.3.0",
|
||||
"domhandler": "^5.0.3"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/domutils?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/encoding-japanese": {
|
||||
"version": "2.3.0",
|
||||
"resolved": "https://registry.npmjs.org/encoding-japanese/-/encoding-japanese-2.3.0.tgz",
|
||||
"integrity": "sha512-eQyh1vzHz13DUkZcJO+0IOAoKXRQwKV5IBffeuYsWZyRLGiSzfzXObCqWvqFXdX0UU8qOk+lBXbkUhMCpdJe4Q==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/entities": {
|
||||
"version": "4.5.0",
|
||||
"resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz",
|
||||
"integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==",
|
||||
"license": "BSD-2-Clause",
|
||||
"engines": {
|
||||
"node": ">=0.12"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/entities?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/he": {
|
||||
"version": "1.2.0",
|
||||
"resolved": "https://registry.npmjs.org/he/-/he-1.2.0.tgz",
|
||||
"integrity": "sha512-F/1DnUGPopORZi0ni+CvrCgHQ5FyEAHRLSApuYWMmrbSwoN2Mn/7k+Gl38gJnR7yyDZk6WLXwiGod1JOWNDKGw==",
|
||||
"license": "MIT",
|
||||
"bin": {
|
||||
"he": "bin/he"
|
||||
}
|
||||
},
|
||||
"node_modules/html-to-text": {
|
||||
"version": "10.0.1",
|
||||
"resolved": "https://registry.npmjs.org/html-to-text/-/html-to-text-10.0.1.tgz",
|
||||
"integrity": "sha512-GiVhRI1BatGARSCmlXWNCjDT0cWrwBWoeduLoV0WSKAgaV/wa+hUWy5LiQLUs4UwiUrE52ZCMfBGiKD87TDPrg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@selderee/plugin-htmlparser2": "~0.12.0",
|
||||
"deepmerge-ts": "^8.0.1",
|
||||
"dom-serializer": "^2.0.0",
|
||||
"htmlparser2": "^10.1.0",
|
||||
"selderee": "~0.12.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.19.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/KillyMXI"
|
||||
}
|
||||
},
|
||||
"node_modules/htmlparser2": {
|
||||
"version": "10.1.0",
|
||||
"resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-10.1.0.tgz",
|
||||
"integrity": "sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ==",
|
||||
"funding": [
|
||||
"https://github.com/fb55/htmlparser2?sponsor=1",
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/fb55"
|
||||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"domelementtype": "^2.3.0",
|
||||
"domhandler": "^5.0.3",
|
||||
"domutils": "^3.2.2",
|
||||
"entities": "^7.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/htmlparser2/node_modules/entities": {
|
||||
"version": "7.0.1",
|
||||
"resolved": "https://registry.npmjs.org/entities/-/entities-7.0.1.tgz",
|
||||
"integrity": "sha512-TWrgLOFUQTH994YUyl1yT4uyavY5nNB5muff+RtWaqNVCAK408b5ZnnbNAUEWLTCpum9w6arT70i1XdQ4UeOPA==",
|
||||
"license": "BSD-2-Clause",
|
||||
"engines": {
|
||||
"node": ">=0.12"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/entities?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/iconv-lite": {
|
||||
"version": "0.7.3",
|
||||
"resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.7.3.tgz",
|
||||
"integrity": "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"safer-buffer": ">= 2.1.2 < 3.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
},
|
||||
"funding": {
|
||||
"type": "opencollective",
|
||||
"url": "https://opencollective.com/express"
|
||||
}
|
||||
},
|
||||
"node_modules/leac": {
|
||||
"version": "0.7.0",
|
||||
"resolved": "https://registry.npmjs.org/leac/-/leac-0.7.0.tgz",
|
||||
"integrity": "sha512-qMrZeyEekgdRQ9o6a4NAB2EQZrv827GJdn1vnapwSJ90hWRB4TzUSunvacPkxQ2TnNqHNI1/zSt0hlo0crG8Jw==",
|
||||
"license": "MIT",
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/KillyMXI"
|
||||
}
|
||||
},
|
||||
"node_modules/libbase64": {
|
||||
"version": "1.3.0",
|
||||
"resolved": "https://registry.npmjs.org/libbase64/-/libbase64-1.3.0.tgz",
|
||||
"integrity": "sha512-GgOXd0Eo6phYgh0DJtjQ2tO8dc0IVINtZJeARPeiIJqge+HdsWSuaDTe8ztQ7j/cONByDZ3zeB325AHiv5O0dg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/libmime": {
|
||||
"version": "5.4.3",
|
||||
"resolved": "https://registry.npmjs.org/libmime/-/libmime-5.4.3.tgz",
|
||||
"integrity": "sha512-di9BoDabBUMqjeD/wGj+hHpSgdqAph5ui7w6OdY6NpzU6O6VFLQsMOg9tqCjm/zf9OHzAM9EZxSOF7uIb8O8Hw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"encoding-japanese": "2.3.0",
|
||||
"iconv-lite": "0.7.3",
|
||||
"libbase64": "1.3.0",
|
||||
"libqp": "2.1.1"
|
||||
}
|
||||
},
|
||||
"node_modules/libqp": {
|
||||
"version": "2.1.1",
|
||||
"resolved": "https://registry.npmjs.org/libqp/-/libqp-2.1.1.tgz",
|
||||
"integrity": "sha512-0Wd+GPz1O134cP62YU2GTOPNA7Qgl09XwCqM5zpBv87ERCXdfDtyKXvV7c9U22yWJh44QZqBocFnXN11K96qow==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/linkify-it": {
|
||||
"version": "5.0.2",
|
||||
"resolved": "https://registry.npmjs.org/linkify-it/-/linkify-it-5.0.2.tgz",
|
||||
"integrity": "sha512-ONTm2jCMAVZjgQa/Fy1kScXsuOoF5NPTsoFBdE1KVIZ2vAh/r9+Bqo+0jINCBYnavTPQZz38QzFTme79ENoN3Q==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/puzrin"
|
||||
},
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/markdown-it"
|
||||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"uc.micro": "^2.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/mailparser": {
|
||||
"version": "3.9.19",
|
||||
"resolved": "https://registry.npmjs.org/mailparser/-/mailparser-3.9.19.tgz",
|
||||
"integrity": "sha512-ik490D4yTo2B9aTdI8au8fOt6xIiC1EbBizi51ZjxX2zBd7k0qvatfvXVH3snXe4NNEovQ7rlP56dChp10pl9A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@zone-eu/mailsplit": "5.4.16",
|
||||
"encoding-japanese": "2.3.0",
|
||||
"he": "1.2.0",
|
||||
"html-to-text": "10.0.1",
|
||||
"iconv-lite": "0.7.3",
|
||||
"libmime": "5.4.3",
|
||||
"linkify-it": "5.0.2",
|
||||
"nodemailer": "9.1.0",
|
||||
"punycode.js": "2.3.1",
|
||||
"tlds": "1.261.0"
|
||||
}
|
||||
},
|
||||
"node_modules/mnemonist": {
|
||||
"version": "0.38.3",
|
||||
"resolved": "https://registry.npmjs.org/mnemonist/-/mnemonist-0.38.3.tgz",
|
||||
|
|
@ -867,90 +563,17 @@
|
|||
"obliterator": "^1.6.1"
|
||||
}
|
||||
},
|
||||
"node_modules/nodemailer": {
|
||||
"version": "9.1.0",
|
||||
"resolved": "https://registry.npmjs.org/nodemailer/-/nodemailer-9.1.0.tgz",
|
||||
"integrity": "sha512-xj1Ri5Sau3qpPffHJwi2bY0oWVVWYq62Ph17l+2v1xXpxjqTls3YPc3L8dub9mcJpxj+1ucNnuYVqLlgh4pmDA==",
|
||||
"license": "MIT-0",
|
||||
"engines": {
|
||||
"node": ">=6.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/obliterator": {
|
||||
"version": "1.6.1",
|
||||
"resolved": "https://registry.npmjs.org/obliterator/-/obliterator-1.6.1.tgz",
|
||||
"integrity": "sha512-9WXswnqINnnhOG/5SLimUlzuU1hFJUc8zkwyD59Sd+dPOMf05PmnYG/d6Q7HZ+KmgkZJa1PxRso6QdM3sTNHig==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/parseley": {
|
||||
"version": "0.13.1",
|
||||
"resolved": "https://registry.npmjs.org/parseley/-/parseley-0.13.1.tgz",
|
||||
"integrity": "sha512-uNBJZzmb60l6p6VWLTmevizNAGnE0xoSf1n0B4q3ntegDNzcS68NRCcBDZTcyXHxt2XhBChsCuqj4M+nChvE/A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"leac": "^0.7.0",
|
||||
"peberminta": "^0.10.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/KillyMXI"
|
||||
}
|
||||
},
|
||||
"node_modules/peberminta": {
|
||||
"version": "0.10.0",
|
||||
"resolved": "https://registry.npmjs.org/peberminta/-/peberminta-0.10.0.tgz",
|
||||
"integrity": "sha512-80B2AsU+I4Qdb0ZAPSfe9UwvGzwkM37IKIFEvdS3D/3Ndgv2bsuJ0bfG1+iEYO+l7Gfd4EUJmuRyq7efLgRMzQ==",
|
||||
"license": "MIT",
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/KillyMXI"
|
||||
}
|
||||
},
|
||||
"node_modules/punycode.js": {
|
||||
"version": "2.3.1",
|
||||
"resolved": "https://registry.npmjs.org/punycode.js/-/punycode.js-2.3.1.tgz",
|
||||
"integrity": "sha512-uxFIHU0YlHYhDQtV4R9J6a52SLx28BCjT+4ieh7IGbgwVJWO+km431c4yRlREUAsAmt/uMjQUyQHNEPf0M39CA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/safer-buffer": {
|
||||
"version": "2.1.2",
|
||||
"resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
|
||||
"integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/selderee": {
|
||||
"version": "0.12.0",
|
||||
"resolved": "https://registry.npmjs.org/selderee/-/selderee-0.12.0.tgz",
|
||||
"integrity": "sha512-b1YMh3+DHZp59DLna3qVwQ5iOla/nrI6mLBNW02XxU77M3046Df6VLkoaJyFz20VsGIG5kkp+FK0kg4K4HnUFw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"parseley": "~0.13.1"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/KillyMXI"
|
||||
}
|
||||
},
|
||||
"node_modules/tlds": {
|
||||
"version": "1.261.0",
|
||||
"resolved": "https://registry.npmjs.org/tlds/-/tlds-1.261.0.tgz",
|
||||
"integrity": "sha512-QXqwfEl9ddlGBaRFXIvNKK6OhipSiLXuRuLJX5DErz0o0Q0rYxulWLdFryTkV5PkdZct5iMInwYEGe/eR++1AA==",
|
||||
"license": "MIT",
|
||||
"bin": {
|
||||
"tlds": "bin.js"
|
||||
}
|
||||
},
|
||||
"node_modules/tslib": {
|
||||
"version": "2.8.1",
|
||||
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
|
||||
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
|
||||
"license": "0BSD"
|
||||
},
|
||||
"node_modules/uc.micro": {
|
||||
"version": "2.1.0",
|
||||
"resolved": "https://registry.npmjs.org/uc.micro/-/uc.micro-2.1.0.tgz",
|
||||
"integrity": "sha512-ARDJmphmdvUk6Glw7y9DQ2bFkKBHwQHLi2lsaH6PPmz/Ka9sFOBsBluozhDltWmnv9u/cF6Rt87znRTPV+yp/A==",
|
||||
"license": "MIT"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -14,9 +14,7 @@
|
|||
"@aws-sdk/client-lambda": "^3.1121.0",
|
||||
"@aws-sdk/client-s3": "^3.1121.0",
|
||||
"@aws-sdk/client-secrets-manager": "^3.1121.0",
|
||||
"@aws-sdk/client-sqs": "^3.1121.0",
|
||||
"@aws-sdk/lib-dynamodb": "^3.1121.0",
|
||||
"csv-parse": "^7.0.2",
|
||||
"mailparser": "^3.9.17"
|
||||
"csv-parse": "^7.0.2"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,478 +0,0 @@
|
|||
import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3";
|
||||
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
|
||||
import {
|
||||
DynamoDBDocumentClient,
|
||||
GetCommand,
|
||||
PutCommand,
|
||||
ScanCommand,
|
||||
BatchWriteCommand,
|
||||
} from "@aws-sdk/lib-dynamodb";
|
||||
import { SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs";
|
||||
import { SecretsManagerClient, GetSecretValueCommand } from "@aws-sdk/client-secrets-manager";
|
||||
import { simpleParser } from "mailparser";
|
||||
|
||||
const s3 = new S3Client();
|
||||
const ddb = DynamoDBDocumentClient.from(new DynamoDBClient());
|
||||
const sqs = new SQSClient();
|
||||
const secrets = new SecretsManagerClient();
|
||||
|
||||
const TABLE_NAME = process.env.TABLE_NAME;
|
||||
const CHANNEL_ID = process.env.PAYROLL_CHANNEL_ID;
|
||||
const QUEUE_URL = process.env.PAYROLL_BATCH_QUEUE_URL;
|
||||
|
||||
let cachedToken;
|
||||
async function getSlackToken() {
|
||||
if (cachedToken) return cachedToken;
|
||||
const { SecretString } = await secrets.send(
|
||||
new GetSecretValueCommand({ SecretId: process.env.SLACK_BOT_TOKEN_SECRET_NAME })
|
||||
);
|
||||
cachedToken = SecretString;
|
||||
return cachedToken;
|
||||
}
|
||||
|
||||
const formatCurrency = (v) =>
|
||||
new Intl.NumberFormat("en-US", {
|
||||
style: "currency",
|
||||
currency: "USD",
|
||||
}).format(Number(v || 0));
|
||||
|
||||
function isAllowedGustoHost(hostname) {
|
||||
const h = (hostname || "").toLowerCase();
|
||||
return h === "gusto.com" || h.endsWith(".gusto.com");
|
||||
}
|
||||
|
||||
function bodyMentionsAllowedGustoUrl(text) {
|
||||
const urlMatches = (text || "").match(/\bhttps?:\/\/[^\s<>"')]+/gi) || [];
|
||||
for (const rawUrl of urlMatches) {
|
||||
try {
|
||||
const parsed = new URL(rawUrl);
|
||||
if (isAllowedGustoHost(parsed.hostname)) return true;
|
||||
} catch {
|
||||
// Ignore malformed URLs in email text.
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function classifyEmail(from, subject, text) {
|
||||
const fromAddr = (from?.text || from || "").toLowerCase();
|
||||
const subj = (subject || "").toLowerCase();
|
||||
|
||||
if (
|
||||
fromAddr.includes("automated@gusto.com") &&
|
||||
subj.includes("payroll confirmation")
|
||||
) {
|
||||
return "employee";
|
||||
}
|
||||
if (
|
||||
fromAddr.includes("gustonoreply@gusto.com") &&
|
||||
subj.includes("payment confirmation")
|
||||
) {
|
||||
return "contractor";
|
||||
}
|
||||
|
||||
const strippedSubj = subj.replace(/^fwd?:\s*/i, "");
|
||||
const bodyMentionsGusto = bodyMentionsAllowedGustoUrl(text || "");
|
||||
|
||||
if (bodyMentionsGusto && strippedSubj.includes("payroll confirmation")) {
|
||||
return "employee";
|
||||
}
|
||||
if (bodyMentionsGusto && strippedSubj.includes("payment confirmation")) {
|
||||
return "contractor";
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
function parseDateFromSubject(subject) {
|
||||
const m = subject.match(/:\s*(\w+,\s*\w+\s+\d+)\s+(?:payroll|payment)\s+confirmation/i);
|
||||
if (!m) return null;
|
||||
const raw = m[1].trim();
|
||||
const year = new Date().getFullYear();
|
||||
const d = new Date(`${raw}, ${year}`);
|
||||
if (isNaN(d)) return null;
|
||||
return {
|
||||
display: raw,
|
||||
iso: d.toISOString().slice(0, 10),
|
||||
};
|
||||
}
|
||||
|
||||
function parseEmployeePayroll(subject, text) {
|
||||
const checkDate = parseDateFromSubject(subject);
|
||||
if (!checkDate) return null;
|
||||
|
||||
const norm = text.replace(/\n/g, " ").replace(/\s+/g, " ");
|
||||
|
||||
const debitMatch = norm.match(
|
||||
/we.ll debit \$([\d,]+\.\d{2}) from the bank account ending in (\d+)/i
|
||||
);
|
||||
if (!debitMatch) return null;
|
||||
|
||||
const totalDebit = parseFloat(debitMatch[1].replace(/,/g, ""));
|
||||
const bankSuffix = debitMatch[2];
|
||||
|
||||
const payPeriodMatch = norm.match(/payroll for the (.+?)\s+pay period/i);
|
||||
const netPayMatch = norm.match(
|
||||
/\$([\d,]+\.\d{2}) will be for your employee net pay/i
|
||||
);
|
||||
const taxesMatch = norm.match(/\$([\d,]+\.\d{2}) will be for taxes/i);
|
||||
const reimbursementsMatch = norm.match(
|
||||
/\$([\d,]+\.\d{2}) will be for reimbursements/i
|
||||
);
|
||||
|
||||
return {
|
||||
checkDate: checkDate.display,
|
||||
dateKey: checkDate.iso,
|
||||
totalDebit,
|
||||
bankSuffix,
|
||||
payPeriod: payPeriodMatch ? payPeriodMatch[1].trim() : null,
|
||||
netPay: netPayMatch ? parseFloat(netPayMatch[1].replace(/,/g, "")) : null,
|
||||
taxes: taxesMatch ? parseFloat(taxesMatch[1].replace(/,/g, "")) : null,
|
||||
reimbursements: reimbursementsMatch
|
||||
? parseFloat(reimbursementsMatch[1].replace(/,/g, ""))
|
||||
: null,
|
||||
};
|
||||
}
|
||||
|
||||
function parseContractorPayment(subject, text) {
|
||||
const checkDate = parseDateFromSubject(subject);
|
||||
if (!checkDate) return null;
|
||||
|
||||
const norm = text.replace(/\n/g, " ").replace(/\s+/g, " ");
|
||||
|
||||
const debitMatch = norm.match(
|
||||
/we.ll debit \$([\d,]+\.\d{2}) from the bank account ending in (\d+)/i
|
||||
);
|
||||
if (!debitMatch) return null;
|
||||
|
||||
const totalDebit = parseFloat(debitMatch[1].replace(/,/g, ""));
|
||||
const bankSuffix = debitMatch[2];
|
||||
|
||||
const nameMatch = norm.match(/at that time for (.+?) to be paid on/i);
|
||||
const contractorName = nameMatch ? nameMatch[1].trim() : "Unknown contractor";
|
||||
|
||||
return {
|
||||
checkDate: checkDate.display,
|
||||
dateKey: checkDate.iso,
|
||||
totalDebit,
|
||||
bankSuffix,
|
||||
contractorName,
|
||||
};
|
||||
}
|
||||
|
||||
function buildCombinedBlocks(dateDisplay, employee, contractors) {
|
||||
const blocks = [
|
||||
{
|
||||
type: "header",
|
||||
text: {
|
||||
type: "plain_text",
|
||||
text: `Payroll Processed — ${dateDisplay}`,
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
if (employee) {
|
||||
const topFields = [
|
||||
{ type: "mrkdwn", text: `*Check Date*\n${employee.checkDate}` },
|
||||
];
|
||||
if (employee.payPeriod) {
|
||||
topFields.unshift({
|
||||
type: "mrkdwn",
|
||||
text: `*Pay Period*\n${employee.payPeriod}`,
|
||||
});
|
||||
}
|
||||
blocks.push({ type: "section", fields: topFields });
|
||||
blocks.push({ type: "divider" });
|
||||
|
||||
const detailFields = [];
|
||||
if (employee.netPay != null) {
|
||||
detailFields.push({
|
||||
type: "mrkdwn",
|
||||
text: `*Employee Net Pay*\n${formatCurrency(employee.netPay)}`,
|
||||
});
|
||||
}
|
||||
if (employee.taxes != null) {
|
||||
detailFields.push({
|
||||
type: "mrkdwn",
|
||||
text: `*Taxes*\n${formatCurrency(employee.taxes)}`,
|
||||
});
|
||||
}
|
||||
if (detailFields.length) blocks.push({ type: "section", fields: detailFields });
|
||||
|
||||
const extraFields = [];
|
||||
if (employee.reimbursements != null) {
|
||||
extraFields.push({
|
||||
type: "mrkdwn",
|
||||
text: `*Reimbursements*\n${formatCurrency(employee.reimbursements)}`,
|
||||
});
|
||||
}
|
||||
extraFields.push({
|
||||
type: "mrkdwn",
|
||||
text: `*Bank Account*\n...${employee.bankSuffix}`,
|
||||
});
|
||||
blocks.push({ type: "section", fields: extraFields });
|
||||
}
|
||||
|
||||
if (contractors.length > 0) {
|
||||
blocks.push({ type: "divider" });
|
||||
blocks.push({
|
||||
type: "section",
|
||||
text: {
|
||||
type: "mrkdwn",
|
||||
text: `*Contractor Payments*`,
|
||||
},
|
||||
});
|
||||
for (const c of contractors) {
|
||||
blocks.push({
|
||||
type: "section",
|
||||
fields: [
|
||||
{ type: "mrkdwn", text: `*Contractor*\n${c.contractorName}` },
|
||||
{ type: "mrkdwn", text: `*Amount*\n${formatCurrency(c.totalDebit)}` },
|
||||
],
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const grandTotal =
|
||||
(employee ? employee.totalDebit : 0) +
|
||||
contractors.reduce((sum, c) => sum + c.totalDebit, 0);
|
||||
|
||||
blocks.push({ type: "divider" });
|
||||
blocks.push({
|
||||
type: "section",
|
||||
text: {
|
||||
type: "mrkdwn",
|
||||
text: `:moneybag: *Total Bank Debit: ${formatCurrency(grandTotal)}*`,
|
||||
},
|
||||
});
|
||||
blocks.push({
|
||||
type: "context",
|
||||
elements: [
|
||||
{ type: "mrkdwn", text: "Source: Gusto payroll confirmation emails" },
|
||||
],
|
||||
});
|
||||
|
||||
return { blocks, grandTotal };
|
||||
}
|
||||
|
||||
async function postSlack(token, blocks, text) {
|
||||
const res = await fetch("https://slack.com/api/chat.postMessage", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({ channel: CHANNEL_ID, blocks, text }),
|
||||
});
|
||||
const data = await res.json();
|
||||
if (!data.ok) {
|
||||
console.error("Slack post failed:", data.error);
|
||||
throw new Error(`Slack API error: ${data.error}`);
|
||||
}
|
||||
return data.ts;
|
||||
}
|
||||
|
||||
const ttl90Days = () => Math.floor(Date.now() / 1000) + 90 * 24 * 60 * 60;
|
||||
|
||||
async function handleS3Event(record) {
|
||||
const bucket = record.s3.bucket.name;
|
||||
const key = decodeURIComponent(record.s3.object.key.replace(/\+/g, " "));
|
||||
|
||||
const { Body } = await s3.send(
|
||||
new GetObjectCommand({ Bucket: bucket, Key: key })
|
||||
);
|
||||
const rawEmail = await Body.transformToByteArray();
|
||||
const parsed = await simpleParser(Buffer.from(rawEmail));
|
||||
|
||||
const type = classifyEmail(parsed.from, parsed.subject, parsed.text);
|
||||
if (!type) {
|
||||
console.log("Unrecognized email, skipping:", parsed.subject);
|
||||
return;
|
||||
}
|
||||
|
||||
if (type === "employee") {
|
||||
const data = parseEmployeePayroll(parsed.subject, parsed.text);
|
||||
if (!data) {
|
||||
console.error("Failed to parse employee payroll:", parsed.subject);
|
||||
return;
|
||||
}
|
||||
|
||||
const pk = `PAYROLL_EMAIL#employee#${data.dateKey}`;
|
||||
const existing = await ddb.send(
|
||||
new GetCommand({ TableName: TABLE_NAME, Key: { pk } })
|
||||
);
|
||||
if (existing.Item) {
|
||||
console.log(`Already recorded: employee ${data.dateKey}`);
|
||||
return;
|
||||
}
|
||||
|
||||
await ddb.send(
|
||||
new PutCommand({
|
||||
TableName: TABLE_NAME,
|
||||
Item: {
|
||||
pk,
|
||||
type: "employee",
|
||||
status: "pending",
|
||||
checkDate: data.checkDate,
|
||||
dateKey: data.dateKey,
|
||||
totalDebit: data.totalDebit,
|
||||
bankSuffix: data.bankSuffix,
|
||||
payPeriod: data.payPeriod,
|
||||
netPay: data.netPay,
|
||||
taxes: data.taxes,
|
||||
reimbursements: data.reimbursements,
|
||||
processedAt: new Date().toISOString(),
|
||||
ttl: ttl90Days(),
|
||||
},
|
||||
})
|
||||
);
|
||||
|
||||
await sqs.send(
|
||||
new SendMessageCommand({
|
||||
QueueUrl: QUEUE_URL,
|
||||
MessageBody: JSON.stringify({ dateKey: data.dateKey }),
|
||||
})
|
||||
);
|
||||
console.log(`Queued employee payroll for ${data.dateKey}`);
|
||||
}
|
||||
|
||||
if (type === "contractor") {
|
||||
const data = parseContractorPayment(parsed.subject, parsed.text);
|
||||
if (!data) {
|
||||
console.error("Failed to parse contractor payment:", parsed.subject);
|
||||
return;
|
||||
}
|
||||
|
||||
const pk = `PAYROLL_EMAIL#contractor#${data.dateKey}#${data.bankSuffix}`;
|
||||
const existing = await ddb.send(
|
||||
new GetCommand({ TableName: TABLE_NAME, Key: { pk } })
|
||||
);
|
||||
if (existing.Item) {
|
||||
console.log(
|
||||
`Already recorded: contractor ${data.contractorName} ${data.dateKey}`
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
await ddb.send(
|
||||
new PutCommand({
|
||||
TableName: TABLE_NAME,
|
||||
Item: {
|
||||
pk,
|
||||
type: "contractor",
|
||||
status: "pending",
|
||||
checkDate: data.checkDate,
|
||||
dateKey: data.dateKey,
|
||||
totalDebit: data.totalDebit,
|
||||
bankSuffix: data.bankSuffix,
|
||||
contractorName: data.contractorName,
|
||||
processedAt: new Date().toISOString(),
|
||||
ttl: ttl90Days(),
|
||||
},
|
||||
})
|
||||
);
|
||||
|
||||
await sqs.send(
|
||||
new SendMessageCommand({
|
||||
QueueUrl: QUEUE_URL,
|
||||
MessageBody: JSON.stringify({ dateKey: data.dateKey }),
|
||||
})
|
||||
);
|
||||
console.log(
|
||||
`Queued contractor: ${data.contractorName} for ${data.dateKey}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleSqsEvent(sqsRecord) {
|
||||
const { dateKey } = JSON.parse(sqsRecord.body);
|
||||
|
||||
const batchPk = `PAYROLL_BATCH#${dateKey}`;
|
||||
const batchCheck = await ddb.send(
|
||||
new GetCommand({ TableName: TABLE_NAME, Key: { pk: batchPk } })
|
||||
);
|
||||
if (batchCheck.Item) {
|
||||
console.log(`Batch already posted for ${dateKey}`);
|
||||
return;
|
||||
}
|
||||
|
||||
const { Items } = await ddb.send(
|
||||
new ScanCommand({
|
||||
TableName: TABLE_NAME,
|
||||
FilterExpression: "begins_with(pk, :prefix) AND #s = :pending",
|
||||
ExpressionAttributeNames: { "#s": "status" },
|
||||
ExpressionAttributeValues: {
|
||||
":prefix": `PAYROLL_EMAIL#`,
|
||||
":pending": "pending",
|
||||
},
|
||||
})
|
||||
);
|
||||
|
||||
const pending = (Items || []).filter((i) => i.dateKey === dateKey);
|
||||
if (pending.length === 0) {
|
||||
console.log(`No pending items for ${dateKey}`);
|
||||
return;
|
||||
}
|
||||
|
||||
const employee = pending.find((i) => i.type === "employee") || null;
|
||||
const contractors = pending.filter((i) => i.type === "contractor");
|
||||
const dateDisplay = pending[0].checkDate;
|
||||
|
||||
const token = await getSlackToken();
|
||||
const { blocks, grandTotal } = buildCombinedBlocks(dateDisplay, employee, contractors);
|
||||
const slackTs = await postSlack(
|
||||
token,
|
||||
blocks,
|
||||
`Payroll processed for ${dateDisplay}: ${formatCurrency(grandTotal)}`
|
||||
);
|
||||
|
||||
await ddb.send(
|
||||
new PutCommand({
|
||||
TableName: TABLE_NAME,
|
||||
Item: {
|
||||
pk: batchPk,
|
||||
dateKey,
|
||||
checkDate: dateDisplay,
|
||||
employeeCount: employee ? 1 : 0,
|
||||
contractorCount: contractors.length,
|
||||
totalDebit: grandTotal,
|
||||
slackTs,
|
||||
processedAt: new Date().toISOString(),
|
||||
ttl: ttl90Days(),
|
||||
},
|
||||
})
|
||||
);
|
||||
|
||||
const updateBatch = pending.map((item) => ({
|
||||
PutRequest: {
|
||||
Item: { ...item, status: "notified" },
|
||||
},
|
||||
}));
|
||||
for (let i = 0; i < updateBatch.length; i += 25) {
|
||||
await ddb.send(
|
||||
new BatchWriteCommand({
|
||||
RequestItems: { [TABLE_NAME]: updateBatch.slice(i, i + 25) },
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
console.log(
|
||||
`Posted batch: ${employee ? 1 : 0} employee + ${contractors.length} contractor(s) for ${dateKey}`
|
||||
);
|
||||
}
|
||||
|
||||
export const handler = async (event) => {
|
||||
if (event.Records?.[0]?.eventSource === "aws:s3") {
|
||||
for (const record of event.Records) {
|
||||
await handleS3Event(record);
|
||||
}
|
||||
} else if (event.Records?.[0]?.eventSource === "aws:sqs") {
|
||||
for (const record of event.Records) {
|
||||
await handleSqsEvent(record);
|
||||
}
|
||||
} else {
|
||||
console.log("Unknown event source:", JSON.stringify(event).slice(0, 200));
|
||||
}
|
||||
|
||||
return { statusCode: 200 };
|
||||
};
|
||||
255
template.yaml
255
template.yaml
|
|
@ -197,11 +197,6 @@ Resources:
|
|||
Status: Enabled
|
||||
ExpirationInDays: 730
|
||||
|
||||
# Defense-in-depth for bank data: deny any non-TLS access. No Allow
|
||||
# statements — access is IAM-only (the Lambda's PutObject grant); unlike
|
||||
# PayrollEmailBucket there is no cross-service principal here. The Deny
|
||||
# covers bucket-level actions too, which is safe because nothing is
|
||||
# granted List/Get — revisit if read access is ever added.
|
||||
BoaRawBucketPolicy:
|
||||
Type: AWS::S3::BucketPolicy
|
||||
Properties:
|
||||
|
|
@ -243,112 +238,11 @@ Resources:
|
|||
SSEType: KMS
|
||||
KMSMasterKeyId: !Ref DynamoDbCmkArn
|
||||
|
||||
PayrollEmailBucket:
|
||||
Type: AWS::S3::Bucket
|
||||
Properties:
|
||||
BucketName: !Sub seahaven-payroll-emails-${AWS::AccountId}
|
||||
PublicAccessBlockConfiguration:
|
||||
BlockPublicAcls: true
|
||||
IgnorePublicAcls: true
|
||||
BlockPublicPolicy: true
|
||||
RestrictPublicBuckets: true
|
||||
LifecycleConfiguration:
|
||||
Rules:
|
||||
- Id: ExpireEmails
|
||||
Status: Enabled
|
||||
ExpirationInDays: 30
|
||||
|
||||
PayrollEmailBucketPolicy:
|
||||
Type: AWS::S3::BucketPolicy
|
||||
Properties:
|
||||
Bucket: !Ref PayrollEmailBucket
|
||||
PolicyDocument:
|
||||
Version: "2012-10-17"
|
||||
Statement:
|
||||
- Sid: AllowSESPut
|
||||
Effect: Allow
|
||||
Principal:
|
||||
Service: ses.amazonaws.com
|
||||
Action: s3:PutObject
|
||||
Resource: !Sub arn:aws:s3:::seahaven-payroll-emails-${AWS::AccountId}/*
|
||||
Condition:
|
||||
StringEquals:
|
||||
AWS:SourceAccount: !Ref AWS::AccountId
|
||||
|
||||
PayrollEmailRule:
|
||||
Type: AWS::SES::ReceiptRule
|
||||
DependsOn: PayrollEmailBucketPolicy
|
||||
Properties:
|
||||
RuleSetName: INBOUND_MAIL
|
||||
After: ExistingRuleSetWorkorderEmailRuleEA29F845-okepxcVarTfu
|
||||
Rule:
|
||||
Name: store-payroll-emails
|
||||
Enabled: true
|
||||
ScanEnabled: true
|
||||
Recipients:
|
||||
- payroll@int.seahaven.com
|
||||
Actions:
|
||||
- S3Action:
|
||||
BucketName: !Ref PayrollEmailBucket
|
||||
ObjectKeyPrefix: inbound/
|
||||
|
||||
PayrollBatchQueue:
|
||||
Type: AWS::SQS::Queue
|
||||
Properties:
|
||||
QueueName: payments-payroll-batch
|
||||
DelaySeconds: 600
|
||||
MessageRetentionPeriod: 86400
|
||||
VisibilityTimeout: 60
|
||||
RedrivePolicy:
|
||||
deadLetterTargetArn: !GetAtt PayrollBatchDLQ.Arn
|
||||
maxReceiveCount: 3
|
||||
|
||||
# Audit L-15: payroll-batch messages were lost after max receives. 14-day
|
||||
# retention so a failure on Friday survives the weekend.
|
||||
PayrollBatchDLQ:
|
||||
Type: AWS::SQS::Queue
|
||||
Properties:
|
||||
QueueName: payments-payroll-batch-dlq
|
||||
MessageRetentionPeriod: 1209600
|
||||
|
||||
PayrollBatchDLQAlarm:
|
||||
Type: AWS::CloudWatch::Alarm
|
||||
Properties:
|
||||
AlarmName: payments-payroll-batch-dlq-messages
|
||||
AlarmDescription: Failed payroll-batch messages landed in the DLQ
|
||||
Namespace: AWS/SQS
|
||||
MetricName: ApproximateNumberOfMessagesVisible
|
||||
Dimensions:
|
||||
- Name: QueueName
|
||||
Value: !GetAtt PayrollBatchDLQ.QueueName
|
||||
Statistic: Maximum
|
||||
Period: 300
|
||||
EvaluationPeriods: 1
|
||||
Threshold: 0
|
||||
ComparisonOperator: GreaterThanThreshold
|
||||
TreatMissingData: notBreaching
|
||||
# ALARM-only notification by convention — no OK/recovery action
|
||||
AlarmActions:
|
||||
- !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts
|
||||
|
||||
# Async-invoke OnFailure DLQs (INFRA-41 / H-8). Reconciles the interim
|
||||
# CLI-created queues into CloudFormation. Names are CFN-generated to avoid
|
||||
# colliding with the live interim payments-<fn>-dlq queues (deleted after
|
||||
# this deploy). 14-day retention mirrors the payments-payroll-batch DLQ so a
|
||||
# Friday failure survives the weekend.
|
||||
# Distinct -async-dlq name (not the live interim payments-<fn>-dlq) so this
|
||||
# CFN queue does not collide with the queue being deleted post-deploy.
|
||||
ProcessPaymentCsvDLQ:
|
||||
Type: AWS::SQS::Queue
|
||||
Properties:
|
||||
QueueName: payments-processPaymentCsv-async-dlq
|
||||
MessageRetentionPeriod: 1209600 # 14d, matches payments-payroll-batch-dlq
|
||||
|
||||
ProcessPayrollEmailDLQ:
|
||||
Type: AWS::SQS::Queue
|
||||
Properties:
|
||||
QueueName: payments-processPayrollEmail-async-dlq
|
||||
MessageRetentionPeriod: 1209600 # 14d, matches payments-payroll-batch-dlq
|
||||
MessageRetentionPeriod: 1209600 # 14d
|
||||
|
||||
# ALARM-only Lambda Errors alarms (INFRA-41 / H-8). Threshold > 0 on the
|
||||
# Errors Sum, no OK/recovery action by convention.
|
||||
|
|
@ -371,25 +265,6 @@ Resources:
|
|||
AlarmActions:
|
||||
- !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts
|
||||
|
||||
ProcessPayrollEmailErrorsAlarm:
|
||||
Type: AWS::CloudWatch::Alarm
|
||||
Properties:
|
||||
AlarmName: payments-processPayrollEmail-errors
|
||||
AlarmDescription: payments-processPayrollEmail invocation errors
|
||||
Namespace: AWS/Lambda
|
||||
MetricName: Errors
|
||||
Dimensions:
|
||||
- Name: FunctionName
|
||||
Value: !Ref ProcessPayrollEmailFunction
|
||||
Statistic: Sum
|
||||
Period: 300
|
||||
EvaluationPeriods: 1
|
||||
Threshold: 0
|
||||
ComparisonOperator: GreaterThanThreshold
|
||||
TreatMissingData: notBreaching
|
||||
AlarmActions:
|
||||
- !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts
|
||||
|
||||
# ── Lambda Errors alarms (Wave 1) ──────────────────────────────────────────
|
||||
# Clone of ProcessPaymentCsvErrorsAlarm for the remaining functions. AWS/Lambda
|
||||
# Errors, Sum over 5m, threshold > 0, ALARM-only by convention.
|
||||
|
|
@ -491,25 +366,6 @@ Resources:
|
|||
AlarmActions:
|
||||
- !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts
|
||||
|
||||
ProcessPayrollEmailThrottlesAlarm:
|
||||
Type: AWS::CloudWatch::Alarm
|
||||
Properties:
|
||||
AlarmName: payments-processPayrollEmail-throttles
|
||||
AlarmDescription: payments-processPayrollEmail invocations throttled
|
||||
Namespace: AWS/Lambda
|
||||
MetricName: Throttles
|
||||
Dimensions:
|
||||
- Name: FunctionName
|
||||
Value: !Ref ProcessPayrollEmailFunction
|
||||
Statistic: Sum
|
||||
Period: 300
|
||||
EvaluationPeriods: 1
|
||||
Threshold: 0
|
||||
ComparisonOperator: GreaterThanThreshold
|
||||
TreatMissingData: notBreaching
|
||||
AlarmActions:
|
||||
- !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts
|
||||
|
||||
FetchBoaTransactionsThrottlesAlarm:
|
||||
Type: AWS::CloudWatch::Alarm
|
||||
Properties:
|
||||
|
|
@ -608,25 +464,6 @@ Resources:
|
|||
AlarmActions:
|
||||
- !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts
|
||||
|
||||
ProcessPayrollEmailDurationAlarm:
|
||||
Type: AWS::CloudWatch::Alarm
|
||||
Properties:
|
||||
AlarmName: payments-processPayrollEmail-duration
|
||||
AlarmDescription: payments-processPayrollEmail approaching timeout (~80% of 60s)
|
||||
Namespace: AWS/Lambda
|
||||
MetricName: Duration
|
||||
Dimensions:
|
||||
- Name: FunctionName
|
||||
Value: !Ref ProcessPayrollEmailFunction
|
||||
Statistic: Maximum
|
||||
Period: 300
|
||||
EvaluationPeriods: 1
|
||||
Threshold: 48000
|
||||
ComparisonOperator: GreaterThanThreshold
|
||||
TreatMissingData: notBreaching
|
||||
AlarmActions:
|
||||
- !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts
|
||||
|
||||
FetchBoaTransactionsDurationAlarm:
|
||||
Type: AWS::CloudWatch::Alarm
|
||||
Properties:
|
||||
|
|
@ -811,8 +648,8 @@ Resources:
|
|||
AlarmActions:
|
||||
- !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts
|
||||
|
||||
# Messages-present alarms on the async-invoke OnFailure DLQs, mirroring
|
||||
# PayrollBatchDLQAlarm. Threshold > 0 on the visible-message count, ALARM-only.
|
||||
# Messages-present alarms on the async-invoke OnFailure DLQs,
|
||||
# Threshold > 0 on the visible-message count, ALARM-only.
|
||||
ProcessPaymentCsvDLQAlarm:
|
||||
Type: AWS::CloudWatch::Alarm
|
||||
Properties:
|
||||
|
|
@ -833,32 +670,6 @@ Resources:
|
|||
AlarmActions:
|
||||
- !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts
|
||||
|
||||
ProcessPayrollEmailDLQAlarm:
|
||||
Type: AWS::CloudWatch::Alarm
|
||||
Properties:
|
||||
AlarmName: payments-processPayrollEmail-async-dlq-messages
|
||||
AlarmDescription: Failed processPayrollEmail async invocations landed in the DLQ
|
||||
Namespace: AWS/SQS
|
||||
MetricName: ApproximateNumberOfMessagesVisible
|
||||
Dimensions:
|
||||
- Name: QueueName
|
||||
Value: !GetAtt ProcessPayrollEmailDLQ.QueueName
|
||||
Statistic: Maximum
|
||||
Period: 300
|
||||
EvaluationPeriods: 1
|
||||
Threshold: 0
|
||||
ComparisonOperator: GreaterThanThreshold
|
||||
TreatMissingData: notBreaching
|
||||
# ALARM-only notification by convention — no OK/recovery action
|
||||
AlarmActions:
|
||||
- !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:site-alerts
|
||||
|
||||
ProcessPayrollEmailLogGroup:
|
||||
Type: AWS::Logs::LogGroup
|
||||
Properties:
|
||||
LogGroupName: /aws/lambda/payments-processPayrollEmail
|
||||
RetentionInDays: 60
|
||||
|
||||
ProcessPaymentCsvLogGroup:
|
||||
Type: AWS::Logs::LogGroup
|
||||
Properties:
|
||||
|
|
@ -889,63 +700,6 @@ Resources:
|
|||
LogGroupName: /aws/lambda/payments-expenseProcessor
|
||||
RetentionInDays: 60
|
||||
|
||||
ProcessPayrollEmailFunction:
|
||||
Type: AWS::Serverless::Function
|
||||
Properties:
|
||||
FunctionName: payments-processPayrollEmail
|
||||
Handler: src/processPayrollEmail.handler
|
||||
Timeout: 60
|
||||
EventInvokeConfig:
|
||||
MaximumRetryAttempts: 2
|
||||
MaximumEventAgeInSeconds: 21600
|
||||
DestinationConfig:
|
||||
OnFailure:
|
||||
Type: SQS
|
||||
Destination: !GetAtt ProcessPayrollEmailDLQ.Arn
|
||||
Environment:
|
||||
Variables:
|
||||
SLACK_BOT_TOKEN_SECRET_NAME: payments-dashboard/slack-bot-token
|
||||
PAYROLL_CHANNEL_ID: C0AV5RBMYKU
|
||||
PAYROLL_BATCH_QUEUE_URL: !Ref PayrollBatchQueue
|
||||
Events:
|
||||
EmailReceived:
|
||||
Type: S3
|
||||
Properties:
|
||||
Bucket: !Ref PayrollEmailBucket
|
||||
Events: s3:ObjectCreated:*
|
||||
Filter:
|
||||
S3Key:
|
||||
Rules:
|
||||
- Name: prefix
|
||||
Value: inbound/
|
||||
PayrollBatch:
|
||||
Type: SQS
|
||||
Properties:
|
||||
Queue: !GetAtt PayrollBatchQueue.Arn
|
||||
BatchSize: 1
|
||||
Policies:
|
||||
- S3ReadPolicy:
|
||||
BucketName: !Sub seahaven-payroll-emails-${AWS::AccountId}
|
||||
- DynamoDBCrudPolicy:
|
||||
TableName: !Ref DashboardTable
|
||||
- Version: "2012-10-17"
|
||||
Statement:
|
||||
- Effect: Allow
|
||||
Action:
|
||||
- kms:Decrypt
|
||||
- kms:GenerateDataKey
|
||||
- kms:DescribeKey
|
||||
Resource: !Ref DynamoDbCmkArn
|
||||
- Version: "2012-10-17"
|
||||
Statement:
|
||||
- Effect: Allow
|
||||
Action: secretsmanager:GetSecretValue
|
||||
Resource: !Sub arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:payments-dashboard/slack-bot-token-*
|
||||
- SQSSendMessagePolicy:
|
||||
QueueName: !GetAtt PayrollBatchQueue.QueueName
|
||||
- SQSPollerPolicy:
|
||||
QueueName: !GetAtt PayrollBatchQueue.QueueName
|
||||
|
||||
ProcessPaymentCsvFunction:
|
||||
Type: AWS::Serverless::Function
|
||||
Properties:
|
||||
|
|
@ -1171,9 +925,6 @@ Outputs:
|
|||
StaticOutboundIp:
|
||||
Description: Static IP for BoA API whitelist
|
||||
Value: !Ref NatEip
|
||||
PayrollEmailBucket:
|
||||
Description: S3 bucket for inbound payroll emails from SES
|
||||
Value: !Ref PayrollEmailBucket
|
||||
ExpenseSlackEventsUrl:
|
||||
Description: URL for Expense Approval Bot Slack Event Subscriptions
|
||||
Value: !Sub https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/slack/expense-events
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue