2026-04-13 16:24:20 -04:00
# Payments Dashboard
2026-06-11 14:13:31 -04:00
2026-09-16 18:29:01 +00:00
2026-06-11 14:13:31 -04:00
2026-09-28 19:41:09 +00:00
HCP Terraform application that ingests payment CSVs, syncs check data with Bank of America CashPro APIs, surfaces an outstanding-payments dashboard in Slack, and routes expense approvals through a Slack reaction-driven workflow. Workspaces `payments-dashboard-dev` and `payments-dashboard-prod` share tag `app:payments-dashboard` (trigger prefix `terraform/**` ). Push to `main` deploys function zips to dev. A human GitHub Release deploys prod.
2026-04-13 16:24:20 -04:00
## Architecture
2026-04-30 14:19:06 -04:00
- **ProcessPaymentCsv** — Lambda triggered by S3 CSV upload. Parses Stampli payment exports, upserts to DynamoDB, and submits new/cancelled checks to the CashPro Check Management API.
2026-09-16 18:29:01 +00:00
- **FetchBoaTransactions** — Scheduled Lambda. Weekdays 9am ET it calls the CashPro **previous-day** Transaction Inquiry (authoritative sweep, trailing 7 days); weekdays at 16:00/19:00/22:00 UTC (~12/3/6pm ET, fixed-UTC so it drifts an hour in winter) it calls the **current-day** inquiry for same-day visibility (EventBridge `Input: {"endpoint":"current-day"}` , today-only, staleness sweep skipped). Every run archives the exact raw response to the `seahaven-payments-boa-raw-*` bucket (`raw/<endpoint>/<fromDate>_<toDate>/<runAt>.json` , SSE-S3, 730-day lifecycle, PutObject-only grant; Retain-protected in Terraform) and upserts per-date `boa_balance#<asOfDate>#<endpoint>` snapshots (latest-wins on `run_at` , no TTL) from the Summary rows. Classifies each transaction and reconciles onto DynamoDB payment records. Event payload: `{fromDate?, toDate?, endpoint?}` (endpoint allowlisted and validated; unknown fields ignored). Intraday runs are disable-able as a unit via the `IntradaySchedule` rule. See [Bank reconciliation ](#bank-reconciliation-fetchboatransactions ).
2026-07-22 16:38:56 -04:00
- **SlackAppHome** — Lambda behind API Gateway (`POST /slack/events` ). Verifies the Slack signing secret (HMAC-SHA256, 5-minute replay window) before processing, then renders the payments dashboard on the Slack App Home tab with outstanding aging buckets, drill-down modals, and an always-visible "Returned — Needs Action" queue (bank-returned payments awaiting a reissue/void decision, sorted oldest return first). Returned records are excluded from Outstanding totals; terminal voided-and-bounced records appear in neither (audit trail only).
2026-04-13 16:24:20 -04:00
2026-05-12 13:08:42 -04:00
- **ExpenseReceiver** — Lambda behind API Gateway (`POST /slack/expense-events` ). Verifies the Slack signing secret (HMAC-SHA256), handles URL verification challenges, and async-invokes ExpenseProcessor. Runs outside VPC.
- **ExpenseProcessor** — Async Lambda invoked by ExpenseReceiver. Processes `:white_check_mark:` reactions to advance expense messages through a four-stage Slack channel pipeline: Submitted → Processed → Authorized → Matched. Runs outside VPC.
2026-09-16 18:29:01 +00:00
ProcessPaymentCsv, FetchBoaTransactions, and SlackAppHome run inside a VPC with a NAT Gateway for a static outbound IP (required by BoA IP whitelisting). ExpenseReceiver and ExpenseProcessor run outside the VPC.
2026-05-12 13:08:42 -04:00
## Expense Approval Bot
Reaction-driven workflow that routes expense submissions through four Slack channels. A separate Slack app ("Expense Approval Bot") posts to a **Submitted** channel. Users react with :white_check_mark: to advance the message to the next stage.
**Channel pipeline:**
2026-10-01 21:39:38 -04:00
| Stage | Channel ID | Action on :white_check_mark: |
| ---------- | ------------- | ------------------------------------------- |
| Submitted | `C0AQ2AWLNEN` | Thread reply on original, copy to Processed |
| Processed | `C0APLSGABAB` | Delete from Processed, post to Authorized |
| Authorized | `C0AQ09CDJH4` | Delete from Authorized, post to Matched |
| Matched | `C0APYUM1JFP` | Terminal stage (no further routing) |
2026-05-12 13:08:42 -04:00
**Architecture:** Two Lambdas — ExpenseReceiver (HTTP endpoint, signature verification, async invoke) and ExpenseProcessor (business logic). This is the same receiver/processor pattern used for Slack's 3-second timeout requirement.
**Secrets (Secrets Manager):**
2026-10-01 21:39:38 -04:00
| Secret | Purpose |
| ------------------------------------------------- | ------------------------------------------------ |
| `payments-dashboard/expense-slack-token` | Slack Bot token for the Expense Approval Bot app |
| `payments-dashboard/expense-slack-signing-secret` | Slack signing secret for request verification |
2026-04-30 14:19:06 -04:00
2026-04-13 16:24:20 -04:00
## BoA CashPro API Integration
Two separate CashPro APIs are used, each with its own OAuth credentials:
2026-10-01 21:39:38 -04:00
| API | Purpose | Endpoint |
| ------------------------------- | ------------------------------- | ---------------------------------------------------------- |
| Check Management | Issue and cancel checks | `/cashpro/checkmanagement/v1/check-issues` |
2026-04-13 16:24:20 -04:00
| Reporting (Transaction Inquiry) | Fetch previous-day transactions | `/cashpro/reporting/v1/transaction-inquiries/previous-day` |
**Authentication flow:**
2026-10-01 21:39:38 -04:00
2026-04-13 16:24:20 -04:00
1. POST to `/authn/v1/client-authentication` with `applicationID` , `client_id` , and `client_secret`
2. Receive a Bearer `access_token` (valid 1 hour)
3. Pass the token in the `Authorization` header for subsequent API calls
**Base URLs:**
2026-10-01 21:39:38 -04:00
2026-04-13 16:24:20 -04:00
- Production: `https://api.bofa.com`
- Sandbox: `https://api-sb.bofa.com`
feat(fetchboa): bank-truth reconciliation v2 — returns, redeposits, ACH confirmation (#73)
* feat(fetchboa): v2 return/redeposit-aware reconciliation (#66)
The two-code 475/255 filter missed every return on the Jan-Jul statement
(29 ARP refer-to-maker credits, 24 electronic return credits, and the
redeposit cycles), leaving 5 paid-then-returned checks stuck at Cleared
($5,256.62, vendors unpaid). Rewrite fetchBoaTransactions around a pure
reconciliation module (src/boaRecon.js) covered by node:test:
- BAI transaction-code event map (only 475 = check paid is confirmed;
remaining codes pend the enumeration replay) with a description-text
fallback classifier for ARP refer-to-maker, return-of-posted-check
(check-numbered and electronic) and ACH DES:PAYMENTS formats. Unknown
check-shaped transactions are logged and counted, never dropped.
- Matching on check number AND amount over all candidates; wrong-amount
or collapsed-number postings fall back to a unique exact-amount match
within checks issued in the last 120 days; ambiguity means unmatched
with no write.
- Transitions always set BOTH status and clear_status (the missed
returns slipped through the divergence between them). clear_status is
bank truth: returns apply even to Cleared records, a second paid debit
on a Returned check is a redeposit back to Cleared, and a return on a
voided check is terminal voided-and-bounced. Every applied event is
appended to a history list; identical replayed events are noops.
- Optional {fromDate, toDate} replay payload (strictly validated) for
gap replays and the BAI-code enumeration runs; default stays
yesterday.
- Each run writes a boa_recon#<runDate> summary item (90-day TTL) with
per-event counts, unmatched check numbers/amounts, and unknown codes;
unmatched/unknown also console.error (Slack alerting is #71).
ACH transactions are classified but not yet acted on; bank-confirming
ACH lands with #69.
* feat(ach): bank-confirm ACH, drop CSV-date auto-clear (#69)
processPaymentCsv auto-cleared ACH the moment send_payment_on passed,
but the bank disagrees often enough to matter: settlement lags the send
date by up to 8 days, settled ACH can bounce and re-debit (Uline
$19,281.12, Heights $10,000.00 on the 5/26 overdrawn week), and voided
ACH that settled anyway returned via electronic credits invisible to a
Check-only feed. Move ACH to bank confirmation:
- processPaymentCsv: ACH rows keep their Stampli status; the send-date
auto-clear is removed. The bankConfirmed protection is unchanged, so
bank-cleared records still cannot be moved backward by the CSV.
- fetchBoaTransactions scans all payments (method filter dropped) and
processes ACH CCD lines: match by stored pmt_id first (reversals
reuse the original PMT id), then by the Stampli payment number
embedded in PMT INFO (internal spaces stripped, amount must agree),
then vendor + exact amount within send_payment_on -2..+14 days.
Settled debit -> Cleared/Cleared with dates, pmt_id persisted on the
record; credit reusing the pmt_id (or a unique same-amount return
credit against a bank-confirmed ACH) -> clear_status=Returned +
returned_date + history event. Ambiguity is unmatched, no write.
Existing DDB ACH records already Cleared by the old auto-clear are
unaffected: bank confirmation is additive and the CSV path never moves
a record backward.
Handler event contract extended (optional fromDate/toDate); cross-family
review required before merge.
* fix(fetchboa): close review findings — coverage gap, matcher corroboration, replay identity (#66)
Security-review gate (detector fan-out + verifier + GPT-4.1 cross-family)
blocked on F6 and confirmed six lower findings; all are closed here.
- F6 (HIGH): the default run now queries a trailing 3-day window
(today-3..today-1) so the Monday run covers Friday-Sunday postings the
previous-day cadence silently skipped. A per-run staleness sweep flags
never-bank-confirmed payments (ACH > 16 days, checks > 60 days) in the
summary and via console.error.
- F2: replay identity is {event, date, amount} — bankRef excluded (feeds
omit/reformat it between runs); a paid event dated on/before the
latest return in history is a replayed original, never a redeposit;
rows without a valid valueDate are routed to unmatched instead of
being applied under a substituted date; within a day, debits sort
before credits.
- F1: a check-number match with the wrong amount no longer falls
through to the amount fallback (altered-check/collapsed-posting is a
human-review case); the amount fallback requires digit-subsequence
corroboration between posting and candidate numbers; check_return
fallback requires a bank-confirmed candidate.
- F4: the ach_return amount fallback requires cleared_date within 45
days before the credit; an unattributable PMT id is an unmatched
alert, never an amount guess.
- F5: the pmt_id rung requires amount equality; mismatch is the
partial-reversal human case.
- F3: vendor prefix matching requires >= 10 normalized chars (short
names must match exactly); candidates with a conflicting stored
pmt_id are excluded; vendor+amount matches are audit-logged.
- F7: payment writes are conditioned on the snapshot's
status/clear_status and retried once against a fresh read; residual
conflicts are counted, not silently interleaved.
- F9/summary bounds: run summary pk is append-only
(boa_recon#<from>_<to>#<runAt>); unmatched list capped at 50, unknown
codes at 20 keys/16 chars, stale list at 50 (full counts kept).
- ReDoS: description bounded to 500 chars before classification;
CHECK/embedded-number regexes use bounded quantifiers.
- FBOA2-1: both BoA res.json() parses are wrapped so a malformed 200
throws a status-only error and cannot leak a body snippet.
Handler event contract extended (optional fromDate/toDate); cross-family
review required before merge.
* fix(boaRecon): add method=Check filter to matchElectronicReturn
Electronic returns only apply to checks, but the matcher was not
filtering by method, so an electronic return could incorrectly
match against a same-amount, bank-confirmed ACH record.
2026-07-21 22:07:37 -04:00
## Bank reconciliation (fetchBoaTransactions)
The scheduled Lambda reconciles the Previous Day feed onto `payment#` records (pure logic lives in `src/boaRecon.js` , covered by `npm test` ).
2026-07-22 15:40:49 -04:00
**Classification.** The API's Detail rows carry the statement line in `detailText` (ACH `DES:PAYMENTS ID:PMT` text, return descriptions) and the posting date in `asOfDate` (ISO); `transactionType: "Summary"` rows are balance/total lines and classify as `summary` (skipped, counted). Only 475 check-paid rows omit `detailText` — their check number rides in `customerReference` (all-zeros references normalize to empty, never check number 0). Classification runs the empirically-enumerated BAI code map first, then the description-text classifiers:
2026-10-01 21:39:38 -04:00
| BAI code | Feed label | Event |
| --------------- | -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 475 | Check Paid | `check_paid` (number in `customerReference` ) |
| 255 | Check Posted and Returned CR | `check_return` (number in `customerReference` ) |
| 252 | Debit Reversal Credit | `check_return` (second return-credit code) |
| 266 | Return Item Credit | text decides (`ach_return` on PMT text, `check_return` /`electronic_return` on return text); bare rows fall back to `electronic_return` ; unreadable text stays `unknown` (loud, no write) |
| 455 | Preauthorized ACH Debit | `ach_debit` via `DES:PAYMENTS` text; DES-less = third-party autopay → ignored |
| 170/201/470/481 | totals, transfers, loan payments | ignored |
2026-07-22 15:40:49 -04:00
2026-10-01 21:39:38 -04:00
A hard code-map event wins over description text; 266/455 carry _fallback_ events consulted only when the text yields nothing. Unmapped codes on check-shaped transactions are logged (`console.error` ) and counted in the run summary — never silently dropped.
feat(fetchboa): bank-truth reconciliation v2 — returns, redeposits, ACH confirmation (#73)
* feat(fetchboa): v2 return/redeposit-aware reconciliation (#66)
The two-code 475/255 filter missed every return on the Jan-Jul statement
(29 ARP refer-to-maker credits, 24 electronic return credits, and the
redeposit cycles), leaving 5 paid-then-returned checks stuck at Cleared
($5,256.62, vendors unpaid). Rewrite fetchBoaTransactions around a pure
reconciliation module (src/boaRecon.js) covered by node:test:
- BAI transaction-code event map (only 475 = check paid is confirmed;
remaining codes pend the enumeration replay) with a description-text
fallback classifier for ARP refer-to-maker, return-of-posted-check
(check-numbered and electronic) and ACH DES:PAYMENTS formats. Unknown
check-shaped transactions are logged and counted, never dropped.
- Matching on check number AND amount over all candidates; wrong-amount
or collapsed-number postings fall back to a unique exact-amount match
within checks issued in the last 120 days; ambiguity means unmatched
with no write.
- Transitions always set BOTH status and clear_status (the missed
returns slipped through the divergence between them). clear_status is
bank truth: returns apply even to Cleared records, a second paid debit
on a Returned check is a redeposit back to Cleared, and a return on a
voided check is terminal voided-and-bounced. Every applied event is
appended to a history list; identical replayed events are noops.
- Optional {fromDate, toDate} replay payload (strictly validated) for
gap replays and the BAI-code enumeration runs; default stays
yesterday.
- Each run writes a boa_recon#<runDate> summary item (90-day TTL) with
per-event counts, unmatched check numbers/amounts, and unknown codes;
unmatched/unknown also console.error (Slack alerting is #71).
ACH transactions are classified but not yet acted on; bank-confirming
ACH lands with #69.
* feat(ach): bank-confirm ACH, drop CSV-date auto-clear (#69)
processPaymentCsv auto-cleared ACH the moment send_payment_on passed,
but the bank disagrees often enough to matter: settlement lags the send
date by up to 8 days, settled ACH can bounce and re-debit (Uline
$19,281.12, Heights $10,000.00 on the 5/26 overdrawn week), and voided
ACH that settled anyway returned via electronic credits invisible to a
Check-only feed. Move ACH to bank confirmation:
- processPaymentCsv: ACH rows keep their Stampli status; the send-date
auto-clear is removed. The bankConfirmed protection is unchanged, so
bank-cleared records still cannot be moved backward by the CSV.
- fetchBoaTransactions scans all payments (method filter dropped) and
processes ACH CCD lines: match by stored pmt_id first (reversals
reuse the original PMT id), then by the Stampli payment number
embedded in PMT INFO (internal spaces stripped, amount must agree),
then vendor + exact amount within send_payment_on -2..+14 days.
Settled debit -> Cleared/Cleared with dates, pmt_id persisted on the
record; credit reusing the pmt_id (or a unique same-amount return
credit against a bank-confirmed ACH) -> clear_status=Returned +
returned_date + history event. Ambiguity is unmatched, no write.
Existing DDB ACH records already Cleared by the old auto-clear are
unaffected: bank confirmation is additive and the CSV path never moves
a record backward.
Handler event contract extended (optional fromDate/toDate); cross-family
review required before merge.
* fix(fetchboa): close review findings — coverage gap, matcher corroboration, replay identity (#66)
Security-review gate (detector fan-out + verifier + GPT-4.1 cross-family)
blocked on F6 and confirmed six lower findings; all are closed here.
- F6 (HIGH): the default run now queries a trailing 3-day window
(today-3..today-1) so the Monday run covers Friday-Sunday postings the
previous-day cadence silently skipped. A per-run staleness sweep flags
never-bank-confirmed payments (ACH > 16 days, checks > 60 days) in the
summary and via console.error.
- F2: replay identity is {event, date, amount} — bankRef excluded (feeds
omit/reformat it between runs); a paid event dated on/before the
latest return in history is a replayed original, never a redeposit;
rows without a valid valueDate are routed to unmatched instead of
being applied under a substituted date; within a day, debits sort
before credits.
- F1: a check-number match with the wrong amount no longer falls
through to the amount fallback (altered-check/collapsed-posting is a
human-review case); the amount fallback requires digit-subsequence
corroboration between posting and candidate numbers; check_return
fallback requires a bank-confirmed candidate.
- F4: the ach_return amount fallback requires cleared_date within 45
days before the credit; an unattributable PMT id is an unmatched
alert, never an amount guess.
- F5: the pmt_id rung requires amount equality; mismatch is the
partial-reversal human case.
- F3: vendor prefix matching requires >= 10 normalized chars (short
names must match exactly); candidates with a conflicting stored
pmt_id are excluded; vendor+amount matches are audit-logged.
- F7: payment writes are conditioned on the snapshot's
status/clear_status and retried once against a fresh read; residual
conflicts are counted, not silently interleaved.
- F9/summary bounds: run summary pk is append-only
(boa_recon#<from>_<to>#<runAt>); unmatched list capped at 50, unknown
codes at 20 keys/16 chars, stale list at 50 (full counts kept).
- ReDoS: description bounded to 500 chars before classification;
CHECK/embedded-number regexes use bounded quantifiers.
- FBOA2-1: both BoA res.json() parses are wrapped so a malformed 200
throws a status-only error and cannot leak a body snippet.
Handler event contract extended (optional fromDate/toDate); cross-family
review required before merge.
* fix(boaRecon): add method=Check filter to matchElectronicReturn
Electronic returns only apply to checks, but the matcher was not
filtering by method, so an electronic return could incorrectly
match against a same-amount, bank-confirmed ACH record.
2026-07-21 22:07:37 -04:00
**Matching.** Check events match on check number AND amount, evaluating all candidates (bank postings can drop/collapse digits on long check numbers). A number match with the WRONG amount never auto-resolves — it is the altered-check/collapsed-posting signal and goes to unmatched for human review. An unknown number falls back to an exact-amount match within checks issued in the last 120 days, and only when the posting's digits are a subsequence of the candidate's check number (or vice versa); return credits additionally require a bank-confirmed candidate. Zero or multiple fallback candidates means unmatched, recorded in the run summary with no write. Electronic returns (no check number) match by exact amount among bank-confirmed payments.
**ACH.** ACH is bank-confirmed too (#69 ): `processPaymentCsv` no longer auto-clears ACH on the send date (rows keep their Stampli status until the bank settles). ACH CCD lines match by the stored `pmt_id` first (reversals reuse the original `PMT <id>` ; a `pmt_id` match with the wrong amount is the partial-reversal human case and goes to unmatched), then by the Stampli payment number embedded at the end of `PMT INFO` (internal spaces stripped, amount must agree), then by vendor + exact amount within `send_payment_on` − `pmt_id` differs are excluded; vendor prefix matching requires ≥ 10 normalized chars). A settled debit clears the record and persists `pmt_id` . A return credit whose `PMT <id>` attributes to no stored `pmt_id` is an unmatched alert ("unknown PMT id"); credits without a `PMT <id>` may fall back to a unique same-amount match against a bank-confirmed ACH whose `cleared_date` is within the prior 45 days.
**Staleness sweep.** Every run also flags never-bank-confirmed payments: ACH with no `clear_status` sent more than 16 days ago (listed in the summary, capped at 50, plus a total count) and checks issued more than 60 days ago with no `clear_status` (count only). Both alert via `console.error` .
**State transitions.** Every write sets BOTH `status` and `clear_status` (`clear_status` is bank truth; "Cleared without a subsequent return is permanent" keys off it):
2026-10-01 21:39:38 -04:00
| Bank event | Result |
| ----------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| Paid debit | `status=Cleared` , `clear_status=Cleared` , `paid_date` , `cleared_date` , `bank_reference` |
feat(fetchboa): bank-truth reconciliation v2 — returns, redeposits, ACH confirmation (#73)
* feat(fetchboa): v2 return/redeposit-aware reconciliation (#66)
The two-code 475/255 filter missed every return on the Jan-Jul statement
(29 ARP refer-to-maker credits, 24 electronic return credits, and the
redeposit cycles), leaving 5 paid-then-returned checks stuck at Cleared
($5,256.62, vendors unpaid). Rewrite fetchBoaTransactions around a pure
reconciliation module (src/boaRecon.js) covered by node:test:
- BAI transaction-code event map (only 475 = check paid is confirmed;
remaining codes pend the enumeration replay) with a description-text
fallback classifier for ARP refer-to-maker, return-of-posted-check
(check-numbered and electronic) and ACH DES:PAYMENTS formats. Unknown
check-shaped transactions are logged and counted, never dropped.
- Matching on check number AND amount over all candidates; wrong-amount
or collapsed-number postings fall back to a unique exact-amount match
within checks issued in the last 120 days; ambiguity means unmatched
with no write.
- Transitions always set BOTH status and clear_status (the missed
returns slipped through the divergence between them). clear_status is
bank truth: returns apply even to Cleared records, a second paid debit
on a Returned check is a redeposit back to Cleared, and a return on a
voided check is terminal voided-and-bounced. Every applied event is
appended to a history list; identical replayed events are noops.
- Optional {fromDate, toDate} replay payload (strictly validated) for
gap replays and the BAI-code enumeration runs; default stays
yesterday.
- Each run writes a boa_recon#<runDate> summary item (90-day TTL) with
per-event counts, unmatched check numbers/amounts, and unknown codes;
unmatched/unknown also console.error (Slack alerting is #71).
ACH transactions are classified but not yet acted on; bank-confirming
ACH lands with #69.
* feat(ach): bank-confirm ACH, drop CSV-date auto-clear (#69)
processPaymentCsv auto-cleared ACH the moment send_payment_on passed,
but the bank disagrees often enough to matter: settlement lags the send
date by up to 8 days, settled ACH can bounce and re-debit (Uline
$19,281.12, Heights $10,000.00 on the 5/26 overdrawn week), and voided
ACH that settled anyway returned via electronic credits invisible to a
Check-only feed. Move ACH to bank confirmation:
- processPaymentCsv: ACH rows keep their Stampli status; the send-date
auto-clear is removed. The bankConfirmed protection is unchanged, so
bank-cleared records still cannot be moved backward by the CSV.
- fetchBoaTransactions scans all payments (method filter dropped) and
processes ACH CCD lines: match by stored pmt_id first (reversals
reuse the original PMT id), then by the Stampli payment number
embedded in PMT INFO (internal spaces stripped, amount must agree),
then vendor + exact amount within send_payment_on -2..+14 days.
Settled debit -> Cleared/Cleared with dates, pmt_id persisted on the
record; credit reusing the pmt_id (or a unique same-amount return
credit against a bank-confirmed ACH) -> clear_status=Returned +
returned_date + history event. Ambiguity is unmatched, no write.
Existing DDB ACH records already Cleared by the old auto-clear are
unaffected: bank confirmation is additive and the CSV path never moves
a record backward.
Handler event contract extended (optional fromDate/toDate); cross-family
review required before merge.
* fix(fetchboa): close review findings — coverage gap, matcher corroboration, replay identity (#66)
Security-review gate (detector fan-out + verifier + GPT-4.1 cross-family)
blocked on F6 and confirmed six lower findings; all are closed here.
- F6 (HIGH): the default run now queries a trailing 3-day window
(today-3..today-1) so the Monday run covers Friday-Sunday postings the
previous-day cadence silently skipped. A per-run staleness sweep flags
never-bank-confirmed payments (ACH > 16 days, checks > 60 days) in the
summary and via console.error.
- F2: replay identity is {event, date, amount} — bankRef excluded (feeds
omit/reformat it between runs); a paid event dated on/before the
latest return in history is a replayed original, never a redeposit;
rows without a valid valueDate are routed to unmatched instead of
being applied under a substituted date; within a day, debits sort
before credits.
- F1: a check-number match with the wrong amount no longer falls
through to the amount fallback (altered-check/collapsed-posting is a
human-review case); the amount fallback requires digit-subsequence
corroboration between posting and candidate numbers; check_return
fallback requires a bank-confirmed candidate.
- F4: the ach_return amount fallback requires cleared_date within 45
days before the credit; an unattributable PMT id is an unmatched
alert, never an amount guess.
- F5: the pmt_id rung requires amount equality; mismatch is the
partial-reversal human case.
- F3: vendor prefix matching requires >= 10 normalized chars (short
names must match exactly); candidates with a conflicting stored
pmt_id are excluded; vendor+amount matches are audit-logged.
- F7: payment writes are conditioned on the snapshot's
status/clear_status and retried once against a fresh read; residual
conflicts are counted, not silently interleaved.
- F9/summary bounds: run summary pk is append-only
(boa_recon#<from>_<to>#<runAt>); unmatched list capped at 50, unknown
codes at 20 keys/16 chars, stale list at 50 (full counts kept).
- ReDoS: description bounded to 500 chars before classification;
CHECK/embedded-number regexes use bounded quantifiers.
- FBOA2-1: both BoA res.json() parses are wrapped so a malformed 200
throws a status-only error and cannot leak a body snippet.
Handler event contract extended (optional fromDate/toDate); cross-family
review required before merge.
* fix(boaRecon): add method=Check filter to matchElectronicReturn
Electronic returns only apply to checks, but the matcher was not
filtering by method, so an electronic return could incorrectly
match against a same-amount, bank-confirmed ACH record.
2026-07-21 22:07:37 -04:00
| Return credit (even if currently Cleared) | `clear_status=Returned` , `returned_date` ; `status` re-written unchanged (the CSV ladder has no Returned rung) |
2026-10-01 21:39:38 -04:00
| Second paid debit on a Returned check | Redeposit: back to Cleared with new dates |
| Return on a Stampli-voided check | Terminal voided-and-bounced (`clear_status=Returned` , cancel status preserved), counted separately |
feat(fetchboa): bank-truth reconciliation v2 — returns, redeposits, ACH confirmation (#73)
* feat(fetchboa): v2 return/redeposit-aware reconciliation (#66)
The two-code 475/255 filter missed every return on the Jan-Jul statement
(29 ARP refer-to-maker credits, 24 electronic return credits, and the
redeposit cycles), leaving 5 paid-then-returned checks stuck at Cleared
($5,256.62, vendors unpaid). Rewrite fetchBoaTransactions around a pure
reconciliation module (src/boaRecon.js) covered by node:test:
- BAI transaction-code event map (only 475 = check paid is confirmed;
remaining codes pend the enumeration replay) with a description-text
fallback classifier for ARP refer-to-maker, return-of-posted-check
(check-numbered and electronic) and ACH DES:PAYMENTS formats. Unknown
check-shaped transactions are logged and counted, never dropped.
- Matching on check number AND amount over all candidates; wrong-amount
or collapsed-number postings fall back to a unique exact-amount match
within checks issued in the last 120 days; ambiguity means unmatched
with no write.
- Transitions always set BOTH status and clear_status (the missed
returns slipped through the divergence between them). clear_status is
bank truth: returns apply even to Cleared records, a second paid debit
on a Returned check is a redeposit back to Cleared, and a return on a
voided check is terminal voided-and-bounced. Every applied event is
appended to a history list; identical replayed events are noops.
- Optional {fromDate, toDate} replay payload (strictly validated) for
gap replays and the BAI-code enumeration runs; default stays
yesterday.
- Each run writes a boa_recon#<runDate> summary item (90-day TTL) with
per-event counts, unmatched check numbers/amounts, and unknown codes;
unmatched/unknown also console.error (Slack alerting is #71).
ACH transactions are classified but not yet acted on; bank-confirming
ACH lands with #69.
* feat(ach): bank-confirm ACH, drop CSV-date auto-clear (#69)
processPaymentCsv auto-cleared ACH the moment send_payment_on passed,
but the bank disagrees often enough to matter: settlement lags the send
date by up to 8 days, settled ACH can bounce and re-debit (Uline
$19,281.12, Heights $10,000.00 on the 5/26 overdrawn week), and voided
ACH that settled anyway returned via electronic credits invisible to a
Check-only feed. Move ACH to bank confirmation:
- processPaymentCsv: ACH rows keep their Stampli status; the send-date
auto-clear is removed. The bankConfirmed protection is unchanged, so
bank-cleared records still cannot be moved backward by the CSV.
- fetchBoaTransactions scans all payments (method filter dropped) and
processes ACH CCD lines: match by stored pmt_id first (reversals
reuse the original PMT id), then by the Stampli payment number
embedded in PMT INFO (internal spaces stripped, amount must agree),
then vendor + exact amount within send_payment_on -2..+14 days.
Settled debit -> Cleared/Cleared with dates, pmt_id persisted on the
record; credit reusing the pmt_id (or a unique same-amount return
credit against a bank-confirmed ACH) -> clear_status=Returned +
returned_date + history event. Ambiguity is unmatched, no write.
Existing DDB ACH records already Cleared by the old auto-clear are
unaffected: bank confirmation is additive and the CSV path never moves
a record backward.
Handler event contract extended (optional fromDate/toDate); cross-family
review required before merge.
* fix(fetchboa): close review findings — coverage gap, matcher corroboration, replay identity (#66)
Security-review gate (detector fan-out + verifier + GPT-4.1 cross-family)
blocked on F6 and confirmed six lower findings; all are closed here.
- F6 (HIGH): the default run now queries a trailing 3-day window
(today-3..today-1) so the Monday run covers Friday-Sunday postings the
previous-day cadence silently skipped. A per-run staleness sweep flags
never-bank-confirmed payments (ACH > 16 days, checks > 60 days) in the
summary and via console.error.
- F2: replay identity is {event, date, amount} — bankRef excluded (feeds
omit/reformat it between runs); a paid event dated on/before the
latest return in history is a replayed original, never a redeposit;
rows without a valid valueDate are routed to unmatched instead of
being applied under a substituted date; within a day, debits sort
before credits.
- F1: a check-number match with the wrong amount no longer falls
through to the amount fallback (altered-check/collapsed-posting is a
human-review case); the amount fallback requires digit-subsequence
corroboration between posting and candidate numbers; check_return
fallback requires a bank-confirmed candidate.
- F4: the ach_return amount fallback requires cleared_date within 45
days before the credit; an unattributable PMT id is an unmatched
alert, never an amount guess.
- F5: the pmt_id rung requires amount equality; mismatch is the
partial-reversal human case.
- F3: vendor prefix matching requires >= 10 normalized chars (short
names must match exactly); candidates with a conflicting stored
pmt_id are excluded; vendor+amount matches are audit-logged.
- F7: payment writes are conditioned on the snapshot's
status/clear_status and retried once against a fresh read; residual
conflicts are counted, not silently interleaved.
- F9/summary bounds: run summary pk is append-only
(boa_recon#<from>_<to>#<runAt>); unmatched list capped at 50, unknown
codes at 20 keys/16 chars, stale list at 50 (full counts kept).
- ReDoS: description bounded to 500 chars before classification;
CHECK/embedded-number regexes use bounded quantifiers.
- FBOA2-1: both BoA res.json() parses are wrapped so a malformed 200
throws a status-only error and cannot leak a body snippet.
Handler event contract extended (optional fromDate/toDate); cross-family
review required before merge.
* fix(boaRecon): add method=Check filter to matchElectronicReturn
Electronic returns only apply to checks, but the matcher was not
filtering by method, so an electronic return could incorrectly
match against a same-amount, bank-confirmed ACH record.
2026-07-21 22:07:37 -04:00
Each applied event is appended to a `history` list attribute (`{event, date, bankRef, amount}` ); identical replayed events are idempotent noops.
2026-07-22 15:40:49 -04:00
**Replay.** The handler accepts an optional payload `{"fromDate": "YYYY-MM-DD", "toDate": "YYYY-MM-DD"}` (strictly validated; `toDate` defaults to `fromDate` ) for weekend/outage gap replays and BAI-code enumeration runs. With no payload it queries the trailing 7-day window (today− − `{event, date, amount}` ), and multi-day responses were verified pagination-free up to 9-day windows. Feed rows without a valid posting date (`asOfDate` , with `valueDate` as a fixture-era fallback) are never applied with a substituted date — they go to unmatched for review.
feat(fetchboa): bank-truth reconciliation v2 — returns, redeposits, ACH confirmation (#73)
* feat(fetchboa): v2 return/redeposit-aware reconciliation (#66)
The two-code 475/255 filter missed every return on the Jan-Jul statement
(29 ARP refer-to-maker credits, 24 electronic return credits, and the
redeposit cycles), leaving 5 paid-then-returned checks stuck at Cleared
($5,256.62, vendors unpaid). Rewrite fetchBoaTransactions around a pure
reconciliation module (src/boaRecon.js) covered by node:test:
- BAI transaction-code event map (only 475 = check paid is confirmed;
remaining codes pend the enumeration replay) with a description-text
fallback classifier for ARP refer-to-maker, return-of-posted-check
(check-numbered and electronic) and ACH DES:PAYMENTS formats. Unknown
check-shaped transactions are logged and counted, never dropped.
- Matching on check number AND amount over all candidates; wrong-amount
or collapsed-number postings fall back to a unique exact-amount match
within checks issued in the last 120 days; ambiguity means unmatched
with no write.
- Transitions always set BOTH status and clear_status (the missed
returns slipped through the divergence between them). clear_status is
bank truth: returns apply even to Cleared records, a second paid debit
on a Returned check is a redeposit back to Cleared, and a return on a
voided check is terminal voided-and-bounced. Every applied event is
appended to a history list; identical replayed events are noops.
- Optional {fromDate, toDate} replay payload (strictly validated) for
gap replays and the BAI-code enumeration runs; default stays
yesterday.
- Each run writes a boa_recon#<runDate> summary item (90-day TTL) with
per-event counts, unmatched check numbers/amounts, and unknown codes;
unmatched/unknown also console.error (Slack alerting is #71).
ACH transactions are classified but not yet acted on; bank-confirming
ACH lands with #69.
* feat(ach): bank-confirm ACH, drop CSV-date auto-clear (#69)
processPaymentCsv auto-cleared ACH the moment send_payment_on passed,
but the bank disagrees often enough to matter: settlement lags the send
date by up to 8 days, settled ACH can bounce and re-debit (Uline
$19,281.12, Heights $10,000.00 on the 5/26 overdrawn week), and voided
ACH that settled anyway returned via electronic credits invisible to a
Check-only feed. Move ACH to bank confirmation:
- processPaymentCsv: ACH rows keep their Stampli status; the send-date
auto-clear is removed. The bankConfirmed protection is unchanged, so
bank-cleared records still cannot be moved backward by the CSV.
- fetchBoaTransactions scans all payments (method filter dropped) and
processes ACH CCD lines: match by stored pmt_id first (reversals
reuse the original PMT id), then by the Stampli payment number
embedded in PMT INFO (internal spaces stripped, amount must agree),
then vendor + exact amount within send_payment_on -2..+14 days.
Settled debit -> Cleared/Cleared with dates, pmt_id persisted on the
record; credit reusing the pmt_id (or a unique same-amount return
credit against a bank-confirmed ACH) -> clear_status=Returned +
returned_date + history event. Ambiguity is unmatched, no write.
Existing DDB ACH records already Cleared by the old auto-clear are
unaffected: bank confirmation is additive and the CSV path never moves
a record backward.
Handler event contract extended (optional fromDate/toDate); cross-family
review required before merge.
* fix(fetchboa): close review findings — coverage gap, matcher corroboration, replay identity (#66)
Security-review gate (detector fan-out + verifier + GPT-4.1 cross-family)
blocked on F6 and confirmed six lower findings; all are closed here.
- F6 (HIGH): the default run now queries a trailing 3-day window
(today-3..today-1) so the Monday run covers Friday-Sunday postings the
previous-day cadence silently skipped. A per-run staleness sweep flags
never-bank-confirmed payments (ACH > 16 days, checks > 60 days) in the
summary and via console.error.
- F2: replay identity is {event, date, amount} — bankRef excluded (feeds
omit/reformat it between runs); a paid event dated on/before the
latest return in history is a replayed original, never a redeposit;
rows without a valid valueDate are routed to unmatched instead of
being applied under a substituted date; within a day, debits sort
before credits.
- F1: a check-number match with the wrong amount no longer falls
through to the amount fallback (altered-check/collapsed-posting is a
human-review case); the amount fallback requires digit-subsequence
corroboration between posting and candidate numbers; check_return
fallback requires a bank-confirmed candidate.
- F4: the ach_return amount fallback requires cleared_date within 45
days before the credit; an unattributable PMT id is an unmatched
alert, never an amount guess.
- F5: the pmt_id rung requires amount equality; mismatch is the
partial-reversal human case.
- F3: vendor prefix matching requires >= 10 normalized chars (short
names must match exactly); candidates with a conflicting stored
pmt_id are excluded; vendor+amount matches are audit-logged.
- F7: payment writes are conditioned on the snapshot's
status/clear_status and retried once against a fresh read; residual
conflicts are counted, not silently interleaved.
- F9/summary bounds: run summary pk is append-only
(boa_recon#<from>_<to>#<runAt>); unmatched list capped at 50, unknown
codes at 20 keys/16 chars, stale list at 50 (full counts kept).
- ReDoS: description bounded to 500 chars before classification;
CHECK/embedded-number regexes use bounded quantifiers.
- FBOA2-1: both BoA res.json() parses are wrapped so a malformed 200
throws a status-only error and cannot leak a body snippet.
Handler event contract extended (optional fromDate/toDate); cross-family
review required before merge.
* fix(boaRecon): add method=Check filter to matchElectronicReturn
Electronic returns only apply to checks, but the matcher was not
filtering by method, so an electronic return could incorrectly
match against a same-amount, bank-confirmed ACH record.
2026-07-21 22:07:37 -04:00
**Run summary.** Each run writes an append-only `boa_recon#<fromDate>_<toDate>#<runAt>` item (90-day TTL) with counts per classified event type, matched/applied/redeposit/voided-and-bounced/write-conflict totals, the unmatched check numbers and amounts (list capped at 50; full count kept), unknown BAI codes (capped at 20 distinct keys), and the staleness sweep results. Unmatched and unknown-code transactions also `console.error` (Slack alerting is tracked in #71 ). Payment writes are conditioned on the read snapshot's `status` /`clear_status` and retried once against a fresh read on conflict.
2026-07-06 17:44:01 -04:00
## Documentation
The canonical map of Sea Haven's AWS infrastructure lives in Confluence. This project's `payments-dashboard` stack is represented there as a Mermaid subgraph.
- **[AWS Architecture Map ](https://seahaven.atlassian.net/wiki/spaces/IT/pages/1540098 )** (Confluence, IT space, page 1540098)
2026-06-02 20:39:35 -04:00
## Secrets
All BoA and Slack credentials are stored in AWS Secrets Manager (per `engineering-handbook/secrets-and-config.md` ). The Slack token is a plaintext secret; the two BoA secrets are JSON grouping each API's credentials:
2026-10-01 21:39:38 -04:00
| Secret | Type | Contents |
| ----------------------------------------- | --------- | ------------------------------------------------------------------------------------------------------- |
| `payments-dashboard/slack-bot-token` | plaintext | Slack Bot OAuth token (used by `slackAppHome` ) |
| `payments-dashboard/slack-signing-secret` | plaintext | Slack signing secret for `slackAppHome` request verification |
| `payments-dashboard/boa-check-mgmt` | JSON | `appId` , `clientId` , `token` , `accountNumber` , `companyId` — Check Management API (`processPaymentCsv` ) |
| `payments-dashboard/boa-reporting` | JSON | `appId` , `clientId` , `token` , `accountNumber` , `bankId` — Reporting API (`fetchBoaTransactions` ) |
2026-06-02 20:39:35 -04:00
`boa-account-number` is duplicated into both BoA secrets. Each Lambda is granted `secretsmanager:GetSecretValue` scoped to only the secret it needs. The Expense Approval Bot uses two additional secrets (`payments-dashboard/expense-slack-token` , `payments-dashboard/expense-slack-signing-secret` ).
2026-04-13 16:24:20 -04:00
2026-07-08 16:20:05 -04:00
## Consumers / data contract
The `PaymentsDashboard` DynamoDB table (`AWS::DynamoDB::Table` , `TableName: PaymentsDashboard` , PK `pk` (S), CMK-encrypted) is **owned by this stack** , which is the sole authoritative writer.
2026-09-16 18:29:01 +00:00
**Former consumer:** `seahaven-slack-bot` (decommissioned 2026-07-23) imported this table by name. No live consumer remains. The table stays owned by this stack.
The decommissioned bot depended on:
2026-07-08 16:20:05 -04:00
- **Key schema:** PK `pk` (S) with the item format `payment#<check_number>` . It does `GetItem` by `pk` and a full-table `Scan` filtered `begins_with(pk, "payment#")` .
- **Attributes:** `check_number` , `payee` , `amount_usd` , `method` , `status` , `send_payment_on` , `clear_status` , `cleared_date` , `invoice_numbers` , `company_subsidiary` , `bank_reference` .
- **Encryption:** the shared customer-managed CMK (`/seahaven/dynamodb/cmk-arn` ). Because the consumer imports the table by name, `grantReadData` does not carry KMS access; a change of CMK requires re-granting on the consumer side or every read fails with `kms:Decrypt AccessDenied` (INFRA-95 / M-3 precedent).
2026-09-16 18:29:01 +00:00
No live stack imports this table. Keep the `pk` format and `payment#` prefix stable for Slack App Home and bank reconciliation.
2026-07-08 16:20:05 -04:00
2026-09-01 20:54:38 +00:00
**Key prefixes in this table** (all owned by this stack): `payment#<check_number>` (payment records), `metadata` (ingest metadata), `boa_txn#<ts>#<action>` (BoA submission journal, 90d TTL), `boa_recon#<from>_<to>#<runAt>` (reconciliation run summaries, 90d TTL), `boa_balance#<asOfDate>#<endpoint>` (daily balance snapshots, latest-wins, no TTL). New prefixes are invisible to `seahaven-slack-bot` 's `begins_with(pk, "payment#")` scan — no consumer coordination needed when adding one.
2026-07-22 16:02:48 -04:00
Add CloudWatch alarm coverage (payments-dashboard) (#51)
* Add CloudWatch alarm coverage for payments-dashboard (Wave 1)
Add 22 CloudWatch alarms to round out observability:
- Lambda Errors alarms for the 4 functions that lacked them
(slackAppHome, fetchBoaTransactions, expenseReceiver, expenseProcessor)
- Lambda Throttles alarms for all 6 functions
- Lambda Duration alarms for all 6 functions (~80% of timeout, Maximum)
- DynamoDB throttle/system-error alarms for the PaymentsDashboard table
(ReadThrottleEvents/WriteThrottleEvents/SystemErrors, TableName dim)
- API Gateway v2 alarms on the implicit ServerlessHttpApi
(5xx, 4xx, Latency p99; ApiId dim)
All alarms publish to the site-alerts SNS topic, ALARM-only, missing data
notBreaching, matching the existing payments-<fn>-errors convention from PR #48.
Documents the full alarm inventory under a new README Monitoring section.
* Drop DynamoDB SystemErrors alarm (never fires at TableName dimension)
AWS/DynamoDB SystemErrors does not emit at the TableName-only dimension,
so payments-dashboard-table-system-errors could never fire. Remove the
alarm resource and its README entry; keep Read/WriteThrottleEvents.
2026-06-17 14:51:59 -04:00
## Monitoring & Alarms
2026-09-16 18:29:01 +00:00
All CloudWatch alarms publish to the shared `site-alerts` SNS topic (`arn:aws:sns:us-east-1:011934824531:site-alerts` in seahaven-prod). Alarms are ALARM-only by convention (no OK/recovery action) and treat missing data as `notBreaching` . Each alarm evaluates a single 5-minute period.
Add CloudWatch alarm coverage (payments-dashboard) (#51)
* Add CloudWatch alarm coverage for payments-dashboard (Wave 1)
Add 22 CloudWatch alarms to round out observability:
- Lambda Errors alarms for the 4 functions that lacked them
(slackAppHome, fetchBoaTransactions, expenseReceiver, expenseProcessor)
- Lambda Throttles alarms for all 6 functions
- Lambda Duration alarms for all 6 functions (~80% of timeout, Maximum)
- DynamoDB throttle/system-error alarms for the PaymentsDashboard table
(ReadThrottleEvents/WriteThrottleEvents/SystemErrors, TableName dim)
- API Gateway v2 alarms on the implicit ServerlessHttpApi
(5xx, 4xx, Latency p99; ApiId dim)
All alarms publish to the site-alerts SNS topic, ALARM-only, missing data
notBreaching, matching the existing payments-<fn>-errors convention from PR #48.
Documents the full alarm inventory under a new README Monitoring section.
* Drop DynamoDB SystemErrors alarm (never fires at TableName dimension)
AWS/DynamoDB SystemErrors does not emit at the TableName-only dimension,
so payments-dashboard-table-system-errors could never fire. Remove the
alarm resource and its README entry; keep Read/WriteThrottleEvents.
2026-06-17 14:51:59 -04:00
**SQS dead-letter queues** (messages-present, Maximum > 0):
2026-10-01 21:39:38 -04:00
| Alarm | Source |
| ----------------------------------------------- | -------------------------- |
Add CloudWatch alarm coverage (payments-dashboard) (#51)
* Add CloudWatch alarm coverage for payments-dashboard (Wave 1)
Add 22 CloudWatch alarms to round out observability:
- Lambda Errors alarms for the 4 functions that lacked them
(slackAppHome, fetchBoaTransactions, expenseReceiver, expenseProcessor)
- Lambda Throttles alarms for all 6 functions
- Lambda Duration alarms for all 6 functions (~80% of timeout, Maximum)
- DynamoDB throttle/system-error alarms for the PaymentsDashboard table
(ReadThrottleEvents/WriteThrottleEvents/SystemErrors, TableName dim)
- API Gateway v2 alarms on the implicit ServerlessHttpApi
(5xx, 4xx, Latency p99; ApiId dim)
All alarms publish to the site-alerts SNS topic, ALARM-only, missing data
notBreaching, matching the existing payments-<fn>-errors convention from PR #48.
Documents the full alarm inventory under a new README Monitoring section.
* Drop DynamoDB SystemErrors alarm (never fires at TableName dimension)
AWS/DynamoDB SystemErrors does not emit at the TableName-only dimension,
so payments-dashboard-table-system-errors could never fire. Remove the
alarm resource and its README entry; keep Read/WriteThrottleEvents.
2026-06-17 14:51:59 -04:00
| `payments-processPaymentCsv-async-dlq-messages` | async-invoke OnFailure DLQ |
**Lambda** (per function — `payments-<fn>-...` ):
2026-10-01 21:39:38 -04:00
| Type | Metric / Statistic | Threshold |
| -------------------- | -------------------- | ------------------------------- |
| `-errors` (all 5) | `Errors` / Sum | > 0 |
| `-throttles` (all 5) | `Throttles` / Sum | > 0 |
2026-09-01 20:54:38 +00:00
| `-duration` (all 5) | `Duration` / Maximum | ~80% of each function's timeout |
Add CloudWatch alarm coverage (payments-dashboard) (#51)
* Add CloudWatch alarm coverage for payments-dashboard (Wave 1)
Add 22 CloudWatch alarms to round out observability:
- Lambda Errors alarms for the 4 functions that lacked them
(slackAppHome, fetchBoaTransactions, expenseReceiver, expenseProcessor)
- Lambda Throttles alarms for all 6 functions
- Lambda Duration alarms for all 6 functions (~80% of timeout, Maximum)
- DynamoDB throttle/system-error alarms for the PaymentsDashboard table
(ReadThrottleEvents/WriteThrottleEvents/SystemErrors, TableName dim)
- API Gateway v2 alarms on the implicit ServerlessHttpApi
(5xx, 4xx, Latency p99; ApiId dim)
All alarms publish to the site-alerts SNS topic, ALARM-only, missing data
notBreaching, matching the existing payments-<fn>-errors convention from PR #48.
Documents the full alarm inventory under a new README Monitoring section.
* Drop DynamoDB SystemErrors alarm (never fires at TableName dimension)
AWS/DynamoDB SystemErrors does not emit at the TableName-only dimension,
so payments-dashboard-table-system-errors could never fire. Remove the
alarm resource and its README entry; keep Read/WriteThrottleEvents.
2026-06-17 14:51:59 -04:00
2026-09-01 20:54:38 +00:00
Duration thresholds (ms): processPaymentCsv 96000, fetchBoaTransactions 48000, slackAppHome 24000, expenseProcessor 12000, expenseReceiver 4000.
Add CloudWatch alarm coverage (payments-dashboard) (#51)
* Add CloudWatch alarm coverage for payments-dashboard (Wave 1)
Add 22 CloudWatch alarms to round out observability:
- Lambda Errors alarms for the 4 functions that lacked them
(slackAppHome, fetchBoaTransactions, expenseReceiver, expenseProcessor)
- Lambda Throttles alarms for all 6 functions
- Lambda Duration alarms for all 6 functions (~80% of timeout, Maximum)
- DynamoDB throttle/system-error alarms for the PaymentsDashboard table
(ReadThrottleEvents/WriteThrottleEvents/SystemErrors, TableName dim)
- API Gateway v2 alarms on the implicit ServerlessHttpApi
(5xx, 4xx, Latency p99; ApiId dim)
All alarms publish to the site-alerts SNS topic, ALARM-only, missing data
notBreaching, matching the existing payments-<fn>-errors convention from PR #48.
Documents the full alarm inventory under a new README Monitoring section.
* Drop DynamoDB SystemErrors alarm (never fires at TableName dimension)
AWS/DynamoDB SystemErrors does not emit at the TableName-only dimension,
so payments-dashboard-table-system-errors could never fire. Remove the
alarm resource and its README entry; keep Read/WriteThrottleEvents.
2026-06-17 14:51:59 -04:00
**DynamoDB** (`PaymentsDashboard` table, `TableName` dimension, Sum > 0): `payments-dashboard-table-read-throttle` (`ReadThrottleEvents` ), `payments-dashboard-table-write-throttle` (`WriteThrottleEvents` ). The table is PAY_PER_REQUEST; these metrics emit only when a throttle occurs. `SystemErrors` is intentionally not alarmed because it does not emit at the `TableName` -only dimension.
**API Gateway** (implicit HTTP API v2 `ServerlessHttpApi` , `ApiId` dimension): `payments-dashboard-api-5xx` (`5xx` Sum > 0), `payments-dashboard-api-4xx` (`4xx` Sum > 10, client-error noise floor), `payments-dashboard-api-latency-p99` (`Latency` p99 > 3000 ms).
2026-04-13 16:24:20 -04:00
## Scripts
2026-10-01 21:39:38 -04:00
| Script | Purpose |
| ----------------------------- | ------------------------------------------------------- |
2026-04-13 16:24:20 -04:00
| `scripts/test-boa-sandbox.js` | One-off sandbox connectivity test for both CashPro APIs |
2026-10-01 21:39:38 -04:00
| `scripts/seed-from-csv.js` | Seed DynamoDB from a local CSV file |
| `scripts/seed-bank-status.js` | Seed bank clear status data into DynamoDB |
2026-04-13 16:24:20 -04:00
## Deployment
2026-09-16 18:29:01 +00:00
See [SETUP.md ](SETUP.md ). Terraform owns infrastructure in workspace `payments-dashboard-prod` . GitHub Actions Environment `prod` ships function zips via `update-function-code` . Do not run `sam deploy` .
2026-04-13 16:24:20 -04:00
2026-09-16 18:29:01 +00:00
The `boa_base_url` Terraform variable controls whether Lambdas hit production (`https://api.bofa.com` ) or sandbox (`https://api-sb.bofa.com` ). All other BoA config is read from Secrets Manager at runtime.
