# Payments Dashboard ![JavaScript](https://img.shields.io/badge/JavaScript-F7DF1E?logo=javascript&logoColor=black) ![AWS SAM](https://img.shields.io/badge/AWS-SAM-FF9900?logo=amazonaws&logoColor=white) ![Slack](https://img.shields.io/badge/Slack-integration-4A154B?logo=slack&logoColor=white) ![CI](https://github.com/Sea-Haven-Industries/payments-dashboard/actions/workflows/ci.yaml/badge.svg) AWS SAM application that ingests payment CSVs, syncs check data with Bank of America CashPro APIs, surfaces an outstanding-payments dashboard in Slack, and routes expense approvals through a Slack reaction-driven workflow. ## Architecture - **ProcessPaymentCsv** — Lambda triggered by S3 CSV upload. Parses Stampli payment exports, upserts to DynamoDB, and submits new/cancelled checks to the CashPro Check Management API. - **FetchBoaTransactions** — Scheduled Lambda. Weekdays 9am ET it calls the CashPro **previous-day** Transaction Inquiry (authoritative sweep, trailing 7 days); weekdays at 16:00/19:00/22:00 UTC (~12/3/6pm ET, fixed-UTC so it drifts an hour in winter) it calls the **current-day** inquiry for same-day visibility (EventBridge `Input: {"endpoint":"current-day"}`, today-only, staleness sweep skipped). Every run archives the exact raw response to the `seahaven-payments-boa-raw-*` bucket (`raw//_/.json`, SSE-S3, 730-day lifecycle, PutObject-only grant; Retain-protected — decommission goes through the CFN decommission runbook) and upserts per-date `boa_balance##` snapshots (latest-wins on `run_at`, no TTL) from the Summary rows. Classifies each transaction and reconciles onto DynamoDB payment records. Event payload: `{fromDate?, toDate?, endpoint?}` (endpoint allowlisted and validated; unknown fields ignored). Intraday runs are disable-able as a unit via the `IntradaySchedule` rule. See [Bank reconciliation](#bank-reconciliation-fetchboatransactions). - **SlackAppHome** — Lambda behind API Gateway (`POST /slack/events`). Verifies the Slack signing secret (HMAC-SHA256, 5-minute replay window) before processing, then renders the payments dashboard on the Slack App Home tab with outstanding aging buckets, drill-down modals, and an always-visible "Returned — Needs Action" queue (bank-returned payments awaiting a reissue/void decision, sorted oldest return first). Returned records are excluded from Outstanding totals; terminal voided-and-bounced records appear in neither (audit trail only). - **ExpenseReceiver** — Lambda behind API Gateway (`POST /slack/expense-events`). Verifies the Slack signing secret (HMAC-SHA256), handles URL verification challenges, and async-invokes ExpenseProcessor. Runs outside VPC. - **ExpenseProcessor** — Async Lambda invoked by ExpenseReceiver. Processes `:white_check_mark:` reactions to advance expense messages through a four-stage Slack channel pipeline: Submitted → Processed → Authorized → Matched. Runs outside VPC. ProcessPaymentCsv, FetchBoaTransactions, and SlackAppHome run inside a VPC with a NAT Gateway for a static outbound IP (required by BoA IP whitelisting). ExpenseReceiver, and ExpenseProcessor run outside the VPC. ## Expense Approval Bot Reaction-driven workflow that routes expense submissions through four Slack channels. A separate Slack app ("Expense Approval Bot") posts to a **Submitted** channel. Users react with :white_check_mark: to advance the message to the next stage. **Channel pipeline:** | Stage | Channel ID | Action on :white_check_mark: | |-------|-----------|------------------------------| | Submitted | `C0AQ2AWLNEN` | Thread reply on original, copy to Processed | | Processed | `C0APLSGABAB` | Delete from Processed, post to Authorized | | Authorized | `C0AQ09CDJH4` | Delete from Authorized, post to Matched | | Matched | `C0APYUM1JFP` | Terminal stage (no further routing) | **Architecture:** Two Lambdas — ExpenseReceiver (HTTP endpoint, signature verification, async invoke) and ExpenseProcessor (business logic). This is the same receiver/processor pattern used for Slack's 3-second timeout requirement. **Secrets (Secrets Manager):** | Secret | Purpose | |--------|---------| | `payments-dashboard/expense-slack-token` | Slack Bot token for the Expense Approval Bot app | | `payments-dashboard/expense-slack-signing-secret` | Slack signing secret for request verification | ## BoA CashPro API Integration Two separate CashPro APIs are used, each with its own OAuth credentials: | API | Purpose | Endpoint | |-----|---------|----------| | Check Management | Issue and cancel checks | `/cashpro/checkmanagement/v1/check-issues` | | Reporting (Transaction Inquiry) | Fetch previous-day transactions | `/cashpro/reporting/v1/transaction-inquiries/previous-day` | **Authentication flow:** 1. POST to `/authn/v1/client-authentication` with `applicationID`, `client_id`, and `client_secret` 2. Receive a Bearer `access_token` (valid 1 hour) 3. Pass the token in the `Authorization` header for subsequent API calls **Base URLs:** - Production: `https://api.bofa.com` - Sandbox: `https://api-sb.bofa.com` ## Bank reconciliation (fetchBoaTransactions) The scheduled Lambda reconciles the Previous Day feed onto `payment#` records (pure logic lives in `src/boaRecon.js`, covered by `npm test`). **Classification.** The API's Detail rows carry the statement line in `detailText` (ACH `DES:PAYMENTS ID:PMT` text, return descriptions) and the posting date in `asOfDate` (ISO); `transactionType: "Summary"` rows are balance/total lines and classify as `summary` (skipped, counted). Only 475 check-paid rows omit `detailText` — their check number rides in `customerReference` (all-zeros references normalize to empty, never check number 0). Classification runs the empirically-enumerated BAI code map first, then the description-text classifiers: | BAI code | Feed label | Event | |---|---|---| | 475 | Check Paid | `check_paid` (number in `customerReference`) | | 255 | Check Posted and Returned CR | `check_return` (number in `customerReference`) | | 252 | Debit Reversal Credit | `check_return` (second return-credit code) | | 266 | Return Item Credit | text decides (`ach_return` on PMT text, `check_return`/`electronic_return` on return text); bare rows fall back to `electronic_return`; unreadable text stays `unknown` (loud, no write) | | 455 | Preauthorized ACH Debit | `ach_debit` via `DES:PAYMENTS` text; DES-less = third-party autopay → ignored | | 170/201/470/481 | totals, transfers, loan payments | ignored | A hard code-map event wins over description text; 266/455 carry *fallback* events consulted only when the text yields nothing. Unmapped codes on check-shaped transactions are logged (`console.error`) and counted in the run summary — never silently dropped. **Matching.** Check events match on check number AND amount, evaluating all candidates (bank postings can drop/collapse digits on long check numbers). A number match with the WRONG amount never auto-resolves — it is the altered-check/collapsed-posting signal and goes to unmatched for human review. An unknown number falls back to an exact-amount match within checks issued in the last 120 days, and only when the posting's digits are a subsequence of the candidate's check number (or vice versa); return credits additionally require a bank-confirmed candidate. Zero or multiple fallback candidates means unmatched, recorded in the run summary with no write. Electronic returns (no check number) match by exact amount among bank-confirmed payments. **ACH.** ACH is bank-confirmed too (#69): `processPaymentCsv` no longer auto-clears ACH on the send date (rows keep their Stampli status until the bank settles). ACH CCD lines match by the stored `pmt_id` first (reversals reuse the original `PMT `; a `pmt_id` match with the wrong amount is the partial-reversal human case and goes to unmatched), then by the Stampli payment number embedded at the end of `PMT INFO` (internal spaces stripped, amount must agree), then by vendor + exact amount within `send_payment_on` −2..+14 days (candidates whose stored `pmt_id` differs are excluded; vendor prefix matching requires ≥ 10 normalized chars). A settled debit clears the record and persists `pmt_id`. A return credit whose `PMT ` attributes to no stored `pmt_id` is an unmatched alert ("unknown PMT id"); credits without a `PMT ` may fall back to a unique same-amount match against a bank-confirmed ACH whose `cleared_date` is within the prior 45 days. **Staleness sweep.** Every run also flags never-bank-confirmed payments: ACH with no `clear_status` sent more than 16 days ago (listed in the summary, capped at 50, plus a total count) and checks issued more than 60 days ago with no `clear_status` (count only). Both alert via `console.error`. **State transitions.** Every write sets BOTH `status` and `clear_status` (`clear_status` is bank truth; "Cleared without a subsequent return is permanent" keys off it): | Bank event | Result | |---|---| | Paid debit | `status=Cleared`, `clear_status=Cleared`, `paid_date`, `cleared_date`, `bank_reference` | | Return credit (even if currently Cleared) | `clear_status=Returned`, `returned_date`; `status` re-written unchanged (the CSV ladder has no Returned rung) | | Second paid debit on a Returned check | Redeposit: back to Cleared with new dates | | Return on a Stampli-voided check | Terminal voided-and-bounced (`clear_status=Returned`, cancel status preserved), counted separately | Each applied event is appended to a `history` list attribute (`{event, date, bankRef, amount}`); identical replayed events are idempotent noops. **Replay.** The handler accepts an optional payload `{"fromDate": "YYYY-MM-DD", "toDate": "YYYY-MM-DD"}` (strictly validated; `toDate` defaults to `fromDate`) for weekend/outage gap replays and BAI-code enumeration runs. With no payload it queries the trailing 7-day window (today−7 .. today−1) — self-healing across missed runs and holiday gaps; overlapping days are idempotent (event identity `{event, date, amount}`), and multi-day responses were verified pagination-free up to 9-day windows. Feed rows without a valid posting date (`asOfDate`, with `valueDate` as a fixture-era fallback) are never applied with a substituted date — they go to unmatched for review. **Run summary.** Each run writes an append-only `boa_recon#_#` item (90-day TTL) with counts per classified event type, matched/applied/redeposit/voided-and-bounced/write-conflict totals, the unmatched check numbers and amounts (list capped at 50; full count kept), unknown BAI codes (capped at 20 distinct keys), and the staleness sweep results. Unmatched and unknown-code transactions also `console.error` (Slack alerting is tracked in #71). Payment writes are conditioned on the read snapshot's `status`/`clear_status` and retried once against a fresh read on conflict. ## Documentation The canonical map of Sea Haven's AWS infrastructure lives in Confluence. This project's `payments-dashboard` stack is represented there as a Mermaid subgraph. - **[AWS Architecture Map](https://seahaven.atlassian.net/wiki/spaces/IT/pages/1540098)** (Confluence, IT space, page 1540098) ## Secrets All BoA and Slack credentials are stored in AWS Secrets Manager (per `engineering-handbook/secrets-and-config.md`). The Slack token is a plaintext secret; the two BoA secrets are JSON grouping each API's credentials: | Secret | Type | Contents | |--------|------|----------| | `payments-dashboard/slack-bot-token` | plaintext | Slack Bot OAuth token (used by `slackAppHome`) | | `payments-dashboard/slack-signing-secret` | plaintext | Slack signing secret for `slackAppHome` request verification | | `payments-dashboard/boa-check-mgmt` | JSON | `appId`, `clientId`, `token`, `accountNumber`, `companyId` — Check Management API (`processPaymentCsv`) | | `payments-dashboard/boa-reporting` | JSON | `appId`, `clientId`, `token`, `accountNumber`, `bankId` — Reporting API (`fetchBoaTransactions`) | `boa-account-number` is duplicated into both BoA secrets. Each Lambda is granted `secretsmanager:GetSecretValue` scoped to only the secret it needs. The Expense Approval Bot uses two additional secrets (`payments-dashboard/expense-slack-token`, `payments-dashboard/expense-slack-signing-secret`). ## Consumers / data contract The `PaymentsDashboard` DynamoDB table (`AWS::DynamoDB::Table`, `TableName: PaymentsDashboard`, PK `pk` (S), CMK-encrypted) is **owned by this stack**, which is the sole authoritative writer. **Consumer (read-only):** `seahaven-slack-bot` imports this table via `Table.fromTableName(...)` and reads it read-only (`grantReadData` plus an explicit `kms:Decrypt` grant on the shared CMK) from its `wo-po-lookup` Lambda, which backs the Bedrock agent's payment-lookup action group. The bot depends on: - **Key schema:** PK `pk` (S) with the item format `payment#`. It does `GetItem` by `pk` and a full-table `Scan` filtered `begins_with(pk, "payment#")`. - **Attributes:** `check_number`, `payee`, `amount_usd`, `method`, `status`, `send_payment_on`, `clear_status`, `cleared_date`, `invoice_numbers`, `company_subsidiary`, `bank_reference`. - **Encryption:** the shared customer-managed CMK (`/seahaven/dynamodb/cmk-arn`). Because the consumer imports the table by name, `grantReadData` does not carry KMS access; a change of CMK requires re-granting on the consumer side or every read fails with `kms:Decrypt AccessDenied` (INFRA-95 / M-3 precedent). The table is imported by name, so there is no compile-time link between the stacks: any change to the table name, `pk` format, these attribute names, the encryption key, or the table's lifecycle policy will silently break the Bedrock agent at runtime. Coordinate such changes with `seahaven-slack-bot` before shipping (INFRA-138). **Key prefixes in this table** (all owned by this stack): `payment#` (payment records), `metadata` (ingest metadata), `boa_txn##` (BoA submission journal, 90d TTL), `boa_recon#_#` (reconciliation run summaries, 90d TTL), `boa_balance##` (daily balance snapshots, latest-wins, no TTL). New prefixes are invisible to `seahaven-slack-bot`'s `begins_with(pk, "payment#")` scan — no consumer coordination needed when adding one. ## Monitoring & Alarms All CloudWatch alarms publish to the shared `site-alerts` SNS topic (`arn:aws:sns:us-east-1:328440206208:site-alerts`). Alarms are ALARM-only by convention (no OK/recovery action) and treat missing data as `notBreaching`. Each alarm evaluates a single 5-minute period. **SQS dead-letter queues** (messages-present, Maximum > 0): | Alarm | Source | |-------|--------| | `payments-processPaymentCsv-async-dlq-messages` | async-invoke OnFailure DLQ | **Lambda** (per function — `payments--...`): | Type | Metric / Statistic | Threshold | |----------------------|--------------------|-----------| | `-errors` (all 5) | `Errors` / Sum | > 0 | | `-throttles` (all 5) | `Throttles` / Sum | > 0 | | `-duration` (all 5) | `Duration` / Maximum | ~80% of each function's timeout | Duration thresholds (ms): processPaymentCsv 96000, fetchBoaTransactions 48000, slackAppHome 24000, expenseProcessor 12000, expenseReceiver 4000. **DynamoDB** (`PaymentsDashboard` table, `TableName` dimension, Sum > 0): `payments-dashboard-table-read-throttle` (`ReadThrottleEvents`), `payments-dashboard-table-write-throttle` (`WriteThrottleEvents`). The table is PAY_PER_REQUEST; these metrics emit only when a throttle occurs. `SystemErrors` is intentionally not alarmed because it does not emit at the `TableName`-only dimension. **API Gateway** (implicit HTTP API v2 `ServerlessHttpApi`, `ApiId` dimension): `payments-dashboard-api-5xx` (`5xx` Sum > 0), `payments-dashboard-api-4xx` (`4xx` Sum > 10, client-error noise floor), `payments-dashboard-api-latency-p99` (`Latency` p99 > 3000 ms). ## Scripts | Script | Purpose | |--------|---------| | `scripts/test-boa-sandbox.js` | One-off sandbox connectivity test for both CashPro APIs | | `scripts/seed-from-csv.js` | Seed DynamoDB from a local CSV file | | `scripts/seed-bank-status.js` | Seed bank clear status data into DynamoDB | ## Deployment ```bash sam build sam deploy --guided ``` The `BOA_BASE_URL` environment variable in `template.yaml` controls whether Lambdas hit production (`https://api.bofa.com`) or sandbox (`https://api-sb.bofa.com`). All other BoA config is read from Secrets Manager at runtime.