payments-dashboard/README.md

Payments Dashboard

HCP Terraform application that ingests payment CSVs, syncs check data with Bank of America CashPro APIs, surfaces an outstanding-payments dashboard in Slack, and routes expense approvals through a Slack reaction-driven workflow. Prod workspace: payments-dashboard-prod (trigger prefix terraform/**). Zip CD is GitHub Actions Environment prod.

Architecture

• ProcessPaymentCsv — Lambda triggered by S3 CSV upload. Parses Stampli payment exports, upserts to DynamoDB, and submits new/cancelled checks to the CashPro Check Management API.

• FetchBoaTransactions — Scheduled Lambda. Weekdays 9am ET it calls the CashPro previous-day Transaction Inquiry (authoritative sweep, trailing 7 days); weekdays at 16:00/19:00/22:00 UTC (~12/3/6pm ET, fixed-UTC so it drifts an hour in winter) it calls the current-day inquiry for same-day visibility (EventBridge Input: {"endpoint":"current-day"}, today-only, staleness sweep skipped). Every run archives the exact raw response to the seahaven-payments-boa-raw-* bucket (raw/<endpoint>/<fromDate>_<toDate>/<runAt>.json, SSE-S3, 730-day lifecycle, PutObject-only grant; Retain-protected in Terraform) and upserts per-date boa_balance#<asOfDate>#<endpoint> snapshots (latest-wins on run_at, no TTL) from the Summary rows. Classifies each transaction and reconciles onto DynamoDB payment records. Event payload: {fromDate?, toDate?, endpoint?} (endpoint allowlisted and validated; unknown fields ignored). Intraday runs are disable-able as a unit via the IntradaySchedule rule. See Bank reconciliation.

• SlackAppHome — Lambda behind API Gateway (POST /slack/events). Verifies the Slack signing secret (HMAC-SHA256, 5-minute replay window) before processing, then renders the payments dashboard on the Slack App Home tab with outstanding aging buckets, drill-down modals, and an always-visible "Returned — Needs Action" queue (bank-returned payments awaiting a reissue/void decision, sorted oldest return first). Returned records are excluded from Outstanding totals; terminal voided-and-bounced records appear in neither (audit trail only).

• ExpenseReceiver — Lambda behind API Gateway (POST /slack/expense-events). Verifies the Slack signing secret (HMAC-SHA256), handles URL verification challenges, and async-invokes ExpenseProcessor. Runs outside VPC.

• ExpenseProcessor — Async Lambda invoked by ExpenseReceiver. Processes :white_check_mark: reactions to advance expense messages through a four-stage Slack channel pipeline: Submitted → Processed → Authorized → Matched. Runs outside VPC.

ProcessPaymentCsv, FetchBoaTransactions, and SlackAppHome run inside a VPC with a NAT Gateway for a static outbound IP (required by BoA IP whitelisting). ExpenseReceiver and ExpenseProcessor run outside the VPC.

Expense Approval Bot

Reaction-driven workflow that routes expense submissions through four Slack channels. A separate Slack app ("Expense Approval Bot") posts to a Submitted channel. Users react with ✅ to advance the message to the next stage.

Channel pipeline:

Stage	Channel ID	Action on ✅
Submitted	C0AQ2AWLNEN	Thread reply on original, copy to Processed
Processed	C0APLSGABAB	Delete from Processed, post to Authorized
Authorized	C0AQ09CDJH4	Delete from Authorized, post to Matched
Matched	C0APYUM1JFP	Terminal stage (no further routing)

Architecture: Two Lambdas — ExpenseReceiver (HTTP endpoint, signature verification, async invoke) and ExpenseProcessor (business logic). This is the same receiver/processor pattern used for Slack's 3-second timeout requirement.

Secrets (Secrets Manager):

Secret	Purpose
payments-dashboard/expense-slack-token	Slack Bot token for the Expense Approval Bot app
payments-dashboard/expense-slack-signing-secret	Slack signing secret for request verification

BoA CashPro API Integration

Two separate CashPro APIs are used, each with its own OAuth credentials:

API	Purpose	Endpoint
Check Management	Issue and cancel checks	/cashpro/checkmanagement/v1/check-issues
Reporting (Transaction Inquiry)	Fetch previous-day transactions	/cashpro/reporting/v1/transaction-inquiries/previous-day

Authentication flow:

1. POST to /authn/v1/client-authentication with applicationID, client_id, and client_secret
2. Receive a Bearer access_token (valid 1 hour)
3. Pass the token in the Authorization header for subsequent API calls

Base URLs:

• Production: https://api.bofa.com
• Sandbox: https://api-sb.bofa.com

Bank reconciliation (fetchBoaTransactions)

The scheduled Lambda reconciles the Previous Day feed onto payment# records (pure logic lives in src/boaRecon.js, covered by npm test).

Classification. The API's Detail rows carry the statement line in detailText (ACH DES:PAYMENTS ID:PMT text, return descriptions) and the posting date in asOfDate (ISO); transactionType: "Summary" rows are balance/total lines and classify as summary (skipped, counted). Only 475 check-paid rows omit detailText — their check number rides in customerReference (all-zeros references normalize to empty, never check number 0). Classification runs the empirically-enumerated BAI code map first, then the description-text classifiers:

BAI code	Feed label	Event
475	Check Paid	check_paid (number in customerReference)
255	Check Posted and Returned CR	check_return (number in customerReference)
252	Debit Reversal Credit	check_return (second return-credit code)
266	Return Item Credit	text decides (ach_return on PMT text, check_return/electronic_return on return text); bare rows fall back to electronic_return; unreadable text stays unknown (loud, no write)
455	Preauthorized ACH Debit	ach_debit via DES:PAYMENTS text; DES-less = third-party autopay → ignored
170/201/470/481	totals, transfers, loan payments	ignored

A hard code-map event wins over description text; 266/455 carry fallback events consulted only when the text yields nothing. Unmapped codes on check-shaped transactions are logged (console.error) and counted in the run summary — never silently dropped.

Matching. Check events match on check number AND amount, evaluating all candidates (bank postings can drop/collapse digits on long check numbers). A number match with the WRONG amount never auto-resolves — it is the altered-check/collapsed-posting signal and goes to unmatched for human review. An unknown number falls back to an exact-amount match within checks issued in the last 120 days, and only when the posting's digits are a subsequence of the candidate's check number (or vice versa); return credits additionally require a bank-confirmed candidate. Zero or multiple fallback candidates means unmatched, recorded in the run summary with no write. Electronic returns (no check number) match by exact amount among bank-confirmed payments.

ACH. ACH is bank-confirmed too (#69): processPaymentCsv no longer auto-clears ACH on the send date (rows keep their Stampli status until the bank settles). ACH CCD lines match by the stored pmt_id first (reversals reuse the original PMT <id>; a pmt_id match with the wrong amount is the partial-reversal human case and goes to unmatched), then by the Stampli payment number embedded at the end of PMT INFO (internal spaces stripped, amount must agree), then by vendor + exact amount within send_payment_on −2..+14 days (candidates whose stored pmt_id differs are excluded; vendor prefix matching requires ≥ 10 normalized chars). A settled debit clears the record and persists pmt_id. A return credit whose PMT <id> attributes to no stored pmt_id is an unmatched alert ("unknown PMT id"); credits without a PMT <id> may fall back to a unique same-amount match against a bank-confirmed ACH whose cleared_date is within the prior 45 days.

Staleness sweep. Every run also flags never-bank-confirmed payments: ACH with no clear_status sent more than 16 days ago (listed in the summary, capped at 50, plus a total count) and checks issued more than 60 days ago with no clear_status (count only). Both alert via console.error.

State transitions. Every write sets BOTH status and clear_status (clear_status is bank truth; "Cleared without a subsequent return is permanent" keys off it):

Bank event	Result
Paid debit	status=Cleared, clear_status=Cleared, paid_date, cleared_date, bank_reference
Return credit (even if currently Cleared)	clear_status=Returned, returned_date; status re-written unchanged (the CSV ladder has no Returned rung)
Second paid debit on a Returned check	Redeposit: back to Cleared with new dates
Return on a Stampli-voided check	Terminal voided-and-bounced (clear_status=Returned, cancel status preserved), counted separately

Each applied event is appended to a history list attribute ({event, date, bankRef, amount}); identical replayed events are idempotent noops.

Replay. The handler accepts an optional payload {"fromDate": "YYYY-MM-DD", "toDate": "YYYY-MM-DD"} (strictly validated; toDate defaults to fromDate) for weekend/outage gap replays and BAI-code enumeration runs. With no payload it queries the trailing 7-day window (today−7 .. today−1) — self-healing across missed runs and holiday gaps; overlapping days are idempotent (event identity {event, date, amount}), and multi-day responses were verified pagination-free up to 9-day windows. Feed rows without a valid posting date (asOfDate, with valueDate as a fixture-era fallback) are never applied with a substituted date — they go to unmatched for review.

Run summary. Each run writes an append-only boa_recon#<fromDate>_<toDate>#<runAt> item (90-day TTL) with counts per classified event type, matched/applied/redeposit/voided-and-bounced/write-conflict totals, the unmatched check numbers and amounts (list capped at 50; full count kept), unknown BAI codes (capped at 20 distinct keys), and the staleness sweep results. Unmatched and unknown-code transactions also console.error (Slack alerting is tracked in #71). Payment writes are conditioned on the read snapshot's status/clear_status and retried once against a fresh read on conflict.

Documentation

The canonical map of Sea Haven's AWS infrastructure lives in Confluence. This project's payments-dashboard stack is represented there as a Mermaid subgraph.

• AWS Architecture Map (Confluence, IT space, page 1540098)

Secrets

All BoA and Slack credentials are stored in AWS Secrets Manager (per engineering-handbook/secrets-and-config.md). The Slack token is a plaintext secret; the two BoA secrets are JSON grouping each API's credentials:

Secret	Type	Contents
payments-dashboard/slack-bot-token	plaintext	Slack Bot OAuth token (used by slackAppHome)
payments-dashboard/slack-signing-secret	plaintext	Slack signing secret for slackAppHome request verification
payments-dashboard/boa-check-mgmt	JSON	appId, clientId, token, accountNumber, companyId — Check Management API (processPaymentCsv)
payments-dashboard/boa-reporting	JSON	appId, clientId, token, accountNumber, bankId — Reporting API (fetchBoaTransactions)

boa-account-number is duplicated into both BoA secrets. Each Lambda is granted secretsmanager:GetSecretValue scoped to only the secret it needs. The Expense Approval Bot uses two additional secrets (payments-dashboard/expense-slack-token, payments-dashboard/expense-slack-signing-secret).

Consumers / data contract

The PaymentsDashboard DynamoDB table (AWS::DynamoDB::Table, TableName: PaymentsDashboard, PK pk (S), CMK-encrypted) is owned by this stack, which is the sole authoritative writer.

Former consumer: seahaven-slack-bot (decommissioned 2026-07-23) imported this table by name. No live consumer remains. The table stays owned by this stack.

The decommissioned bot depended on:

• Key schema: PK pk (S) with the item format payment#<check_number>. It does GetItem by pk and a full-table Scan filtered begins_with(pk, "payment#").
• Attributes: check_number, payee, amount_usd, method, status, send_payment_on, clear_status, cleared_date, invoice_numbers, company_subsidiary, bank_reference.
• Encryption: the shared customer-managed CMK (/seahaven/dynamodb/cmk-arn). Because the consumer imports the table by name, grantReadData does not carry KMS access; a change of CMK requires re-granting on the consumer side or every read fails with kms:Decrypt AccessDenied (INFRA-95 / M-3 precedent).

No live stack imports this table. Keep the pk format and payment# prefix stable for Slack App Home and bank reconciliation.

Key prefixes in this table (all owned by this stack): payment#<check_number> (payment records), metadata (ingest metadata), boa_txn#<ts>#<action> (BoA submission journal, 90d TTL), boa_recon#<from>_<to>#<runAt> (reconciliation run summaries, 90d TTL), boa_balance#<asOfDate>#<endpoint> (daily balance snapshots, latest-wins, no TTL). New prefixes are invisible to seahaven-slack-bot's begins_with(pk, "payment#") scan — no consumer coordination needed when adding one.

Monitoring & Alarms

All CloudWatch alarms publish to the shared site-alerts SNS topic (arn:aws:sns:us-east-1:011934824531:site-alerts in seahaven-prod). Alarms are ALARM-only by convention (no OK/recovery action) and treat missing data as notBreaching. Each alarm evaluates a single 5-minute period.

SQS dead-letter queues (messages-present, Maximum > 0):

Alarm	Source
payments-processPaymentCsv-async-dlq-messages	async-invoke OnFailure DLQ

Lambda (per function — payments-<fn>-...):

Type	Metric / Statistic	Threshold
-errors (all 5)	Errors / Sum	> 0
-throttles (all 5)	Throttles / Sum	> 0
-duration (all 5)	Duration / Maximum	~80% of each function's timeout

Duration thresholds (ms): processPaymentCsv 96000, fetchBoaTransactions 48000, slackAppHome 24000, expenseProcessor 12000, expenseReceiver 4000.

DynamoDB (PaymentsDashboard table, TableName dimension, Sum > 0): payments-dashboard-table-read-throttle (ReadThrottleEvents), payments-dashboard-table-write-throttle (WriteThrottleEvents). The table is PAY_PER_REQUEST; these metrics emit only when a throttle occurs. SystemErrors is intentionally not alarmed because it does not emit at the TableName-only dimension.

API Gateway (implicit HTTP API v2 ServerlessHttpApi, ApiId dimension): payments-dashboard-api-5xx (5xx Sum > 0), payments-dashboard-api-4xx (4xx Sum > 10, client-error noise floor), payments-dashboard-api-latency-p99 (Latency p99 > 3000 ms).

Scripts

Script	Purpose
scripts/test-boa-sandbox.js	One-off sandbox connectivity test for both CashPro APIs
scripts/seed-from-csv.js	Seed DynamoDB from a local CSV file
scripts/seed-bank-status.js	Seed bank clear status data into DynamoDB

Deployment

See SETUP.md. Terraform owns infrastructure in workspace payments-dashboard-prod. GitHub Actions Environment prod ships function zips via update-function-code. Do not run sam deploy.

The boa_base_url Terraform variable controls whether Lambdas hit production (https://api.bofa.com) or sandbox (https://api-sb.bofa.com). All other BoA config is read from Secrets Manager at runtime.