From f7adb6e8b882a8f10d5db05c6880d053b657ed25 Mon Sep 17 00:00:00 2001 From: Adam Moussa <166072409+amoussa1229@users.noreply.github.com> Date: Tue, 21 Jul 2026 22:07:37 -0400 Subject: [PATCH] =?UTF-8?q?feat(fetchboa):=20bank-truth=20reconciliation?= =?UTF-8?q?=20v2=20=E2=80=94=20returns,=20redeposits,=20ACH=20confirmation?= =?UTF-8?q?=20(#73)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(fetchboa): v2 return/redeposit-aware reconciliation (#66) The two-code 475/255 filter missed every return on the Jan-Jul statement (29 ARP refer-to-maker credits, 24 electronic return credits, and the redeposit cycles), leaving 5 paid-then-returned checks stuck at Cleared ($5,256.62, vendors unpaid). Rewrite fetchBoaTransactions around a pure reconciliation module (src/boaRecon.js) covered by node:test: - BAI transaction-code event map (only 475 = check paid is confirmed; remaining codes pend the enumeration replay) with a description-text fallback classifier for ARP refer-to-maker, return-of-posted-check (check-numbered and electronic) and ACH DES:PAYMENTS formats. Unknown check-shaped transactions are logged and counted, never dropped. - Matching on check number AND amount over all candidates; wrong-amount or collapsed-number postings fall back to a unique exact-amount match within checks issued in the last 120 days; ambiguity means unmatched with no write. - Transitions always set BOTH status and clear_status (the missed returns slipped through the divergence between them). clear_status is bank truth: returns apply even to Cleared records, a second paid debit on a Returned check is a redeposit back to Cleared, and a return on a voided check is terminal voided-and-bounced. Every applied event is appended to a history list; identical replayed events are noops. - Optional {fromDate, toDate} replay payload (strictly validated) for gap replays and the BAI-code enumeration runs; default stays yesterday. - Each run writes a boa_recon# summary item (90-day TTL) with per-event counts, unmatched check numbers/amounts, and unknown codes; unmatched/unknown also console.error (Slack alerting is #71). ACH transactions are classified but not yet acted on; bank-confirming ACH lands with #69. * feat(ach): bank-confirm ACH, drop CSV-date auto-clear (#69) processPaymentCsv auto-cleared ACH the moment send_payment_on passed, but the bank disagrees often enough to matter: settlement lags the send date by up to 8 days, settled ACH can bounce and re-debit (Uline $19,281.12, Heights $10,000.00 on the 5/26 overdrawn week), and voided ACH that settled anyway returned via electronic credits invisible to a Check-only feed. Move ACH to bank confirmation: - processPaymentCsv: ACH rows keep their Stampli status; the send-date auto-clear is removed. The bankConfirmed protection is unchanged, so bank-cleared records still cannot be moved backward by the CSV. - fetchBoaTransactions scans all payments (method filter dropped) and processes ACH CCD lines: match by stored pmt_id first (reversals reuse the original PMT id), then by the Stampli payment number embedded in PMT INFO (internal spaces stripped, amount must agree), then vendor + exact amount within send_payment_on -2..+14 days. Settled debit -> Cleared/Cleared with dates, pmt_id persisted on the record; credit reusing the pmt_id (or a unique same-amount return credit against a bank-confirmed ACH) -> clear_status=Returned + returned_date + history event. Ambiguity is unmatched, no write. Existing DDB ACH records already Cleared by the old auto-clear are unaffected: bank confirmation is additive and the CSV path never moves a record backward. Handler event contract extended (optional fromDate/toDate); cross-family review required before merge. * fix(fetchboa): close review findings — coverage gap, matcher corroboration, replay identity (#66) Security-review gate (detector fan-out + verifier + GPT-4.1 cross-family) blocked on F6 and confirmed six lower findings; all are closed here. - F6 (HIGH): the default run now queries a trailing 3-day window (today-3..today-1) so the Monday run covers Friday-Sunday postings the previous-day cadence silently skipped. A per-run staleness sweep flags never-bank-confirmed payments (ACH > 16 days, checks > 60 days) in the summary and via console.error. - F2: replay identity is {event, date, amount} — bankRef excluded (feeds omit/reformat it between runs); a paid event dated on/before the latest return in history is a replayed original, never a redeposit; rows without a valid valueDate are routed to unmatched instead of being applied under a substituted date; within a day, debits sort before credits. - F1: a check-number match with the wrong amount no longer falls through to the amount fallback (altered-check/collapsed-posting is a human-review case); the amount fallback requires digit-subsequence corroboration between posting and candidate numbers; check_return fallback requires a bank-confirmed candidate. - F4: the ach_return amount fallback requires cleared_date within 45 days before the credit; an unattributable PMT id is an unmatched alert, never an amount guess. - F5: the pmt_id rung requires amount equality; mismatch is the partial-reversal human case. - F3: vendor prefix matching requires >= 10 normalized chars (short names must match exactly); candidates with a conflicting stored pmt_id are excluded; vendor+amount matches are audit-logged. - F7: payment writes are conditioned on the snapshot's status/clear_status and retried once against a fresh read; residual conflicts are counted, not silently interleaved. - F9/summary bounds: run summary pk is append-only (boa_recon#_#); unmatched list capped at 50, unknown codes at 20 keys/16 chars, stale list at 50 (full counts kept). - ReDoS: description bounded to 500 chars before classification; CHECK/embedded-number regexes use bounded quantifiers. - FBOA2-1: both BoA res.json() parses are wrapped so a malformed 200 throws a status-only error and cannot leak a body snippet. Handler event contract extended (optional fromDate/toDate); cross-family review required before merge. * fix(boaRecon): add method=Check filter to matchElectronicReturn Electronic returns only apply to checks, but the matcher was not filtering by method, so an electronic return could incorrectly match against a same-amount, bank-confirmed ACH record. --- README.md | 29 +- package.json | 3 + src/boaRecon.js | 551 +++++++++++++++++++++++++ src/fetchBoaTransactions.js | 310 +++++++++++--- src/processPaymentCsv.js | 12 +- tests/boaRecon.test.js | 784 ++++++++++++++++++++++++++++++++++++ 6 files changed, 1626 insertions(+), 63 deletions(-) create mode 100644 src/boaRecon.js create mode 100644 tests/boaRecon.test.js diff --git a/README.md b/README.md index 3603522..7e6ffef 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ AWS SAM application that ingests payment CSVs, syncs check data with Bank of Ame - **ProcessPayrollEmail** — Lambda triggered by S3 (inbound email) and SQS (batch timer). SES receives Gusto payroll emails at `payroll@int.seahaven.com`, stores them to S3, and this Lambda parses the email body, extracts financial data, and posts a combined Slack notification (employee payroll + contractor payments) after a 10-minute batching window. Runs outside VPC. - **ProcessPaymentCsv** — Lambda triggered by S3 CSV upload. Parses Stampli payment exports, upserts to DynamoDB, and submits new/cancelled checks to the CashPro Check Management API. -- **FetchBoaTransactions** — Scheduled Lambda (weekdays 9am ET). Calls the CashPro Previous Day Transaction Inquiry API and matches cleared/returned checks back to DynamoDB records. +- **FetchBoaTransactions** — Scheduled Lambda (weekdays 9am ET). Calls the CashPro Previous Day Transaction Inquiry API, classifies each transaction (paid checks, ARP refer-to-maker and return-of-posted-check credits, electronic returns), and reconciles them onto DynamoDB payment records. See [Bank reconciliation](#bank-reconciliation-fetchboatransactions). - **SlackAppHome** — Lambda behind API Gateway (`POST /slack/events`). Verifies the Slack signing secret (HMAC-SHA256, 5-minute replay window) before processing, then renders the payments dashboard on the Slack App Home tab with outstanding aging buckets and drill-down modals. - **ExpenseReceiver** — Lambda behind API Gateway (`POST /slack/expense-events`). Verifies the Slack signing secret (HMAC-SHA256), handles URL verification challenges, and async-invokes ExpenseProcessor. Runs outside VPC. @@ -67,6 +67,33 @@ Two separate CashPro APIs are used, each with its own OAuth credentials: - Production: `https://api.bofa.com` - Sandbox: `https://api-sb.bofa.com` +## Bank reconciliation (fetchBoaTransactions) + +The scheduled Lambda reconciles the Previous Day feed onto `payment#` records (pure logic lives in `src/boaRecon.js`, covered by `npm test`). + +**Classification.** Transactions are classified via a BAI transaction-code map (`BAI_CODE_EVENTS`; only 475 = check paid debit is empirically confirmed so far — the remaining codes are pending an enumeration replay over known event dates) with a fallback classifier over the statement description formats (ARP refer-to-maker, return-of-posted-check check-numbered and electronic variants, ACH CCD `DES:PAYMENTS ID:PMT` lines). Unmapped codes on check-shaped transactions are logged (`console.error`) and counted in the run summary — never silently dropped. + +**Matching.** Check events match on check number AND amount, evaluating all candidates (bank postings can drop/collapse digits on long check numbers). A number match with the WRONG amount never auto-resolves — it is the altered-check/collapsed-posting signal and goes to unmatched for human review. An unknown number falls back to an exact-amount match within checks issued in the last 120 days, and only when the posting's digits are a subsequence of the candidate's check number (or vice versa); return credits additionally require a bank-confirmed candidate. Zero or multiple fallback candidates means unmatched, recorded in the run summary with no write. Electronic returns (no check number) match by exact amount among bank-confirmed payments. + +**ACH.** ACH is bank-confirmed too (#69): `processPaymentCsv` no longer auto-clears ACH on the send date (rows keep their Stampli status until the bank settles). ACH CCD lines match by the stored `pmt_id` first (reversals reuse the original `PMT `; a `pmt_id` match with the wrong amount is the partial-reversal human case and goes to unmatched), then by the Stampli payment number embedded at the end of `PMT INFO` (internal spaces stripped, amount must agree), then by vendor + exact amount within `send_payment_on` −2..+14 days (candidates whose stored `pmt_id` differs are excluded; vendor prefix matching requires ≥ 10 normalized chars). A settled debit clears the record and persists `pmt_id`. A return credit whose `PMT ` attributes to no stored `pmt_id` is an unmatched alert ("unknown PMT id"); credits without a `PMT ` may fall back to a unique same-amount match against a bank-confirmed ACH whose `cleared_date` is within the prior 45 days. + +**Staleness sweep.** Every run also flags never-bank-confirmed payments: ACH with no `clear_status` sent more than 16 days ago (listed in the summary, capped at 50, plus a total count) and checks issued more than 60 days ago with no `clear_status` (count only). Both alert via `console.error`. + +**State transitions.** Every write sets BOTH `status` and `clear_status` (`clear_status` is bank truth; "Cleared without a subsequent return is permanent" keys off it): + +| Bank event | Result | +|---|---| +| Paid debit | `status=Cleared`, `clear_status=Cleared`, `paid_date`, `cleared_date`, `bank_reference` | +| Return credit (even if currently Cleared) | `clear_status=Returned`, `returned_date`; `status` re-written unchanged (the CSV ladder has no Returned rung) | +| Second paid debit on a Returned check | Redeposit: back to Cleared with new dates | +| Return on a Stampli-voided check | Terminal voided-and-bounced (`clear_status=Returned`, cancel status preserved), counted separately | + +Each applied event is appended to a `history` list attribute (`{event, date, bankRef, amount}`); identical replayed events are idempotent noops. + +**Replay.** The handler accepts an optional payload `{"fromDate": "YYYY-MM-DD", "toDate": "YYYY-MM-DD"}` (strictly validated; `toDate` defaults to `fromDate`) for weekend/outage gap replays and BAI-code enumeration runs. With no payload it queries the trailing 3-day window (today−3 .. today−1), so the Monday run covers Friday–Sunday; overlapping days are idempotent (event identity `{event, date, amount}`). Feed rows without a valid `valueDate` are never applied with a substituted date — they go to unmatched for review. + +**Run summary.** Each run writes an append-only `boa_recon#_#` item (90-day TTL) with counts per classified event type, matched/applied/redeposit/voided-and-bounced/write-conflict totals, the unmatched check numbers and amounts (list capped at 50; full count kept), unknown BAI codes (capped at 20 distinct keys), and the staleness sweep results. Unmatched and unknown-code transactions also `console.error` (Slack alerting is tracked in #71). Payment writes are conditioned on the read snapshot's `status`/`clear_status` and retried once against a fresh read on conflict. + ## Documentation The canonical map of Sea Haven's AWS infrastructure lives in Confluence. This project's `payments-dashboard` stack is represented there as a Mermaid subgraph. diff --git a/package.json b/package.json index aadcb98..5f7be0f 100644 --- a/package.json +++ b/package.json @@ -6,6 +6,9 @@ "files": [ "src/" ], + "scripts": { + "test": "node --test \"tests/**/*.test.js\"" + }, "dependencies": { "@aws-sdk/client-dynamodb": "^3.1087.0", "@aws-sdk/client-lambda": "^3.1087.0", diff --git a/src/boaRecon.js b/src/boaRecon.js new file mode 100644 index 0000000..4a84788 --- /dev/null +++ b/src/boaRecon.js @@ -0,0 +1,551 @@ +// Pure reconciliation logic for fetchBoaTransactions (payments-dashboard#66). +// No AWS clients or environment access here so node:test can exercise the +// classifier, matcher, and state transitions directly. + +import { toISODate } from "./dates.js"; + +// Untrusted values (bank descriptions, references) must be JSON-encoded +// before log interpolation — bank text can carry newlines, which would forge +// CloudWatch log lines (CWE-117). Same rule as processPaymentCsv's logSafe. +export const logSafe = (v) => JSON.stringify(String(v ?? "").slice(0, 128)); + +// Comma-tolerant amount parsing ("1,234.56" bank strings and stored values). +export const parseAmount = (value) => { + const num = parseFloat(String(value ?? "0").replace(/,/g, "").trim()); + return isNaN(num) ? 0 : num; +}; + +const cents = (v) => Math.round(Math.abs(parseAmount(v)) * 100); + +// Sign-insensitive cent equality; zero never matches (a blank amount must +// not pair with another blank amount). +export const amountsEqual = (a, b) => cents(a) > 0 && cents(a) === cents(b); + +// BAI transaction-code -> event map. BoA's Previous Day feed uses standard +// BAI type codes; only 475 (check paid, debit) is empirically confirmed so +// far. The codes for ARP refer-to-maker return credits, return-of-posted- +// check credits (check-numbered and electronic variants), ACH CCD debits and +// return credits, and the true meaning of 255 (the old filter assumed +// "returned check", unverified) are pending the enumeration replay over +// known event dates (2/20, 3/31, 4/8, 6/16-6/23) — add them here as they +// are confirmed. Until then classification falls back to description text, +// and unmapped codes on check-shaped transactions are logged and counted, +// never silently dropped. +export const BAI_CODE_EVENTS = { + 475: { event: "check_paid", direction: "debit" }, +}; + +// Standard BAI ranges: 100-399 are credit type codes, 400-699 are debit +// type codes. Used only when the feed carries no explicit indicator. +export function directionFromCode(code) { + const n = parseInt(String(code ?? ""), 10); + if (!Number.isInteger(n)) return null; + if (n >= 100 && n < 400) return "credit"; + if (n >= 400 && n < 700) return "debit"; + return null; +} + +export function directionOf(txn) { + const indicator = String( + txn.debitCreditIndicator ?? txn.creditDebitIndicator ?? "" + ).toUpperCase(); + if (indicator.includes("DEBIT")) return "debit"; + if (indicator.includes("CREDIT")) return "credit"; + const fromCode = directionFromCode(txn.transactionCode); + if (fromCode) return fromCode; + const amt = parseAmount(txn.amount); + if (amt < 0) return "debit"; + if (amt > 0) return "credit"; + return null; +} + +// Statement description formats observed on the 2026-07-21 reconciliation. +const ARP_RETURN_RE = /^ARP RETURNED CHECK REFER TO MAKER CHECK #\s*(\d+)\b/i; +const POSTED_RETURN_CHECK_RE = + /^RETURN OF POSTED CHECK \/ ITEM \(RECEIVED ON \d{2}-\d{2}\)\s*CHECK #\s*(\d+)\b/i; +const POSTED_RETURN_ELECTRONIC_RE = + /^RETURN OF POSTED CHECK \/ ITEM \(RECEIVED ON \d{2}-\d{2}\)\s*ELECTRONIC TRANSACTION\b/i; +const ACH_PMT_RE = /DES:PAYMENTS\s+ID:PMT\s*(\d+)/i; +const PMT_INFO_RE = /PMT INFO:\s*(.*)$/i; +const CHECK_PAID_RE = /^CHECK\s{0,10}#?\s{0,10}0*(\d{1,12})$/i; +// Trailing digit run at the end of PMT INFO — Stampli recently started +// embedding the payment number there, and the bank wraps it with arbitrary +// internal spaces ("21 222000108", "2122200 0256"). Run length is bounded +// against pathological input. +const EMBEDDED_NUMBER_RE = /(\d[\d ]{6,40}\d)\s*$/; + +// Bound untrusted text before any regex work (ReDoS hardening). +const MAX_DESCRIPTION_LEN = 500; + +const stripLeadingZeros = (s) => String(s ?? "").replace(/^0+(?=\d)/, ""); + +// Classify one Previous Day transaction into a reconciliation event. +// Precedence: confirmed BAI code map first, then description text. Anything +// unmapped that still looks check/payment-shaped comes back as "unknown" so +// the caller can log and count it — never silently drop it. Everything else +// (transfers, misc bank activity) is "ignored". +export function classifyTransaction(txn) { + const code = String(txn.transactionCode ?? "").trim(); + const description = String( + txn.text ?? txn.description ?? txn.transactionDescription ?? "" + ) + .slice(0, MAX_DESCRIPTION_LEN) + .trim(); + const customerReference = stripLeadingZeros(String(txn.customerReference ?? "").trim()); + const bankReference = String(txn.bankReference ?? "").trim(); + const amount = Math.abs(parseAmount(txn.amount)); + const direction = BAI_CODE_EVENTS[code]?.direction ?? directionOf(txn); + + const base = { + code, + description, + customerReference, + bankReference, + amount, + direction, + checkNumber: null, + pmtId: null, + embeddedPaymentNumber: null, + vendorText: null, + checkShaped: false, + }; + + // Description facts, extracted regardless of code so a code-mapped event + // still carries the check number / PMT id it references. + let descriptionEvent = null; + let m; + if ((m = ARP_RETURN_RE.exec(description))) { + descriptionEvent = "check_return"; + base.checkNumber = stripLeadingZeros(m[1]); + } else if ((m = POSTED_RETURN_CHECK_RE.exec(description))) { + descriptionEvent = "check_return"; + base.checkNumber = stripLeadingZeros(m[1]); + } else if (POSTED_RETURN_ELECTRONIC_RE.test(description)) { + descriptionEvent = "electronic_return"; + } else if ((m = ACH_PMT_RE.exec(description))) { + base.pmtId = m[1]; + const info = PMT_INFO_RE.exec(description); + if (info) { + let vendorText = info[1].trim(); + const embedded = EMBEDDED_NUMBER_RE.exec(vendorText); + if (embedded && embedded[1].replace(/ /g, "").length >= 8) { + base.embeddedPaymentNumber = embedded[1].replace(/ /g, ""); + vendorText = vendorText.slice(0, embedded.index).trim(); + } + base.vendorText = vendorText || null; + } + if (direction === "debit") descriptionEvent = "ach_debit"; + else if (direction === "credit") descriptionEvent = "ach_return"; + // direction unknown -> leave null; falls through to "unknown" below. + } else if ((m = CHECK_PAID_RE.exec(description))) { + descriptionEvent = "check_paid"; + base.checkNumber = stripLeadingZeros(m[1]); + } + + const event = BAI_CODE_EVENTS[code]?.event ?? descriptionEvent; + + if (event === "check_paid" && !base.checkNumber) { + base.checkNumber = customerReference || null; + } + if (event === "check_return" && !base.checkNumber) { + base.checkNumber = customerReference || null; + } + + if (event) return { ...base, event }; + + base.checkShaped = + Boolean(customerReference) || /CHECK/i.test(description) || Boolean(base.pmtId); + return { ...base, event: base.checkShaped ? "unknown" : "ignored" }; +} + +// Days from a stored Stampli send date (canonical MM/DD/YYYY) to an ISO +// reference date; null when the stored date does not parse. +function daysSinceIssue(payment, refISO) { + const iso = toISODate(payment.send_payment_on); + if (!iso) return null; + return Math.round((Date.parse(refISO) - Date.parse(iso)) / 86400000); +} + +const issuedWithinDays = (payment, refISO, days) => { + const d = daysSinceIssue(payment, refISO); + return d !== null && d >= 0 && d <= days; +}; + +// True when a's digits appear, in order, inside b (digit-dropped mangling: +// posting 1222000012 came from issued 11222000012). +function isDigitSubsequence(a, b) { + if (a.length > b.length) return false; + let i = 0; + for (let j = 0; j < b.length && i < a.length; j++) { + if (a[i] === b[j]) i++; + } + return i === a.length; +} + +export const digitsCorroborate = (a, b) => + Boolean(a && b) && (isDigitSubsequence(a, b) || isDigitSubsequence(b, a)); + +// Match a check event (paid debit or return credit) against payment records. +// Match on check number AND amount, evaluating all candidates — never +// first-match on number alone: bank postings drop/collapse digits on long +// check numbers, so a posting under one number can belong to another check +// (or to a number we never issued). +// +// A number match with the WRONG amount never falls through to the amount +// fallback: that pattern is the altered-check / collapsed-posting signal a +// human must review, so it goes to unmatched. The amount-only fallback +// (number unknown) is limited to checks issued in the last 120 days AND +// requires digit-subsequence corroboration between the posting's number and +// the candidate's; return credits additionally require a bank-confirmed +// candidate. Anything but a single fallback candidate is unmatched: no +// write on ambiguity. +export function matchCheckTransaction(classified, payments, refDateISO) { + const { checkNumber, amount, event } = classified; + const checks = payments.filter((p) => p.method === "Check" && p.check_number); + + const numberMatches = checkNumber + ? checks.filter((p) => p.check_number === checkNumber) + : []; + const exact = numberMatches.filter((p) => amountsEqual(p.amount_usd, amount)); + if (exact.length === 1) return { payment: exact[0], matchedBy: "number+amount" }; + if (exact.length > 1) return { unmatched: "multiple number+amount matches" }; + if (numberMatches.length) { + return { unmatched: "number matched, amount mismatch" }; + } + + const fallback = checks.filter( + (p) => + amountsEqual(p.amount_usd, amount) && + issuedWithinDays(p, refDateISO, 120) && + digitsCorroborate(checkNumber, p.check_number) && + (event !== "check_return" || + p.clear_status === "Cleared" || + p.clear_status === "Returned") + ); + if (fallback.length === 1) return { payment: fallback[0], matchedBy: "amount" }; + if (fallback.length > 1) { + return { unmatched: `amount fallback ambiguous (${fallback.length} candidates)` }; + } + return { unmatched: "no number match; no corroborated amount fallback" }; +} + +// Electronic return credits carry no check number at all. Match by exact +// amount among bank-confirmed payments issued in the last 120 days +// (clear_status Cleared, or Returned so replays of an already-applied +// return dedupe to a noop instead of alerting). Ambiguity is unmatched. +export function matchElectronicReturn(classified, payments, refDateISO) { + const candidates = payments.filter( + (p) => + p.method === "Check" && + (p.clear_status === "Cleared" || p.clear_status === "Returned") && + amountsEqual(p.amount_usd, classified.amount) && + issuedWithinDays(p, refDateISO, 120) + ); + if (candidates.length === 1) return { payment: candidates[0], matchedBy: "amount+cleared" }; + if (candidates.length > 1) { + return { unmatched: `electronic return ambiguous (${candidates.length} candidates)` }; + } + return { unmatched: "no bank-confirmed payment with this amount" }; +} + +// Bank originator names are truncated ("ALLIANCE SANITAT") and punctuation +// drifts, so vendor comparison is prefix-based over normalized text. +const normalizeVendor = (s) => + String(s ?? "") + .toUpperCase() + .replace(/[^A-Z0-9]/g, ""); + +// Prefix matching only counts when the shorter normalized string is at +// least 10 chars; short names must match exactly ("ACME" must not claim +// "ACME Plumbing Co"). +export function vendorMatches(payee, vendorText) { + const a = normalizeVendor(payee); + const b = normalizeVendor(vendorText); + if (!a || !b) return false; + if (a === b) return true; + if (Math.min(a.length, b.length) < 10) return false; + return a.startsWith(b) || b.startsWith(a); +} + +// ACH settles up to 8 days after the Stampli send date, and can post a +// couple of days early; the posting must fall within send_payment_on +// -2..+14 days. +const withinSendWindow = (payment, postingISO) => { + const d = daysSinceIssue(payment, postingISO); + return d !== null && d >= -2 && d <= 14; +}; + +// Candidate cleared_date must sit within `days` before the credit posting; +// records without a valid cleared_date are excluded. +const clearedWithinDaysBefore = (payment, postingISO, days) => { + if (!isValidISODate(payment.cleared_date)) return false; + const d = Math.round((Date.parse(postingISO) - Date.parse(payment.cleared_date)) / 86400000); + return d >= 0 && d <= days; +}; + +// Match an ACH CCD debit or return credit (payments-dashboard#69). +// Rungs, every one amount-corroborated: +// 1. stored pmt_id (reversals reuse the original PMT id); a pmt_id match +// with the wrong amount is the partial-reversal human case — unmatched. +// 2. Stampli payment number embedded in PMT INFO (spaces stripped); a +// wrong-amount embedded match falls through. +// 3. vendor + exact amount within the send window (candidates whose stored +// pmt_id differs from the transaction's are excluded). +// 4. return credits only: if the credit's PMT id attributes to no stored +// pmt_id, that is an unmatched alert ("unknown PMT id") — never an +// amount guess. Credits without a PMT id may fall back to a unique +// same-amount match among bank-confirmed ACH payments whose +// cleared_date is within 45 days before the credit. +// Ambiguity is always unmatched — no write. +export function matchAchTransaction(classified, payments, postingISO) { + const { event, pmtId, embeddedPaymentNumber, vendorText, amount } = classified; + const achs = payments.filter((p) => p.method === "ACH"); + + if (pmtId) { + const byPmtId = achs.filter((p) => p.pmt_id === pmtId); + if (byPmtId.length === 1) { + if (amountsEqual(byPmtId[0].amount_usd, amount)) { + return { payment: byPmtId[0], matchedBy: "pmt_id" }; + } + return { unmatched: "pmt_id matched, amount mismatch" }; + } + if (byPmtId.length > 1) return { unmatched: "multiple pmt_id matches" }; + } + + if (embeddedPaymentNumber) { + const byNumber = achs.filter( + (p) => p.check_number === embeddedPaymentNumber && amountsEqual(p.amount_usd, amount) + ); + if (byNumber.length === 1) return { payment: byNumber[0], matchedBy: "payment-number+amount" }; + } + + const pmtIdConflicts = (p) => Boolean(pmtId && p.pmt_id && p.pmt_id !== pmtId); + + const byVendor = achs.filter( + (p) => + !pmtIdConflicts(p) && + amountsEqual(p.amount_usd, amount) && + vendorMatches(p.payee, vendorText) && + withinSendWindow(p, postingISO) + ); + if (byVendor.length === 1) return { payment: byVendor[0], matchedBy: "vendor+amount" }; + if (byVendor.length > 1) { + return { unmatched: `vendor+amount ambiguous (${byVendor.length} candidates)` }; + } + + if (event === "ach_return") { + if (pmtId) return { unmatched: "unknown PMT id" }; + const byAmount = achs.filter( + (p) => + !pmtIdConflicts(p) && + (p.clear_status === "Cleared" || p.clear_status === "Returned") && + amountsEqual(p.amount_usd, amount) && + clearedWithinDaysBefore(p, postingISO, 45) + ); + if (byAmount.length === 1) return { payment: byAmount[0], matchedBy: "amount+cleared" }; + if (byAmount.length > 1) { + return { unmatched: `return amount ambiguous (${byAmount.length} candidates)` }; + } + } + + return { unmatched: "no unique pmt_id, payment-number, or vendor+amount match" }; +} + +const CANCEL_STATUSES = ["voided", "cancelled", "canceled", "marked as void"]; +export const isCancelStatus = (s) => CANCEL_STATUSES.includes(String(s ?? "").toLowerCase()); + +const RETURN_EVENTS = new Set(["check_return", "electronic_return", "ach_return"]); + +// Decide the DDB write for a matched bank event. Pure: returns the fields +// to set plus the history entry; the handler turns it into an UpdateCommand +// and mirrors the fields onto its in-memory copy. +// +// Invariant: every non-noop result sets BOTH status and clear_status — the +// 2026-07-21 reconciliation traced five missed returns ($5,256.62) to +// writers touching one field but not the other. +// +// Status semantics (documented decision, #66): `clear_status` is bank truth +// ("Cleared without a subsequent return is permanent" keys off it). `status` +// stays on the CSV lifecycle ladder, which has no "Returned" rung, so on a +// return it is re-written with its current value to keep the both-fields +// invariant; consumers that need bounce visibility read clear_status. On a +// paid debit against a canceled record (a voided check the bank paid — +// expected without Positive Pay), the cancel status is likewise preserved: +// the ARP return that follows lands as terminal voided-and-bounced. +export function applyEvent(payment, classified, eventDateISO) { + const { event, amount, bankReference } = classified; + const canceled = isCancelStatus(payment.status); + const historyEvent = { + event, + date: eventDateISO, + bankRef: bankReference, + amount, + }; + + // Replay idempotence: event identity is {event, date, amount} — bankRef + // is deliberately excluded because feeds omit/reformat it between runs. + // Assumption: the bank never posts two DISTINCT same-type events for the + // same payment on the same date with the same amount; an identical + // identity already in history is therefore the same event, and a noop. + const alreadyApplied = (payment.history || []).some( + (h) => + h.event === historyEvent.event && + h.date === historyEvent.date && + amountsEqual(h.amount, amount) + ); + if (alreadyApplied) return { kind: "noop", updates: null, historyEvent: null }; + + if (event === "check_paid" || event === "ach_debit") { + // A paid event dated on/before the latest known return is a replayed + // original paid debit, not a redeposit — never re-clear from it. + const latestReturnDate = (payment.history || []) + .filter((h) => RETURN_EVENTS.has(h.event) && typeof h.date === "string") + .map((h) => h.date) + .sort() + .pop(); + if (latestReturnDate && eventDateISO <= latestReturnDate) { + return { kind: "noop", updates: null, historyEvent: null }; + } + const redeposit = payment.clear_status === "Returned"; + const updates = { + status: canceled ? payment.status : "Cleared", + clear_status: "Cleared", + paid_date: eventDateISO, + cleared_date: eventDateISO, + bank_reference: bankReference, + }; + if (event === "ach_debit" && classified.pmtId) updates.pmt_id = classified.pmtId; + return { + kind: redeposit ? "redeposit" : canceled ? "cleared_on_canceled" : "cleared", + updates, + historyEvent, + }; + } + + if (event === "check_return" || event === "electronic_return" || event === "ach_return") { + const updates = { + status: payment.status ?? "", + clear_status: "Returned", + returned_date: eventDateISO, + bank_reference: bankReference, + }; + if (event === "ach_return" && classified.pmtId) updates.pmt_id = classified.pmtId; + return { + // A return against a canceled record is the expected void-then-bounce + // ARP cycle: terminal voided-and-bounced, counted separately. + kind: canceled ? "voided_and_bounced" : "returned", + updates, + historyEvent, + }; + } + + return { kind: "noop", updates: null, historyEvent: null }; +} + +const ISO_DATE_RE = /^\d{4}-\d{2}-\d{2}$/; + +export function isValidISODate(s) { + if (typeof s !== "string" || !ISO_DATE_RE.test(s)) return false; + const [y, mo, d] = s.split("-").map(Number); + const dt = new Date(Date.UTC(y, mo - 1, d)); + return ( + dt.getUTCFullYear() === y && dt.getUTCMonth() === mo - 1 && dt.getUTCDate() === d + ); +} + +// Optional {fromDate, toDate} replay payload. The default is a trailing +// 3-day window (today-3 .. today-1): the previous-day feed only runs on +// weekdays, so the Monday run must cover Friday through Sunday. Overlapping +// days are safe — event identity makes replays idempotent. Strings are +// validated strictly so a malformed payload fails loudly instead of +// querying a garbage range. +export function resolveDateRange(event, now = new Date()) { + const hasFrom = event?.fromDate != null; + const hasTo = event?.toDate != null; + if (!hasFrom && !hasTo) { + const day = 24 * 60 * 60 * 1000; + return { + fromDate: new Date(now.getTime() - 3 * day).toISOString().split("T")[0], + toDate: new Date(now.getTime() - day).toISOString().split("T")[0], + }; + } + const fromDate = hasFrom ? event.fromDate : event.toDate; + const toDate = hasTo ? event.toDate : event.fromDate; + if (!isValidISODate(fromDate) || !isValidISODate(toDate)) { + throw new Error( + `fromDate/toDate must be valid YYYY-MM-DD strings: ` + + `fromDate=${logSafe(fromDate)}, toDate=${logSafe(toDate)}` + ); + } + if (fromDate > toDate) { + throw new Error( + `fromDate must be <= toDate: fromDate=${logSafe(fromDate)}, toDate=${logSafe(toDate)}` + ); + } + return { fromDate, toDate }; +} + +// Staleness sweep (#66 review F6): flag records the bank has never +// confirmed. ACH with no clear_status and a send date older than 16 days +// (settlement lags at most ~8 business days) and checks older than 60 days +// are surfaced in the run summary; cancel-status records are exempt. +export const STALE_LIST_CAP = 50; +export function sweepStalePayments(payments, todayISO) { + const staleAch = []; + let staleAchCount = 0; + let staleChecksCount = 0; + for (const p of payments) { + if (p.clear_status || isCancelStatus(p.status)) continue; + const age = daysSinceIssue(p, todayISO); + if (age === null) continue; + if (p.method === "ACH" && age > 16) { + staleAchCount++; + if (staleAch.length < STALE_LIST_CAP) { + staleAch.push({ + check_number: p.check_number, + amount: p.amount_usd, + send_payment_on: p.send_payment_on, + }); + } + } else if (p.method === "Check" && age > 60) { + staleChecksCount++; + } + } + return { staleAch, staleAchCount, staleChecksCount }; +} + +// Conditioned write for an applied event (#66 review F7): the update only +// lands if the snapshot's status/clear_status are still current, so a +// concurrent CSV upsert can't be silently interleaved. The handler retries +// once against a fresh read on ConditionalCheckFailedException. +export function buildEventUpdate(tableName, payment, applied) { + const sets = ["#history = list_append(if_not_exists(#history, :empty), :hist)"]; + const names = { "#history": "history" }; + const values = { ":empty": [], ":hist": [applied.historyEvent] }; + Object.entries(applied.updates).forEach(([field, value], i) => { + names[`#f${i}`] = field; + values[`:v${i}`] = value; + sets.push(`#f${i} = :v${i}`); + }); + const conditions = []; + [ + ["status", payment.status], + ["clear_status", payment.clear_status], + ].forEach(([field, snapshot], i) => { + names[`#c${i}`] = field; + if (snapshot == null) { + conditions.push(`attribute_not_exists(#c${i})`); + } else { + values[`:c${i}`] = snapshot; + conditions.push(`#c${i} = :c${i}`); + } + }); + return { + TableName: tableName, + Key: { pk: payment.pk }, + UpdateExpression: `SET ${sets.join(", ")}`, + ConditionExpression: conditions.join(" AND "), + ExpressionAttributeNames: names, + ExpressionAttributeValues: values, + }; +} diff --git a/src/fetchBoaTransactions.js b/src/fetchBoaTransactions.js index 660e8ab..45d2eec 100644 --- a/src/fetchBoaTransactions.js +++ b/src/fetchBoaTransactions.js @@ -1,12 +1,27 @@ import { DynamoDBClient } from "@aws-sdk/client-dynamodb"; -import { DynamoDBDocumentClient, ScanCommand, UpdateCommand } from "@aws-sdk/lib-dynamodb"; +import { DynamoDBDocumentClient, GetCommand, PutCommand, ScanCommand, UpdateCommand } from "@aws-sdk/lib-dynamodb"; import { SecretsManagerClient, GetSecretValueCommand } from "@aws-sdk/client-secrets-manager"; +import { + applyEvent, + buildEventUpdate, + classifyTransaction, + isValidISODate, + logSafe, + matchAchTransaction, + matchCheckTransaction, + matchElectronicReturn, + resolveDateRange, + sweepStalePayments, +} from "./boaRecon.js"; const ddb = DynamoDBDocumentClient.from(new DynamoDBClient()); const secrets = new SecretsManagerClient(); const TABLE_NAME = process.env.TABLE_NAME; const BOA_BASE_URL = process.env.BOA_BASE_URL; +const UNMATCHED_LIST_CAP = 50; +const UNKNOWN_CODES_CAP = 20; + let cachedCreds; async function getReportingCreds() { if (cachedCreds) return cachedCreds; @@ -17,6 +32,16 @@ async function getReportingCreds() { return cachedCreds; } +// Parse a BoA response body without ever surfacing a body snippet — a +// malformed 200 must not leak account data into logs/errors. +async function parseJsonResponse(res, label) { + try { + return await res.json(); + } catch { + throw new Error(`BoA ${label} response not JSON: HTTP ${res.status}`); + } +} + async function getAccessToken(applicationID, clientId, clientSecret) { const res = await fetch(`${BOA_BASE_URL}/authn/v1/client-authentication`, { method: "POST", @@ -28,24 +53,24 @@ async function getAccessToken(applicationID, clientId, clientSecret) { }); if (!res.ok) { - const text = await res.text(); - throw new Error(`OAuth token exchange failed: ${res.status} - ${text}`); + throw new Error(`OAuth token exchange failed: HTTP ${res.status}`); } - const data = await res.json(); + const data = await parseJsonResponse(res, "authn"); return data.access_token; } -export const handler = async () => { +const directionRank = (d) => (d === "debit" ? 0 : d === "credit" ? 1 : 2); + +export const handler = async (event) => { + // Optional replay payload {fromDate, toDate}; default is the trailing + // 3-day window so the Monday run covers Friday through Sunday. + const { fromDate, toDate } = resolveDateRange(event ?? {}); + const { appId, clientId, token: clientSecret, accountNumber, bankId } = await getReportingCreds(); const bearerToken = await getAccessToken(appId, clientId, clientSecret); - // Get yesterday's date in YYYY-MM-DD - const yesterday = new Date(); - yesterday.setDate(yesterday.getDate() - 1); - const dateStr = yesterday.toISOString().split("T")[0]; - // Call CashPro Previous Day Transaction Inquiry const res = await fetch(`${BOA_BASE_URL}/cashpro/reporting/v1/transaction-inquiries/previous-day`, { method: "POST", @@ -54,44 +79,46 @@ export const handler = async () => { Authorization: `Bearer ${bearerToken}`, }, body: JSON.stringify({ - fromDate: dateStr, - toDate: dateStr, + fromDate, + toDate, accounts: [{ accountNumber, bankId }], }), }); if (!res.ok) { - const text = await res.text(); - throw new Error(`BoA API error ${res.status}: ${text}`); + throw new Error(`BoA API error: HTTP ${res.status}`); } - const data = await res.json(); + const data = await parseJsonResponse(res, "reporting"); // Response shape: { accountTransactions: [{ accountNumber, bankId, currency, transactions: [...] }] } const allTransactions = (data.accountTransactions || []).flatMap( (acct) => acct.transactions || [] ); - // Filter for cleared checks (475) and returned checks (255) - const relevant = allTransactions.filter( - (t) => t.transactionCode === "475" || t.transactionCode === "255" + // Process in posting-date order so multi-day replays apply paid -> return + // -> redeposit sequences in the order the bank did; within a day, debits + // before credits (deterministic, and a same-day return follows its debit). + const classifiedTxns = allTransactions.map((txn) => ({ + txn, + classified: classifyTransaction(txn), + })); + classifiedTxns.sort( + (a, b) => + String(a.txn.valueDate ?? "").localeCompare(String(b.txn.valueDate ?? "")) || + directionRank(a.classified.direction) - directionRank(b.classified.direction) ); - if (!relevant.length) { - console.log(`No check transactions (475/255) found for ${dateStr}`); - return { statusCode: 200, body: `No relevant transactions for ${dateStr}` }; - } - - // Load all check payments from DynamoDB to match against + // Load all payment records (Check AND ACH — ACH is bank-confirmed here + // too, #69) to match against. const payments = []; let lastKey; do { const result = await ddb.send( new ScanCommand({ TableName: TABLE_NAME, - FilterExpression: "begins_with(pk, :prefix) AND #m = :method", - ExpressionAttributeNames: { "#m": "method" }, - ExpressionAttributeValues: { ":prefix": "payment#", ":method": "Check" }, + FilterExpression: "begins_with(pk, :prefix)", + ExpressionAttributeValues: { ":prefix": "payment#" }, ExclusiveStartKey: lastKey, }) ); @@ -99,44 +126,219 @@ export const handler = async () => { lastKey = result.LastEvaluatedKey; } while (lastKey); - let matched = 0; + const summary = { + transactions_seen: allTransactions.length, + classified: {}, + matched: 0, + applied: 0, + already_applied: 0, + redeposits: 0, + voided_and_bounced: 0, + write_conflicts: 0, + unmatched_count: 0, + unmatched: [], + unknown_count: 0, + unknown_codes: {}, + }; - for (const txn of relevant) { - const custRef = (txn.customerReference || "").replace(/^0+/, ""); - const bankRef = txn.bankReference || ""; - - // Match customerReference (check number with leading zeros stripped) to our check_number - const matchedPayment = payments.find((p) => - p.check_number && p.check_number === custRef + const recordUnmatched = (classified, reason) => { + summary.unmatched_count++; + if (summary.unmatched.length < UNMATCHED_LIST_CAP) { + summary.unmatched.push({ + event: classified.event, + check_number: classified.checkNumber || classified.embeddedPaymentNumber, + amount: classified.amount, + reason, + }); + } + console.error( + `Unmatched ${classified.event}: check=${logSafe(classified.checkNumber || classified.embeddedPaymentNumber)}, ` + + `amount=${classified.amount}, reason=${reason} ` + + `(bankRef: ${logSafe(classified.bankReference)})` ); + }; - if (!matchedPayment) { - console.log(`No match for customerReference: ${txn.customerReference} (bankRef: ${bankRef})`); + for (const { txn, classified } of classifiedTxns) { + summary.classified[classified.event] = (summary.classified[classified.event] || 0) + 1; + + if (classified.event === "ignored") continue; + + if (classified.event === "unknown") { + summary.unknown_count++; + const codeKey = (classified.code || "?").slice(0, 16); + if ( + codeKey in summary.unknown_codes || + Object.keys(summary.unknown_codes).length < UNKNOWN_CODES_CAP + ) { + summary.unknown_codes[codeKey] = (summary.unknown_codes[codeKey] || 0) + 1; + } + console.error( + `Unknown check-shaped transaction: code=${logSafe(classified.code)}, ` + + `description=${logSafe(classified.description)}, ` + + `ref=${logSafe(classified.customerReference)}, amount=${classified.amount}` + ); continue; } - const clearStatus = txn.transactionCode === "475" ? "Cleared" : "Returned"; + // Never substitute a date: event identity and transition ordering both + // key on the posting date, so a feed row without one is a human case. + if (!isValidISODate(txn.valueDate)) { + recordUnmatched(classified, "missing valueDate"); + continue; + } + const eventDate = txn.valueDate; - await ddb.send( - new UpdateCommand({ - TableName: TABLE_NAME, - Key: { pk: matchedPayment.pk }, - UpdateExpression: "SET clear_status = :status, bank_reference = :ref, cleared_date = :date", - ExpressionAttributeValues: { - ":status": clearStatus, - ":ref": bankRef, - ":date": txn.valueDate || dateStr, - }, - }) + let match; + if (classified.event === "ach_debit" || classified.event === "ach_return") { + match = matchAchTransaction(classified, payments, eventDate); + } else if (classified.event === "electronic_return") { + match = matchElectronicReturn(classified, payments, eventDate); + } else { + match = matchCheckTransaction(classified, payments, eventDate); + } + + if (!match.payment) { + recordUnmatched(classified, match.unmatched); + continue; + } + + if (match.matchedBy === "vendor+amount") { + // Audit trail for the loosest ACH rung. + console.log( + `ACH vendor+amount match: payee=${logSafe(match.payment.payee)}, ` + + `bankVendor=${logSafe(classified.vendorText)}, pmt=${logSafe(classified.pmtId)}` + ); + } + + summary.matched++; + const target = match.payment; + let applied = applyEvent(target, classified, eventDate); + if (applied.kind === "noop") { + summary.already_applied++; + continue; + } + + // Conditioned write: assert the snapshot's status/clear_status are + // still current; on conflict, re-read, re-derive, retry once. + let outcome = "conflict"; + for (let attempt = 0; attempt < 2; attempt++) { + try { + await ddb.send(new UpdateCommand(buildEventUpdate(TABLE_NAME, target, applied))); + outcome = "written"; + break; + } catch (err) { + if (err.name !== "ConditionalCheckFailedException") throw err; + if (attempt === 1) break; + const { Item: fresh } = await ddb.send( + new GetCommand({ TableName: TABLE_NAME, Key: { pk: target.pk } }) + ); + if (!fresh) break; + // Refresh the in-memory record in place (it is shared with the + // payments array) and re-derive the transition. + for (const k of Object.keys(target)) delete target[k]; + Object.assign(target, fresh); + applied = applyEvent(target, classified, eventDate); + if (applied.kind === "noop") { + outcome = "noop"; + break; + } + } + } + + if (outcome === "noop") { + summary.already_applied++; + continue; + } + if (outcome !== "written") { + summary.write_conflicts++; + console.error( + `Write conflict (gave up after retry): pk=${logSafe(target.pk)}, event=${classified.event}` + ); + continue; + } + + summary.applied++; + if (applied.kind === "redeposit") summary.redeposits++; + if (applied.kind === "voided_and_bounced") summary.voided_and_bounced++; + + // Mirror the write onto the in-memory copy so later transactions in the + // same run (return after paid, redeposit after return) see current state. + Object.assign(target, applied.updates); + target.history = [...(target.history || []), applied.historyEvent]; + + console.log( + `${applied.kind}: check ${logSafe(target.check_number)} via ${match.matchedBy} ` + + `(bankRef: ${logSafe(classified.bankReference)})` ); - - matched++; - console.log(`${clearStatus}: check ${matchedPayment.check_number} (bankRef: ${bankRef})`); } - console.log(`Processed ${relevant.length} transactions, matched ${matched} payments`); + // Staleness sweep: payments the bank has never confirmed (#69). + const todayISO = new Date().toISOString().split("T")[0]; + const stale = sweepStalePayments(payments, todayISO); + if (stale.staleAchCount) { + console.error( + `Stale ACH (no bank settlement, sent > 16 days ago): ${stale.staleAchCount} records, ` + + `checks=${logSafe(stale.staleAch.map((s) => s.check_number).join(","))}` + ); + } + if (stale.staleChecksCount) { + console.error( + `Stale checks (no bank activity, issued > 60 days ago): ${stale.staleChecksCount} records` + ); + } + + // Run summary record for auditing/alerting (Slack wiring is #71). The pk + // is append-only (run timestamp suffix) so re-runs never overwrite a + // prior run's record. + const runAt = new Date().toISOString(); + const runKey = `boa_recon#${fromDate}_${toDate}#${runAt}`; + await ddb.send( + new PutCommand({ + TableName: TABLE_NAME, + Item: { + pk: runKey, + run_at: runAt, + from_date: fromDate, + to_date: toDate, + transactions_seen: summary.transactions_seen, + classified: summary.classified, + matched: summary.matched, + applied: summary.applied, + already_applied: summary.already_applied, + redeposits: summary.redeposits, + voided_and_bounced: summary.voided_and_bounced, + write_conflicts: summary.write_conflicts, + unmatched_count: summary.unmatched_count, + unmatched: summary.unmatched, + unknown_count: summary.unknown_count, + unknown_codes: summary.unknown_codes, + stale_ach: stale.staleAch, + stale_ach_count: stale.staleAchCount, + stale_checks_count: stale.staleChecksCount, + ttl: Math.floor(Date.now() / 1000) + 90 * 24 * 60 * 60, + }, + }) + ); + + if (summary.unmatched_count || summary.unknown_count || summary.write_conflicts) { + console.error( + `Reconciliation ${fromDate}..${toDate}: ${summary.unmatched_count} unmatched, ` + + `${summary.unknown_count} unknown-code transactions, ` + + `${summary.write_conflicts} write conflicts (see ${logSafe(runKey)})` + ); + } + + console.log( + `Processed ${summary.transactions_seen} transactions ${fromDate}..${toDate}: ` + + `${summary.matched} matched, ${summary.applied} applied, ` + + `${summary.already_applied} already applied, ${summary.redeposits} redeposits, ` + + `${summary.voided_and_bounced} voided-and-bounced, ${summary.unmatched_count} unmatched` + ); + return { statusCode: 200, - body: `${matched} of ${relevant.length} transactions matched to payments`, + body: + `${summary.matched} of ${summary.transactions_seen} transactions matched ` + + `(${summary.applied} applied, ${summary.unmatched_count} unmatched)`, }; }; diff --git a/src/processPaymentCsv.js b/src/processPaymentCsv.js index bb29558..3087c43 100644 --- a/src/processPaymentCsv.js +++ b/src/processPaymentCsv.js @@ -299,14 +299,10 @@ export const handler = async (event) => { continue; } - // ACH payments clear automatically on their send date - if (method === "ACH" && !cancelStatuses.includes(status.toLowerCase())) { - const sendDate = toISODate(sendOn); - const today = new Date().toISOString().slice(0, 10); - if (sendDate && sendDate <= today) { - status = "Cleared"; - } - } + // ACH rows keep their Stampli status — the bank confirms settlement via + // fetchBoaTransactions (#69). The old send-date auto-clear marked ACH + // Cleared days before the debit settled (and past bounced settlements + // that re-debited later were invisible to it). const pk = `payment#${checkNumber}`; diff --git a/tests/boaRecon.test.js b/tests/boaRecon.test.js new file mode 100644 index 0000000..ff945ad --- /dev/null +++ b/tests/boaRecon.test.js @@ -0,0 +1,784 @@ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { + BAI_CODE_EVENTS, + STALE_LIST_CAP, + amountsEqual, + applyEvent, + buildEventUpdate, + classifyTransaction, + digitsCorroborate, + isCancelStatus, + isValidISODate, + matchAchTransaction, + matchCheckTransaction, + matchElectronicReturn, + resolveDateRange, + sweepStalePayments, + vendorMatches, +} from "../src/boaRecon.js"; + +// Description fixtures are real statement lines from the 2026-07-21 +// reconciliation (amounts/refs anonymized where they don't matter). + +test("classifier: BAI code 475 is a check paid debit", () => { + const c = classifyTransaction({ + transactionCode: "475", + customerReference: "0001122200030", + bankReference: "813312345", + amount: "1,892.62", + }); + assert.equal(c.event, "check_paid"); + assert.equal(c.direction, "debit"); + assert.equal(c.checkNumber, "1122200030"); + assert.equal(c.amount, 1892.62); + assert.equal(BAI_CODE_EVENTS["475"].event, "check_paid"); +}); + +test("classifier: statement-style 'Check ' description is check paid", () => { + const c = classifyTransaction({ text: "Check 3176", amount: "-350" }); + assert.equal(c.event, "check_paid"); + assert.equal(c.checkNumber, "3176"); +}); + +test("classifier: ARP refer-to-maker return credit", () => { + const c = classifyTransaction({ + transactionCode: "354", + text: "ARP RETURNED CHECK REFER TO MAKER CHECK # 1122200030 PAID DATE 02/19/26", + amount: "1,892.62", + }); + assert.equal(c.event, "check_return"); + assert.equal(c.direction, "credit"); + assert.equal(c.checkNumber, "1122200030"); +}); + +test("classifier: return of posted check, check-numbered variant", () => { + const c = classifyTransaction({ + text: "RETURN OF POSTED CHECK / ITEM (RECEIVED ON 03-02) CHECK #1122200236", + amount: "3,350.00", + }); + assert.equal(c.event, "check_return"); + assert.equal(c.checkNumber, "1122200236"); +}); + +test("classifier: return of posted check, electronic variant has no check number", () => { + const c = classifyTransaction({ + text: "RETURN OF POSTED CHECK / ITEM (RECEIVED ON 03-02) ELECTRONIC TRANSACTION", + amount: "3,992.15", + }); + assert.equal(c.event, "electronic_return"); + assert.equal(c.checkNumber, null); +}); + +test("classifier: ACH CCD debit without embedded payment number", () => { + const c = classifyTransaction({ + text: "FISK EXCAVATING DES:PAYMENTS ID:PMT 6814295 INDN:Sea Haven Industries, CO ID:1811679038 CCD PMT INFO:Fisk Excavating Inc", + amount: "-650", + }); + assert.equal(c.event, "ach_debit"); + assert.equal(c.pmtId, "6814295"); + assert.equal(c.embeddedPaymentNumber, null); + assert.equal(c.vendorText, "Fisk Excavating Inc"); +}); + +test("classifier: ACH debit with embedded payment number", () => { + const c = classifyTransaction({ + text: "COBRA SEPTIC DES:PAYMENTS ID:PMT 7584198 INDN:Sea Haven Industries, CO ID:1811679038 CCD PMT INFO:Cobra Septic 21222000264", + amount: "-8,300.00", + }); + assert.equal(c.event, "ach_debit"); + assert.equal(c.pmtId, "7584198"); + assert.equal(c.embeddedPaymentNumber, "21222000264"); + assert.equal(c.vendorText, "Cobra Septic"); +}); + +test("classifier: embedded payment number with internal space '21 222000273'", () => { + const c = classifyTransaction({ + text: "HALL PUMP SALES DES:PAYMENTS ID:PMT 7617407 INDN:Sea Haven Industries, CO ID:1811679038 CCD PMT INFO:Hall Pump Sales & Service Corporation 21 222000273", + amount: "-5,495.00", + }); + assert.equal(c.embeddedPaymentNumber, "21222000273"); + assert.equal(c.vendorText, "Hall Pump Sales & Service Corporation"); +}); + +test("classifier: embedded payment number with internal space '2122200 0256'", () => { + const c = classifyTransaction({ + text: "STERLING SEPTIC DES:PAYMENTS ID:PMT 7603631 INDN:Sea Haven Industries, CO ID:1811679038 CCD PMT INFO:Sterling Septic & Plumbing, LLC. 2122200 0256", + amount: "-1,407.59", + }); + assert.equal(c.embeddedPaymentNumber, "21222000256"); + assert.equal(c.vendorText, "Sterling Septic & Plumbing, LLC."); +}); + +test("classifier: ACH reversal credit reusing PMT id is ach_return", () => { + const c = classifyTransaction({ + text: "SEA HAVEN INDUST DES:PAYMENTS ID:PMT 7176466 INDN:Sea Haven Industries, CO ID:1811679038 CCD PMT INFO:Sea Haven Industries, Inc 21222000128", + amount: "2,100.00", + }); + assert.equal(c.event, "ach_return"); + assert.equal(c.pmtId, "7176466"); + assert.equal(c.embeddedPaymentNumber, "21222000128"); +}); + +test("classifier: short trailing digits are not an embedded payment number", () => { + const c = classifyTransaction({ + text: "SOME VENDOR DES:PAYMENTS ID:PMT 700001 INDN:Sea Haven Industries, CO ID:1811679038 CCD PMT INFO:Vendor Company 2000", + amount: "-100.00", + }); + assert.equal(c.embeddedPaymentNumber, null); + assert.equal(c.vendorText, "Vendor Company 2000"); +}); + +test("classifier: unknown code on a check-shaped transaction is 'unknown', never dropped", () => { + const c = classifyTransaction({ + transactionCode: "699", + customerReference: "3301", + text: "SOMETHING NEW", + amount: "-100.00", + }); + assert.equal(c.event, "unknown"); + assert.equal(c.checkShaped, true); + assert.equal(c.code, "699"); +}); + +test("classifier: transfers and misc bank activity are ignored", () => { + const c = classifyTransaction({ + text: "ACCOUNT TRANSFER TRSF FROM 483096772516", + amount: "15,000.00", + }); + assert.equal(c.event, "ignored"); + // Not check-shaped (no code, no reference, no CHECK text) -> ignored too. + const c2 = classifyTransaction({ text: "RETURN ITEM CHARGEBACK", amount: "-215" }); + assert.equal(c2.event, "ignored"); +}); + +test("amountsEqual: comma-tolerant, sign-insensitive, zero never matches", () => { + assert.equal(amountsEqual("1,892.62", -1892.62), true); + assert.equal(amountsEqual(0, 0), false); + assert.equal(amountsEqual("", ""), false); +}); + +// ---------------------------------------------------------------- matcher + +const payment = (over = {}) => ({ + pk: `payment#${over.check_number ?? "1001"}`, + check_number: "1001", + method: "Check", + amount_usd: 500, + status: "Outstanding", + send_payment_on: "07/01/2026", + ...over, +}); + +test("matcher: check number AND amount must both match", () => { + const payments = [payment({ check_number: "1001", amount_usd: 500 })]; + const m = matchCheckTransaction({ checkNumber: "1001", amount: 500 }, payments, "2026-07-20"); + assert.equal(m.payment, payments[0]); + assert.equal(m.matchedBy, "number+amount"); +}); + +test("matcher: number match with wrong amount NEVER falls through — human review", () => { + const payments = [ + payment({ check_number: "1001", amount_usd: 500 }), + payment({ check_number: "1002", amount_usd: 750.25, pk: "payment#1002" }), + ]; + // Altered-check / collapsed-posting signal: alert, no guessing. + const m = matchCheckTransaction({ checkNumber: "1001", amount: 750.25 }, payments, "2026-07-20"); + assert.equal(m.payment, undefined); + assert.equal(m.unmatched, "number matched, amount mismatch"); +}); + +test("matcher: digit-dropped number we never issued recovers via corroborated amount fallback", () => { + // 1222000012 is a digit-subsequence of issued 11222000012. + const payments = [payment({ check_number: "11222000012", amount_usd: 6413 })]; + const m = matchCheckTransaction({ checkNumber: "1222000012", amount: 6413 }, payments, "2026-07-20"); + assert.equal(m.payment.check_number, "11222000012"); + assert.equal(m.matchedBy, "amount"); +}); + +test("matcher: amount fallback without digit-subsequence corroboration is unmatched", () => { + // Same amount, but 9876 shares no digit-subsequence relation with 11222000012. + const payments = [payment({ check_number: "11222000012", amount_usd: 6413 })]; + const m = matchCheckTransaction({ checkNumber: "9876", amount: 6413 }, payments, "2026-07-20"); + assert.equal(m.payment, undefined); + assert.match(m.unmatched, /no corroborated amount fallback/); +}); + +test("matcher: check_return amount fallback requires a bank-confirmed candidate", () => { + const unconfirmed = [payment({ check_number: "11222000012", amount_usd: 6413 })]; + const r1 = matchCheckTransaction( + { event: "check_return", checkNumber: "1222000012", amount: 6413 }, + unconfirmed, + "2026-07-20" + ); + assert.equal(r1.payment, undefined); + + const confirmed = [ + payment({ check_number: "11222000012", amount_usd: 6413, clear_status: "Cleared" }), + ]; + const r2 = matchCheckTransaction( + { event: "check_return", checkNumber: "1222000012", amount: 6413 }, + confirmed, + "2026-07-20" + ); + assert.equal(r2.payment.check_number, "11222000012"); +}); + +test("digitsCorroborate: digit-subsequence in either direction, nothing else", () => { + assert.equal(digitsCorroborate("1222000012", "11222000012"), true); // dropped digit + assert.equal(digitsCorroborate("11222000012", "1222000012"), true); // symmetric + assert.equal(digitsCorroborate("9876", "11222000012"), false); + assert.equal(digitsCorroborate("", "1234"), false); +}); + +test("matcher: ambiguous amount fallback is unmatched — no write", () => { + // "101" digit-corroborates both 1001 and 1011; same amount -> ambiguous. + const payments = [ + payment({ check_number: "1001", amount_usd: 350 }), + payment({ check_number: "1011", amount_usd: 350, pk: "payment#1011" }), + ]; + const m = matchCheckTransaction({ checkNumber: "101", amount: 350 }, payments, "2026-07-20"); + assert.equal(m.payment, undefined); + assert.match(m.unmatched, /ambiguous/); +}); + +test("matcher: zero candidates is unmatched", () => { + const m = matchCheckTransaction({ checkNumber: "9999", amount: 123.45 }, [payment()], "2026-07-20"); + assert.equal(m.payment, undefined); +}); + +test("matcher: amount fallback only considers checks issued in the last 120 days", () => { + // Corroborated ("101" is a subsequence of "1001") but issued too long ago. + const payments = [ + payment({ check_number: "1001", amount_usd: 350, send_payment_on: "01/02/2026" }), + ]; + const m = matchCheckTransaction({ checkNumber: "101", amount: 350 }, payments, "2026-07-20"); + assert.equal(m.payment, undefined); +}); + +test("matcher: non-Check payments are never check-match candidates", () => { + const payments = [payment({ method: "ACH", check_number: "21222000264", amount_usd: 350 })]; + const m = matchCheckTransaction({ checkNumber: "21222000264", amount: 350 }, payments, "2026-07-20"); + assert.equal(m.payment, undefined); +}); + +test("electronic return: unique amount among bank-confirmed payments matches", () => { + const payments = [ + payment({ check_number: "1001", amount_usd: 3992.15, clear_status: "Cleared" }), + payment({ check_number: "1002", amount_usd: 3992.15, pk: "payment#1002" }), // not bank-confirmed + ]; + const m = matchElectronicReturn({ amount: 3992.15 }, payments, "2026-07-20"); + assert.equal(m.payment.check_number, "1001"); +}); + +test("electronic return: never matches ACH payments", () => { + const achCleared = payment({ + method: "ACH", + check_number: "21222000264", + amount_usd: 3992.15, + clear_status: "Cleared", + pk: "payment#21222000264", + }); + const checkCleared = payment({ + check_number: "1001", + amount_usd: 3992.15, + clear_status: "Cleared", + }); + // Only the Check should be a candidate — ACH is excluded. + const m = matchElectronicReturn({ amount: 3992.15 }, [achCleared, checkCleared], "2026-07-20"); + assert.equal(m.payment.check_number, "1001"); + assert.equal(m.matchedBy, "amount+cleared"); +}); + +test("electronic return: ambiguity is unmatched", () => { + const payments = [ + payment({ check_number: "1001", amount_usd: 2500, clear_status: "Cleared" }), + payment({ check_number: "1002", amount_usd: 2500, clear_status: "Cleared", pk: "payment#1002" }), + ]; + const m = matchElectronicReturn({ amount: 2500 }, payments, "2026-07-20"); + assert.equal(m.payment, undefined); + assert.match(m.unmatched, /ambiguous/); +}); + +// ------------------------------------------------------------ transitions + +const paidEvent = (over = {}) => ({ + event: "check_paid", + amount: 500, + bankReference: "813300001", + ...over, +}); +const returnEvent = (over = {}) => ({ + event: "check_return", + amount: 500, + bankReference: "813300002", + ...over, +}); + +test("transitions: paid debit sets BOTH status and clear_status to Cleared with dates", () => { + const p = payment(); + const r = applyEvent(p, paidEvent(), "2026-07-19"); + assert.equal(r.kind, "cleared"); + assert.equal(r.updates.status, "Cleared"); + assert.equal(r.updates.clear_status, "Cleared"); + assert.equal(r.updates.paid_date, "2026-07-19"); + assert.equal(r.updates.cleared_date, "2026-07-19"); + assert.equal(r.updates.bank_reference, "813300001"); + assert.deepEqual(r.historyEvent, { + event: "check_paid", + date: "2026-07-19", + bankRef: "813300001", + amount: 500, + }); +}); + +test("transitions: return applies EVEN IF currently Cleared; clear_status is bank truth", () => { + const p = payment({ status: "Cleared", clear_status: "Cleared" }); + const r = applyEvent(p, returnEvent(), "2026-07-20"); + assert.equal(r.kind, "returned"); + assert.equal(r.updates.clear_status, "Returned"); + assert.equal(r.updates.returned_date, "2026-07-20"); + // status has no "Returned" rung on the CSV ladder — re-written unchanged. + assert.equal(r.updates.status, "Cleared"); +}); + +test("transitions: second paid debit on a Returned check is a redeposit back to Cleared", () => { + const p = payment({ status: "Cleared", clear_status: "Returned" }); + const r = applyEvent(p, paidEvent({ bankReference: "813300003" }), "2026-07-21"); + assert.equal(r.kind, "redeposit"); + assert.equal(r.updates.status, "Cleared"); + assert.equal(r.updates.clear_status, "Cleared"); + assert.equal(r.updates.paid_date, "2026-07-21"); +}); + +test("transitions: paid -> returned -> redeposit sequence accumulates history", () => { + const p = payment(); + p.history = p.history || []; + const seq = [ + [paidEvent(), "2026-07-18"], + [returnEvent(), "2026-07-19"], + [paidEvent({ bankReference: "813300004" }), "2026-07-20"], + ]; + const kinds = []; + for (const [ev, date] of seq) { + const r = applyEvent(p, ev, date); + kinds.push(r.kind); + Object.assign(p, r.updates); + p.history.push(r.historyEvent); + } + assert.deepEqual(kinds, ["cleared", "returned", "redeposit"]); + assert.equal(p.clear_status, "Cleared"); + assert.equal(p.history.length, 3); + assert.deepEqual( + p.history.map((h) => h.event), + ["check_paid", "check_return", "check_paid"] + ); +}); + +test("transitions: return on a canceled record is terminal voided-and-bounced", () => { + const p = payment({ status: "Voided", clear_status: "Cleared" }); + const r = applyEvent(p, returnEvent(), "2026-07-20"); + assert.equal(r.kind, "voided_and_bounced"); + assert.equal(r.updates.clear_status, "Returned"); + assert.equal(r.updates.status, "Voided"); // cancel status stays visible +}); + +test("transitions: paid debit on a canceled record keeps the cancel status", () => { + const p = payment({ status: "Marked as Void" }); + const r = applyEvent(p, paidEvent(), "2026-07-19"); + assert.equal(r.kind, "cleared_on_canceled"); + assert.equal(r.updates.status, "Marked as Void"); + assert.equal(r.updates.clear_status, "Cleared"); +}); + +test("transitions: identical replayed event is a noop (idempotent replays)", () => { + const p = payment({ + status: "Cleared", + clear_status: "Cleared", + history: [{ event: "check_paid", date: "2026-07-19", bankRef: "813300001", amount: 500 }], + }); + const r = applyEvent(p, paidEvent(), "2026-07-19"); + assert.equal(r.kind, "noop"); + assert.equal(r.updates, null); +}); + +test("transitions: dedup identity is {event, date, amount} — bankRef differences are ignored", () => { + const p = payment({ + status: "Cleared", + clear_status: "Cleared", + history: [{ event: "check_paid", date: "2026-07-19", bankRef: "OLD-FORMAT-REF", amount: 500 }], + }); + // Same event replayed with a reformatted/omitted bankRef must still dedup. + const r = applyEvent(p, paidEvent({ bankReference: "813399999" }), "2026-07-19"); + assert.equal(r.kind, "noop"); +}); + +test("transitions: a paid event dated on/before the latest return is a replayed original, not a redeposit", () => { + const p = payment({ + status: "Cleared", + clear_status: "Returned", + history: [ + { event: "check_paid", date: "2026-07-18", bankRef: "a", amount: 500 }, + { event: "check_return", date: "2026-07-19", bankRef: "b", amount: 500 }, + ], + }); + // Replayed original paid (different bankRef, date <= return date): noop. + const equalDate = applyEvent(p, paidEvent({ bankReference: "c" }), "2026-07-19"); + assert.equal(equalDate.kind, "noop"); + const beforeDate = applyEvent(p, paidEvent({ bankReference: "c" }), "2026-07-17"); + assert.equal(beforeDate.kind, "noop"); + // Strictly after the return: genuine redeposit. + const after = applyEvent(p, paidEvent({ bankReference: "c" }), "2026-07-20"); + assert.equal(after.kind, "redeposit"); +}); + +test("transitions: every non-noop result sets both status and clear_status", () => { + const cases = [ + [payment(), paidEvent()], + [payment({ status: "Cleared", clear_status: "Cleared" }), returnEvent()], + [payment({ status: "Cleared", clear_status: "Returned" }), paidEvent()], + [payment({ status: "Voided" }), returnEvent()], + [payment({ clear_status: "Cleared" }), { event: "electronic_return", amount: 500, bankReference: "x" }], + ]; + for (const [p, ev] of cases) { + const r = applyEvent(p, ev, "2026-07-20"); + assert.notEqual(r.kind, "noop"); + assert.ok("status" in r.updates, `${r.kind} must set status`); + assert.ok("clear_status" in r.updates, `${r.kind} must set clear_status`); + } +}); + +test("isCancelStatus covers the Stampli cancel vocabulary", () => { + for (const s of ["Voided", "cancelled", "Canceled", "Marked as Void"]) { + assert.equal(isCancelStatus(s), true); + } + assert.equal(isCancelStatus("Cleared"), false); +}); + +// ------------------------------------------------------------ ACH (#69) + +const achPayment = (over = {}) => ({ + pk: `payment#${over.check_number ?? "21222000264"}`, + check_number: "21222000264", + method: "ACH", + payee: "Cobra Septic", + amount_usd: 8300, + status: "Payment Submitted", + send_payment_on: "06/02/2026", + ...over, +}); + +test("ach matcher: embedded payment number + amount wins", () => { + const payments = [achPayment(), achPayment({ check_number: "21222000265", pk: "payment#21222000265" })]; + const m = matchAchTransaction( + { event: "ach_debit", pmtId: "7584198", embeddedPaymentNumber: "21222000264", vendorText: "Cobra Septic", amount: 8300 }, + payments, + "2026-06-08" + ); + assert.equal(m.payment.check_number, "21222000264"); + assert.equal(m.matchedBy, "payment-number+amount"); +}); + +test("ach matcher: embedded number with wrong amount falls through to vendor+amount", () => { + const payments = [ + achPayment({ check_number: "21222000264", amount_usd: 100 }), + achPayment({ check_number: "21222000265", pk: "payment#21222000265", amount_usd: 8300 }), + ]; + const m = matchAchTransaction( + { event: "ach_debit", pmtId: "7584198", embeddedPaymentNumber: "21222000264", vendorText: "Cobra Septic", amount: 8300 }, + payments, + "2026-06-08" + ); + assert.equal(m.payment.check_number, "21222000265"); + assert.equal(m.matchedBy, "vendor+amount"); +}); + +test("ach matcher: stored pmt_id attributes a reversal credit", () => { + const payments = [ + achPayment({ pmt_id: "7584198", clear_status: "Cleared", status: "Cleared" }), + ]; + const m = matchAchTransaction( + { event: "ach_return", pmtId: "7584198", embeddedPaymentNumber: null, vendorText: "Sea Haven Industries, Inc", amount: 8300 }, + payments, + "2026-06-10" + ); + assert.equal(m.matchedBy, "pmt_id"); +}); + +test("ach matcher: vendor+amount only within send_payment_on -2..+14 days", () => { + // Reddi 21222000211: sent 5/27, debited 6/4 (8-day lag) must match... + const inWindow = [achPayment({ payee: "Reddi Services", send_payment_on: "05/27/2026", amount_usd: 1958 })]; + const m1 = matchAchTransaction( + { event: "ach_debit", pmtId: "1", embeddedPaymentNumber: null, vendorText: "Reddi Services", amount: 1958 }, + inWindow, + "2026-06-04" + ); + assert.equal(m1.matchedBy, "vendor+amount"); + // ...but a posting 30 days after the send date must not. + const m2 = matchAchTransaction( + { event: "ach_debit", pmtId: "1", embeddedPaymentNumber: null, vendorText: "Reddi Services", amount: 1958 }, + inWindow, + "2026-06-26" + ); + assert.equal(m2.payment, undefined); +}); + +test("ach matcher: PMT-id-less same-amount return credit matches a recently cleared ACH", () => { + const payments = [ + achPayment({ + payee: "Uline", + clear_status: "Cleared", + status: "Cleared", + cleared_date: "2026-05-22", + amount_usd: 19281.12, + }), + ]; + const m = matchAchTransaction( + { event: "ach_return", pmtId: null, embeddedPaymentNumber: null, vendorText: null, amount: 19281.12 }, + payments, + "2026-05-26" + ); + assert.equal(m.matchedBy, "amount+cleared"); +}); + +test("ach matcher: amount+cleared fallback requires cleared_date within 45 days before the credit", () => { + const base = { + payee: "Uline", + clear_status: "Cleared", + status: "Cleared", + amount_usd: 19281.12, + }; + const tooOld = [achPayment({ ...base, cleared_date: "2026-03-01" })]; + const m1 = matchAchTransaction( + { event: "ach_return", pmtId: null, embeddedPaymentNumber: null, vendorText: null, amount: 19281.12 }, + tooOld, + "2026-05-26" + ); + assert.equal(m1.payment, undefined); + + const noDate = [achPayment({ ...base })]; + const m2 = matchAchTransaction( + { event: "ach_return", pmtId: null, embeddedPaymentNumber: null, vendorText: null, amount: 19281.12 }, + noDate, + "2026-05-26" + ); + assert.equal(m2.payment, undefined); +}); + +test("ach matcher: return credit with an unattributable PMT id is unmatched, never amount-guessed", () => { + const payments = [ + achPayment({ clear_status: "Cleared", status: "Cleared", cleared_date: "2026-05-22", amount_usd: 8300 }), + ]; + const m = matchAchTransaction( + { event: "ach_return", pmtId: "999", embeddedPaymentNumber: null, vendorText: null, amount: 8300 }, + payments, + "2026-05-26" + ); + assert.equal(m.payment, undefined); + assert.equal(m.unmatched, "unknown PMT id"); +}); + +test("ach matcher: pmt_id match with amount mismatch is unmatched — partial-reversal human case", () => { + const payments = [achPayment({ pmt_id: "7584198", amount_usd: 8300 })]; + const m = matchAchTransaction( + { event: "ach_return", pmtId: "7584198", embeddedPaymentNumber: null, vendorText: null, amount: 4150 }, + payments, + "2026-06-10" + ); + assert.equal(m.payment, undefined); + assert.equal(m.unmatched, "pmt_id matched, amount mismatch"); +}); + +test("ach matcher: candidates with a DIFFERENT stored pmt_id are excluded from vendor+amount", () => { + const payments = [ + achPayment({ pmt_id: "1111111", amount_usd: 8300 }), + achPayment({ check_number: "21222000270", pk: "payment#21222000270", amount_usd: 8300 }), + ]; + // Both are Cobra Septic @ 8300 in-window; the pmt_id conflict on the + // first disambiguates to the second instead of going ambiguous. + const m = matchAchTransaction( + { event: "ach_debit", pmtId: "2222222", embeddedPaymentNumber: null, vendorText: "Cobra Septic", amount: 8300 }, + payments, + "2026-06-08" + ); + assert.equal(m.payment.check_number, "21222000270"); + assert.equal(m.matchedBy, "vendor+amount"); +}); + +test("ach matcher: ambiguity is unmatched — no write", () => { + const payments = [ + achPayment({ pk: "payment#a", check_number: "a", clear_status: "Cleared", cleared_date: "2026-06-01", amount_usd: 500 }), + achPayment({ pk: "payment#b", check_number: "b", clear_status: "Cleared", cleared_date: "2026-06-02", amount_usd: 500 }), + ]; + const m = matchAchTransaction( + { event: "ach_return", pmtId: null, embeddedPaymentNumber: null, vendorText: null, amount: 500 }, + payments, + "2026-06-08" + ); + assert.equal(m.payment, undefined); + assert.match(m.unmatched, /ambiguous/); +}); + +test("vendorMatches: truncated bank originator vs full payee, punctuation-insensitive", () => { + assert.equal(vendorMatches("Hugill's Septic Service LLC", "Hugill's Septic Service LLC"), true); + assert.equal(vendorMatches("Alliance Sanitation LLC", "ALLIANCE SANITAT"), true); + assert.equal(vendorMatches("Cobra Septic", "Reddi Services"), false); + assert.equal(vendorMatches("", "Cobra Septic"), false); +}); + +test("vendorMatches: prefix only counts at >= 10 normalized chars; short names need exact equality", () => { + // "ACME" (4 chars) must not claim "ACME Plumbing Co" by prefix... + assert.equal(vendorMatches("ACME Plumbing Co", "ACME"), false); + // ...but exact short-name equality still matches. + assert.equal(vendorMatches("ACME", "A.C.M.E."), true); + // 10+ char prefix still matches (truncated originators). + assert.equal(vendorMatches("Alliance Sanitation LLC", "ALLIANCESANITAT"), true); +}); + +test("ach transitions: settled debit clears both fields and persists pmt_id", () => { + const p = achPayment(); + const r = applyEvent( + { ...p }, + { event: "ach_debit", pmtId: "7584198", amount: 8300, bankReference: "905512345" }, + "2026-06-04" + ); + assert.equal(r.kind, "cleared"); + assert.equal(r.updates.status, "Cleared"); + assert.equal(r.updates.clear_status, "Cleared"); + assert.equal(r.updates.paid_date, "2026-06-04"); + assert.equal(r.updates.pmt_id, "7584198"); +}); + +test("ach transitions: return credit then re-debit (bounce and re-settle)", () => { + const p = achPayment({ status: "Cleared", clear_status: "Cleared", pmt_id: "7500000", history: [] }); + const ret = applyEvent(p, { event: "ach_return", pmtId: "7500000", amount: 8300, bankReference: "r1" }, "2026-05-26"); + assert.equal(ret.kind, "returned"); + assert.equal(ret.updates.clear_status, "Returned"); + assert.equal(ret.updates.returned_date, "2026-05-26"); + Object.assign(p, ret.updates); + p.history.push(ret.historyEvent); + + const redebit = applyEvent(p, { event: "ach_debit", pmtId: "7500000", amount: 8300, bankReference: "d2" }, "2026-06-01"); + assert.equal(redebit.kind, "redeposit"); + assert.equal(redebit.updates.status, "Cleared"); + assert.equal(redebit.updates.clear_status, "Cleared"); +}); + +test("ach transitions: electronic return on a voided-but-settled ACH is voided-and-bounced", () => { + const p = achPayment({ status: "Voided", clear_status: "Cleared" }); + const r = applyEvent(p, { event: "electronic_return", amount: 8300, bankReference: "r2" }, "2026-06-09"); + assert.equal(r.kind, "voided_and_bounced"); + assert.equal(r.updates.status, "Voided"); + assert.equal(r.updates.clear_status, "Returned"); +}); + +// -------------------------------------------------- staleness sweep (F6) + +test("stale sweep: unconfirmed ACH older than 16 days is listed; checks older than 60 days counted", () => { + const payments = [ + achPayment({ check_number: "21222000300", send_payment_on: "06/20/2026" }), // 31d, stale + achPayment({ check_number: "21222000301", pk: "payment#21222000301", send_payment_on: "07/10/2026" }), // 11d, fresh + achPayment({ check_number: "21222000302", pk: "payment#21222000302", send_payment_on: "06/01/2026", clear_status: "Cleared" }), // confirmed + achPayment({ check_number: "21222000303", pk: "payment#21222000303", send_payment_on: "06/01/2026", status: "Voided" }), // cancel-exempt + payment({ check_number: "3001", send_payment_on: "04/01/2026" }), // check, 111d, stale + payment({ check_number: "3002", pk: "payment#3002", send_payment_on: "07/01/2026" }), // check, fresh + ]; + const s = sweepStalePayments(payments, "2026-07-21"); + assert.equal(s.staleAchCount, 1); + assert.deepEqual(s.staleAch, [ + { check_number: "21222000300", amount: 8300, send_payment_on: "06/20/2026" }, + ]); + assert.equal(s.staleChecksCount, 1); +}); + +test("stale sweep: ACH list is capped, count is not", () => { + const payments = []; + for (let i = 0; i < STALE_LIST_CAP + 5; i++) { + payments.push( + achPayment({ check_number: `2122200${1000 + i}`, pk: `payment#s${i}`, send_payment_on: "06/01/2026" }) + ); + } + const s = sweepStalePayments(payments, "2026-07-21"); + assert.equal(s.staleAchCount, STALE_LIST_CAP + 5); + assert.equal(s.staleAch.length, STALE_LIST_CAP); +}); + +// ---------------------------------------------- conditioned writes (F7) + +test("buildEventUpdate: condition asserts the snapshot's status and clear_status", () => { + const p = payment({ status: "Outstanding", clear_status: "Cleared" }); + const applied = applyEvent(p, returnEvent(), "2026-07-20"); + const cmd = buildEventUpdate("Table", p, applied); + assert.equal(cmd.TableName, "Table"); + assert.deepEqual(cmd.Key, { pk: p.pk }); + assert.equal(cmd.ConditionExpression, "#c0 = :c0 AND #c1 = :c1"); + assert.equal(cmd.ExpressionAttributeNames["#c0"], "status"); + assert.equal(cmd.ExpressionAttributeNames["#c1"], "clear_status"); + assert.equal(cmd.ExpressionAttributeValues[":c0"], "Outstanding"); + assert.equal(cmd.ExpressionAttributeValues[":c1"], "Cleared"); + assert.match(cmd.UpdateExpression, /^SET #history = list_append/); + // Every applied field is present in the SET clause. + const setFields = Object.entries(cmd.ExpressionAttributeNames) + .filter(([k]) => k.startsWith("#f")) + .map(([, v]) => v); + assert.deepEqual(setFields.sort(), Object.keys(applied.updates).sort()); +}); + +test("buildEventUpdate: missing snapshot clear_status becomes attribute_not_exists", () => { + const p = payment(); // no clear_status yet + const applied = applyEvent(p, paidEvent(), "2026-07-19"); + const cmd = buildEventUpdate("Table", p, applied); + assert.equal(cmd.ConditionExpression, "#c0 = :c0 AND attribute_not_exists(#c1)"); + assert.equal(":c1" in cmd.ExpressionAttributeValues, false); +}); + +// ----------------------------------------------------- ReDoS hardening + +test("classifier: bounded regexes still parse zero-padded and normal check descriptions", () => { + const c = classifyTransaction({ text: "CHECK # 0003176", amount: "-350" }); + assert.equal(c.event, "check_paid"); + assert.equal(c.checkNumber, "3176"); +}); + +test("classifier: pathological long input is bounded, classified without hanging", () => { + const junk = `CHECK ${" ".repeat(2000)}${"9".repeat(2000)}`; + const start = Date.now(); + const c = classifyTransaction({ text: junk, customerReference: "42", amount: "-1" }); + assert.ok(Date.now() - start < 1000); + // Not parseable as a paid check -> unknown (check-shaped), never dropped. + assert.equal(c.event, "unknown"); + assert.ok(c.description.length <= 500); +}); + +// ------------------------------------------------------------- date range + +test("resolveDateRange: default is the trailing 3-day window (Mon covers Fri-Sun)", () => { + const r = resolveDateRange({}, new Date("2026-07-21T13:00:00Z")); + assert.deepEqual(r, { fromDate: "2026-07-18", toDate: "2026-07-20" }); +}); + +test("resolveDateRange: explicit valid range passes through", () => { + const r = resolveDateRange({ fromDate: "2026-06-16", toDate: "2026-06-23" }); + assert.deepEqual(r, { fromDate: "2026-06-16", toDate: "2026-06-23" }); +}); + +test("resolveDateRange: fromDate only replays a single day", () => { + const r = resolveDateRange({ fromDate: "2026-06-08" }); + assert.deepEqual(r, { fromDate: "2026-06-08", toDate: "2026-06-08" }); +}); + +test("resolveDateRange: malformed and impossible dates throw, naming the offending value", () => { + assert.throws(() => resolveDateRange({ fromDate: "06/08/2026" }), /06\/08\/2026/); + assert.throws(() => resolveDateRange({ fromDate: "2026-02-30" }), /2026-02-30/); + assert.throws( + () => resolveDateRange({ fromDate: "2026-07-02", toDate: "2026-07-01" }), + /fromDate="2026-07-02", toDate="2026-07-01"/ + ); + assert.throws(() => resolveDateRange({ fromDate: "2026-07-01; DROP", toDate: "2026-07-02" })); +}); + +test("isValidISODate rejects non-strings and bad calendar dates", () => { + assert.equal(isValidISODate("2026-07-20"), true); + assert.equal(isValidISODate("2026-13-01"), false); + assert.equal(isValidISODate(20260720), false); +});