diff --git a/README.md b/README.md index dbd18dd..7e6ffef 100644 --- a/README.md +++ b/README.md @@ -73,9 +73,11 @@ The scheduled Lambda reconciles the Previous Day feed onto `payment#` records (p **Classification.** Transactions are classified via a BAI transaction-code map (`BAI_CODE_EVENTS`; only 475 = check paid debit is empirically confirmed so far — the remaining codes are pending an enumeration replay over known event dates) with a fallback classifier over the statement description formats (ARP refer-to-maker, return-of-posted-check check-numbered and electronic variants, ACH CCD `DES:PAYMENTS ID:PMT` lines). Unmapped codes on check-shaped transactions are logged (`console.error`) and counted in the run summary — never silently dropped. -**Matching.** Check events match on check number AND amount, evaluating all candidates (bank postings can drop/collapse digits on long check numbers). A number match with the wrong amount — or an unknown number — falls back to an exact-amount match within checks issued in the last 120 days; zero or multiple fallback candidates means unmatched, and unmatched transactions are recorded in the run summary with no write. Electronic returns (no check number) match by exact amount among bank-confirmed payments. +**Matching.** Check events match on check number AND amount, evaluating all candidates (bank postings can drop/collapse digits on long check numbers). A number match with the WRONG amount never auto-resolves — it is the altered-check/collapsed-posting signal and goes to unmatched for human review. An unknown number falls back to an exact-amount match within checks issued in the last 120 days, and only when the posting's digits are a subsequence of the candidate's check number (or vice versa); return credits additionally require a bank-confirmed candidate. Zero or multiple fallback candidates means unmatched, recorded in the run summary with no write. Electronic returns (no check number) match by exact amount among bank-confirmed payments. -**ACH.** ACH is bank-confirmed too (#69): `processPaymentCsv` no longer auto-clears ACH on the send date (rows keep their Stampli status until the bank settles). ACH CCD lines match by the stored `pmt_id` first (reversals reuse the original `PMT `), then by the Stampli payment number embedded at the end of `PMT INFO` (internal spaces stripped, amount must agree), then by vendor + exact amount within `send_payment_on` −2..+14 days. A settled debit clears the record and persists `pmt_id`; a credit reusing the `pmt_id` (or a unique same-amount return credit against a bank-confirmed ACH) marks it Returned. +**ACH.** ACH is bank-confirmed too (#69): `processPaymentCsv` no longer auto-clears ACH on the send date (rows keep their Stampli status until the bank settles). ACH CCD lines match by the stored `pmt_id` first (reversals reuse the original `PMT `; a `pmt_id` match with the wrong amount is the partial-reversal human case and goes to unmatched), then by the Stampli payment number embedded at the end of `PMT INFO` (internal spaces stripped, amount must agree), then by vendor + exact amount within `send_payment_on` −2..+14 days (candidates whose stored `pmt_id` differs are excluded; vendor prefix matching requires ≥ 10 normalized chars). A settled debit clears the record and persists `pmt_id`. A return credit whose `PMT ` attributes to no stored `pmt_id` is an unmatched alert ("unknown PMT id"); credits without a `PMT ` may fall back to a unique same-amount match against a bank-confirmed ACH whose `cleared_date` is within the prior 45 days. + +**Staleness sweep.** Every run also flags never-bank-confirmed payments: ACH with no `clear_status` sent more than 16 days ago (listed in the summary, capped at 50, plus a total count) and checks issued more than 60 days ago with no `clear_status` (count only). Both alert via `console.error`. **State transitions.** Every write sets BOTH `status` and `clear_status` (`clear_status` is bank truth; "Cleared without a subsequent return is permanent" keys off it): @@ -88,9 +90,9 @@ The scheduled Lambda reconciles the Previous Day feed onto `payment#` records (p Each applied event is appended to a `history` list attribute (`{event, date, bankRef, amount}`); identical replayed events are idempotent noops. -**Replay.** The handler accepts an optional payload `{"fromDate": "YYYY-MM-DD", "toDate": "YYYY-MM-DD"}` (strictly validated; `toDate` defaults to `fromDate`; no payload = yesterday) for weekend/outage gap replays and BAI-code enumeration runs. +**Replay.** The handler accepts an optional payload `{"fromDate": "YYYY-MM-DD", "toDate": "YYYY-MM-DD"}` (strictly validated; `toDate` defaults to `fromDate`) for weekend/outage gap replays and BAI-code enumeration runs. With no payload it queries the trailing 3-day window (today−3 .. today−1), so the Monday run covers Friday–Sunday; overlapping days are idempotent (event identity `{event, date, amount}`). Feed rows without a valid `valueDate` are never applied with a substituted date — they go to unmatched for review. -**Run summary.** Each run writes a `boa_recon#` item (90-day TTL) with counts per classified event type, matched/applied/redeposit/voided-and-bounced totals, the unmatched check numbers and amounts, and any unknown BAI codes encountered. Unmatched and unknown-code transactions also `console.error` (Slack alerting is tracked in #71). +**Run summary.** Each run writes an append-only `boa_recon#_#` item (90-day TTL) with counts per classified event type, matched/applied/redeposit/voided-and-bounced/write-conflict totals, the unmatched check numbers and amounts (list capped at 50; full count kept), unknown BAI codes (capped at 20 distinct keys), and the staleness sweep results. Unmatched and unknown-code transactions also `console.error` (Slack alerting is tracked in #71). Payment writes are conditioned on the read snapshot's `status`/`clear_status` and retried once against a fresh read on conflict. ## Documentation diff --git a/src/boaRecon.js b/src/boaRecon.js index 9b7df43..72a7f54 100644 --- a/src/boaRecon.js +++ b/src/boaRecon.js @@ -67,11 +67,15 @@ const POSTED_RETURN_ELECTRONIC_RE = /^RETURN OF POSTED CHECK \/ ITEM \(RECEIVED ON \d{2}-\d{2}\)\s*ELECTRONIC TRANSACTION\b/i; const ACH_PMT_RE = /DES:PAYMENTS\s+ID:PMT\s*(\d+)/i; const PMT_INFO_RE = /PMT INFO:\s*(.*)$/i; -const CHECK_PAID_RE = /^CHECK\s*#?\s*(\d+)$/i; +const CHECK_PAID_RE = /^CHECK\s{0,10}#?\s{0,10}0*(\d{1,12})$/i; // Trailing digit run at the end of PMT INFO — Stampli recently started // embedding the payment number there, and the bank wraps it with arbitrary -// internal spaces ("21 222000108", "2122200 0256"). -const EMBEDDED_NUMBER_RE = /(\d[\d ]{6,}\d)\s*$/; +// internal spaces ("21 222000108", "2122200 0256"). Run length is bounded +// against pathological input. +const EMBEDDED_NUMBER_RE = /(\d[\d ]{6,40}\d)\s*$/; + +// Bound untrusted text before any regex work (ReDoS hardening). +const MAX_DESCRIPTION_LEN = 500; const stripLeadingZeros = (s) => String(s ?? "").replace(/^0+(?=\d)/, ""); @@ -84,7 +88,9 @@ export function classifyTransaction(txn) { const code = String(txn.transactionCode ?? "").trim(); const description = String( txn.text ?? txn.description ?? txn.transactionDescription ?? "" - ).trim(); + ) + .slice(0, MAX_DESCRIPTION_LEN) + .trim(); const customerReference = stripLeadingZeros(String(txn.customerReference ?? "").trim()); const bankReference = String(txn.bankReference ?? "").trim(); const amount = Math.abs(parseAmount(txn.amount)); @@ -165,16 +171,36 @@ const issuedWithinDays = (payment, refISO, days) => { return d !== null && d >= 0 && d <= days; }; +// True when a's digits appear, in order, inside b (digit-dropped mangling: +// posting 1222000012 came from issued 11222000012). +function isDigitSubsequence(a, b) { + if (a.length > b.length) return false; + let i = 0; + for (let j = 0; j < b.length && i < a.length; j++) { + if (a[i] === b[j]) i++; + } + return i === a.length; +} + +export const digitsCorroborate = (a, b) => + Boolean(a && b) && (isDigitSubsequence(a, b) || isDigitSubsequence(b, a)); + // Match a check event (paid debit or return credit) against payment records. // Match on check number AND amount, evaluating all candidates — never // first-match on number alone: bank postings drop/collapse digits on long // check numbers, so a posting under one number can belong to another check -// (or to a number we never issued). When the number matches but the amount -// does not — or the number is unknown — fall back to an exact-amount match -// within checks issued in the last 120 days. Anything but a single fallback -// candidate is unmatched: no write on ambiguity. +// (or to a number we never issued). +// +// A number match with the WRONG amount never falls through to the amount +// fallback: that pattern is the altered-check / collapsed-posting signal a +// human must review, so it goes to unmatched. The amount-only fallback +// (number unknown) is limited to checks issued in the last 120 days AND +// requires digit-subsequence corroboration between the posting's number and +// the candidate's; return credits additionally require a bank-confirmed +// candidate. Anything but a single fallback candidate is unmatched: no +// write on ambiguity. export function matchCheckTransaction(classified, payments, refDateISO) { - const { checkNumber, amount } = classified; + const { checkNumber, amount, event } = classified; const checks = payments.filter((p) => p.method === "Check" && p.check_number); const numberMatches = checkNumber @@ -183,19 +209,24 @@ export function matchCheckTransaction(classified, payments, refDateISO) { const exact = numberMatches.filter((p) => amountsEqual(p.amount_usd, amount)); if (exact.length === 1) return { payment: exact[0], matchedBy: "number+amount" }; if (exact.length > 1) return { unmatched: "multiple number+amount matches" }; + if (numberMatches.length) { + return { unmatched: "number matched, amount mismatch" }; + } const fallback = checks.filter( - (p) => amountsEqual(p.amount_usd, amount) && issuedWithinDays(p, refDateISO, 120) + (p) => + amountsEqual(p.amount_usd, amount) && + issuedWithinDays(p, refDateISO, 120) && + digitsCorroborate(checkNumber, p.check_number) && + (event !== "check_return" || + p.clear_status === "Cleared" || + p.clear_status === "Returned") ); if (fallback.length === 1) return { payment: fallback[0], matchedBy: "amount" }; if (fallback.length > 1) { return { unmatched: `amount fallback ambiguous (${fallback.length} candidates)` }; } - return { - unmatched: numberMatches.length - ? "number matched with wrong amount; no unique amount fallback" - : "no number match; no unique amount fallback", - }; + return { unmatched: "no number match; no corroborated amount fallback" }; } // Electronic return credits carry no check number at all. Match by exact @@ -223,10 +254,15 @@ const normalizeVendor = (s) => .toUpperCase() .replace(/[^A-Z0-9]/g, ""); +// Prefix matching only counts when the shorter normalized string is at +// least 10 chars; short names must match exactly ("ACME" must not claim +// "ACME Plumbing Co"). export function vendorMatches(payee, vendorText) { const a = normalizeVendor(payee); const b = normalizeVendor(vendorText); if (!a || !b) return false; + if (a === b) return true; + if (Math.min(a.length, b.length) < 10) return false; return a.startsWith(b) || b.startsWith(a); } @@ -238,21 +274,41 @@ const withinSendWindow = (payment, postingISO) => { return d !== null && d >= -2 && d <= 14; }; +// Candidate cleared_date must sit within `days` before the credit posting; +// records without a valid cleared_date are excluded. +const clearedWithinDaysBefore = (payment, postingISO, days) => { + if (!isValidISODate(payment.cleared_date)) return false; + const d = Math.round((Date.parse(postingISO) - Date.parse(payment.cleared_date)) / 86400000); + return d >= 0 && d <= days; +}; + // Match an ACH CCD debit or return credit (payments-dashboard#69). -// Order: stored pmt_id (reversals reuse the original PMT id), then the -// Stampli payment number embedded in PMT INFO (spaces stripped; amount must -// also agree — a mismatch falls through), then vendor + exact amount within -// the send window. Return credits additionally fall back to a unique -// same-amount match among bank-confirmed ACH payments, since reversal -// credits carry the originator's name, not the vendor's. Ambiguity is -// always unmatched — no write. +// Rungs, every one amount-corroborated: +// 1. stored pmt_id (reversals reuse the original PMT id); a pmt_id match +// with the wrong amount is the partial-reversal human case — unmatched. +// 2. Stampli payment number embedded in PMT INFO (spaces stripped); a +// wrong-amount embedded match falls through. +// 3. vendor + exact amount within the send window (candidates whose stored +// pmt_id differs from the transaction's are excluded). +// 4. return credits only: if the credit's PMT id attributes to no stored +// pmt_id, that is an unmatched alert ("unknown PMT id") — never an +// amount guess. Credits without a PMT id may fall back to a unique +// same-amount match among bank-confirmed ACH payments whose +// cleared_date is within 45 days before the credit. +// Ambiguity is always unmatched — no write. export function matchAchTransaction(classified, payments, postingISO) { const { event, pmtId, embeddedPaymentNumber, vendorText, amount } = classified; const achs = payments.filter((p) => p.method === "ACH"); if (pmtId) { const byPmtId = achs.filter((p) => p.pmt_id === pmtId); - if (byPmtId.length === 1) return { payment: byPmtId[0], matchedBy: "pmt_id" }; + if (byPmtId.length === 1) { + if (amountsEqual(byPmtId[0].amount_usd, amount)) { + return { payment: byPmtId[0], matchedBy: "pmt_id" }; + } + return { unmatched: "pmt_id matched, amount mismatch" }; + } + if (byPmtId.length > 1) return { unmatched: "multiple pmt_id matches" }; } if (embeddedPaymentNumber) { @@ -262,8 +318,11 @@ export function matchAchTransaction(classified, payments, postingISO) { if (byNumber.length === 1) return { payment: byNumber[0], matchedBy: "payment-number+amount" }; } + const pmtIdConflicts = (p) => Boolean(pmtId && p.pmt_id && p.pmt_id !== pmtId); + const byVendor = achs.filter( (p) => + !pmtIdConflicts(p) && amountsEqual(p.amount_usd, amount) && vendorMatches(p.payee, vendorText) && withinSendWindow(p, postingISO) @@ -274,10 +333,13 @@ export function matchAchTransaction(classified, payments, postingISO) { } if (event === "ach_return") { + if (pmtId) return { unmatched: "unknown PMT id" }; const byAmount = achs.filter( (p) => + !pmtIdConflicts(p) && (p.clear_status === "Cleared" || p.clear_status === "Returned") && - amountsEqual(p.amount_usd, amount) + amountsEqual(p.amount_usd, amount) && + clearedWithinDaysBefore(p, postingISO, 45) ); if (byAmount.length === 1) return { payment: byAmount[0], matchedBy: "amount+cleared" }; if (byAmount.length > 1) { @@ -291,6 +353,8 @@ export function matchAchTransaction(classified, payments, postingISO) { const CANCEL_STATUSES = ["voided", "cancelled", "canceled", "marked as void"]; export const isCancelStatus = (s) => CANCEL_STATUSES.includes(String(s ?? "").toLowerCase()); +const RETURN_EVENTS = new Set(["check_return", "electronic_return", "ach_return"]); + // Decide the DDB write for a matched bank event. Pure: returns the fields // to set plus the history entry; the handler turns it into an UpdateCommand // and mirrors the fields onto its in-memory copy. @@ -317,18 +381,30 @@ export function applyEvent(payment, classified, eventDateISO) { amount, }; - // Replay idempotence: an identical event already recorded in history is a - // noop, so re-running a date range never double-applies transitions. + // Replay idempotence: event identity is {event, date, amount} — bankRef + // is deliberately excluded because feeds omit/reformat it between runs. + // Assumption: the bank never posts two DISTINCT same-type events for the + // same payment on the same date with the same amount; an identical + // identity already in history is therefore the same event, and a noop. const alreadyApplied = (payment.history || []).some( (h) => h.event === historyEvent.event && h.date === historyEvent.date && - h.bankRef === historyEvent.bankRef && amountsEqual(h.amount, amount) ); if (alreadyApplied) return { kind: "noop", updates: null, historyEvent: null }; if (event === "check_paid" || event === "ach_debit") { + // A paid event dated on/before the latest known return is a replayed + // original paid debit, not a redeposit — never re-clear from it. + const latestReturnDate = (payment.history || []) + .filter((h) => RETURN_EVENTS.has(h.event) && typeof h.date === "string") + .map((h) => h.date) + .sort() + .pop(); + if (latestReturnDate && eventDateISO <= latestReturnDate) { + return { kind: "noop", updates: null, historyEvent: null }; + } const redeposit = payment.clear_status === "Returned"; const updates = { status: canceled ? payment.status : "Cleared", @@ -376,24 +452,99 @@ export function isValidISODate(s) { ); } -// Optional {fromDate, toDate} replay payload; the default stays yesterday, -// matching the previous-day inquiry cadence. Strings are validated strictly -// so a malformed payload fails loudly instead of querying a garbage range. +// Optional {fromDate, toDate} replay payload. The default is a trailing +// 3-day window (today-3 .. today-1): the previous-day feed only runs on +// weekdays, so the Monday run must cover Friday through Sunday. Overlapping +// days are safe — event identity makes replays idempotent. Strings are +// validated strictly so a malformed payload fails loudly instead of +// querying a garbage range. export function resolveDateRange(event, now = new Date()) { const hasFrom = event?.fromDate != null; const hasTo = event?.toDate != null; if (!hasFrom && !hasTo) { - const yesterday = new Date(now.getTime() - 24 * 60 * 60 * 1000); - const d = yesterday.toISOString().split("T")[0]; - return { fromDate: d, toDate: d }; + const day = 24 * 60 * 60 * 1000; + return { + fromDate: new Date(now.getTime() - 3 * day).toISOString().split("T")[0], + toDate: new Date(now.getTime() - day).toISOString().split("T")[0], + }; } const fromDate = hasFrom ? event.fromDate : event.toDate; const toDate = hasTo ? event.toDate : event.fromDate; if (!isValidISODate(fromDate) || !isValidISODate(toDate)) { - throw new Error("fromDate/toDate must be valid YYYY-MM-DD strings"); + throw new Error( + `fromDate/toDate must be valid YYYY-MM-DD strings: ` + + `fromDate=${logSafe(fromDate)}, toDate=${logSafe(toDate)}` + ); } if (fromDate > toDate) { - throw new Error("fromDate must be <= toDate"); + throw new Error( + `fromDate must be <= toDate: fromDate=${logSafe(fromDate)}, toDate=${logSafe(toDate)}` + ); } return { fromDate, toDate }; } + +// Staleness sweep (#66 review F6): flag records the bank has never +// confirmed. ACH with no clear_status and a send date older than 16 days +// (settlement lags at most ~8 business days) and checks older than 60 days +// are surfaced in the run summary; cancel-status records are exempt. +export const STALE_LIST_CAP = 50; +export function sweepStalePayments(payments, todayISO) { + const staleAch = []; + let staleAchCount = 0; + let staleChecksCount = 0; + for (const p of payments) { + if (p.clear_status || isCancelStatus(p.status)) continue; + const age = daysSinceIssue(p, todayISO); + if (age === null) continue; + if (p.method === "ACH" && age > 16) { + staleAchCount++; + if (staleAch.length < STALE_LIST_CAP) { + staleAch.push({ + check_number: p.check_number, + amount: p.amount_usd, + send_payment_on: p.send_payment_on, + }); + } + } else if (p.method === "Check" && age > 60) { + staleChecksCount++; + } + } + return { staleAch, staleAchCount, staleChecksCount }; +} + +// Conditioned write for an applied event (#66 review F7): the update only +// lands if the snapshot's status/clear_status are still current, so a +// concurrent CSV upsert can't be silently interleaved. The handler retries +// once against a fresh read on ConditionalCheckFailedException. +export function buildEventUpdate(tableName, payment, applied) { + const sets = ["#history = list_append(if_not_exists(#history, :empty), :hist)"]; + const names = { "#history": "history" }; + const values = { ":empty": [], ":hist": [applied.historyEvent] }; + Object.entries(applied.updates).forEach(([field, value], i) => { + names[`#f${i}`] = field; + values[`:v${i}`] = value; + sets.push(`#f${i} = :v${i}`); + }); + const conditions = []; + [ + ["status", payment.status], + ["clear_status", payment.clear_status], + ].forEach(([field, snapshot], i) => { + names[`#c${i}`] = field; + if (snapshot == null) { + conditions.push(`attribute_not_exists(#c${i})`); + } else { + values[`:c${i}`] = snapshot; + conditions.push(`#c${i} = :c${i}`); + } + }); + return { + TableName: tableName, + Key: { pk: payment.pk }, + UpdateExpression: `SET ${sets.join(", ")}`, + ConditionExpression: conditions.join(" AND "), + ExpressionAttributeNames: names, + ExpressionAttributeValues: values, + }; +} diff --git a/src/fetchBoaTransactions.js b/src/fetchBoaTransactions.js index 953c62a..45d2eec 100644 --- a/src/fetchBoaTransactions.js +++ b/src/fetchBoaTransactions.js @@ -1,8 +1,9 @@ import { DynamoDBClient } from "@aws-sdk/client-dynamodb"; -import { DynamoDBDocumentClient, PutCommand, ScanCommand, UpdateCommand } from "@aws-sdk/lib-dynamodb"; +import { DynamoDBDocumentClient, GetCommand, PutCommand, ScanCommand, UpdateCommand } from "@aws-sdk/lib-dynamodb"; import { SecretsManagerClient, GetSecretValueCommand } from "@aws-sdk/client-secrets-manager"; import { applyEvent, + buildEventUpdate, classifyTransaction, isValidISODate, logSafe, @@ -10,6 +11,7 @@ import { matchCheckTransaction, matchElectronicReturn, resolveDateRange, + sweepStalePayments, } from "./boaRecon.js"; const ddb = DynamoDBDocumentClient.from(new DynamoDBClient()); @@ -17,6 +19,9 @@ const secrets = new SecretsManagerClient(); const TABLE_NAME = process.env.TABLE_NAME; const BOA_BASE_URL = process.env.BOA_BASE_URL; +const UNMATCHED_LIST_CAP = 50; +const UNKNOWN_CODES_CAP = 20; + let cachedCreds; async function getReportingCreds() { if (cachedCreds) return cachedCreds; @@ -27,6 +32,16 @@ async function getReportingCreds() { return cachedCreds; } +// Parse a BoA response body without ever surfacing a body snippet — a +// malformed 200 must not leak account data into logs/errors. +async function parseJsonResponse(res, label) { + try { + return await res.json(); + } catch { + throw new Error(`BoA ${label} response not JSON: HTTP ${res.status}`); + } +} + async function getAccessToken(applicationID, clientId, clientSecret) { const res = await fetch(`${BOA_BASE_URL}/authn/v1/client-authentication`, { method: "POST", @@ -41,12 +56,15 @@ async function getAccessToken(applicationID, clientId, clientSecret) { throw new Error(`OAuth token exchange failed: HTTP ${res.status}`); } - const data = await res.json(); + const data = await parseJsonResponse(res, "authn"); return data.access_token; } +const directionRank = (d) => (d === "debit" ? 0 : d === "credit" ? 1 : 2); + export const handler = async (event) => { - // Optional replay payload {fromDate, toDate}; default is yesterday. + // Optional replay payload {fromDate, toDate}; default is the trailing + // 3-day window so the Monday run covers Friday through Sunday. const { fromDate, toDate } = resolveDateRange(event ?? {}); const { appId, clientId, token: clientSecret, accountNumber, bankId } = await getReportingCreds(); @@ -71,7 +89,7 @@ export const handler = async (event) => { throw new Error(`BoA API error: HTTP ${res.status}`); } - const data = await res.json(); + const data = await parseJsonResponse(res, "reporting"); // Response shape: { accountTransactions: [{ accountNumber, bankId, currency, transactions: [...] }] } const allTransactions = (data.accountTransactions || []).flatMap( @@ -79,9 +97,16 @@ export const handler = async (event) => { ); // Process in posting-date order so multi-day replays apply paid -> return - // -> redeposit sequences in the order the bank did. - allTransactions.sort((a, b) => - String(a.valueDate ?? "").localeCompare(String(b.valueDate ?? "")) + // -> redeposit sequences in the order the bank did; within a day, debits + // before credits (deterministic, and a same-day return follows its debit). + const classifiedTxns = allTransactions.map((txn) => ({ + txn, + classified: classifyTransaction(txn), + })); + classifiedTxns.sort( + (a, b) => + String(a.txn.valueDate ?? "").localeCompare(String(b.txn.valueDate ?? "")) || + directionRank(a.classified.direction) - directionRank(b.classified.direction) ); // Load all payment records (Check AND ACH — ACH is bank-confirmed here @@ -109,19 +134,44 @@ export const handler = async (event) => { already_applied: 0, redeposits: 0, voided_and_bounced: 0, + write_conflicts: 0, + unmatched_count: 0, unmatched: [], + unknown_count: 0, unknown_codes: {}, }; - for (const txn of allTransactions) { - const classified = classifyTransaction(txn); + const recordUnmatched = (classified, reason) => { + summary.unmatched_count++; + if (summary.unmatched.length < UNMATCHED_LIST_CAP) { + summary.unmatched.push({ + event: classified.event, + check_number: classified.checkNumber || classified.embeddedPaymentNumber, + amount: classified.amount, + reason, + }); + } + console.error( + `Unmatched ${classified.event}: check=${logSafe(classified.checkNumber || classified.embeddedPaymentNumber)}, ` + + `amount=${classified.amount}, reason=${reason} ` + + `(bankRef: ${logSafe(classified.bankReference)})` + ); + }; + + for (const { txn, classified } of classifiedTxns) { summary.classified[classified.event] = (summary.classified[classified.event] || 0) + 1; if (classified.event === "ignored") continue; if (classified.event === "unknown") { - summary.unknown_codes[classified.code || "?"] = - (summary.unknown_codes[classified.code || "?"] || 0) + 1; + summary.unknown_count++; + const codeKey = (classified.code || "?").slice(0, 16); + if ( + codeKey in summary.unknown_codes || + Object.keys(summary.unknown_codes).length < UNKNOWN_CODES_CAP + ) { + summary.unknown_codes[codeKey] = (summary.unknown_codes[codeKey] || 0) + 1; + } console.error( `Unknown check-shaped transaction: code=${logSafe(classified.code)}, ` + `description=${logSafe(classified.description)}, ` + @@ -130,7 +180,13 @@ export const handler = async (event) => { continue; } - const eventDate = isValidISODate(txn.valueDate) ? txn.valueDate : toDate; + // Never substitute a date: event identity and transition ordering both + // key on the posting date, so a feed row without one is a human case. + if (!isValidISODate(txn.valueDate)) { + recordUnmatched(classified, "missing valueDate"); + continue; + } + const eventDate = txn.valueDate; let match; if (classified.event === "ach_debit" || classified.event === "ach_return") { @@ -142,68 +198,106 @@ export const handler = async (event) => { } if (!match.payment) { - summary.unmatched.push({ - event: classified.event, - check_number: classified.checkNumber || classified.embeddedPaymentNumber, - amount: classified.amount, - reason: match.unmatched, - }); - console.error( - `Unmatched ${classified.event}: check=${logSafe(classified.checkNumber || classified.embeddedPaymentNumber)}, ` + - `amount=${classified.amount}, reason=${match.unmatched} ` + - `(bankRef: ${logSafe(classified.bankReference)})` - ); + recordUnmatched(classified, match.unmatched); continue; } + if (match.matchedBy === "vendor+amount") { + // Audit trail for the loosest ACH rung. + console.log( + `ACH vendor+amount match: payee=${logSafe(match.payment.payee)}, ` + + `bankVendor=${logSafe(classified.vendorText)}, pmt=${logSafe(classified.pmtId)}` + ); + } + summary.matched++; - const applied = applyEvent(match.payment, classified, eventDate); + const target = match.payment; + let applied = applyEvent(target, classified, eventDate); if (applied.kind === "noop") { summary.already_applied++; continue; } + + // Conditioned write: assert the snapshot's status/clear_status are + // still current; on conflict, re-read, re-derive, retry once. + let outcome = "conflict"; + for (let attempt = 0; attempt < 2; attempt++) { + try { + await ddb.send(new UpdateCommand(buildEventUpdate(TABLE_NAME, target, applied))); + outcome = "written"; + break; + } catch (err) { + if (err.name !== "ConditionalCheckFailedException") throw err; + if (attempt === 1) break; + const { Item: fresh } = await ddb.send( + new GetCommand({ TableName: TABLE_NAME, Key: { pk: target.pk } }) + ); + if (!fresh) break; + // Refresh the in-memory record in place (it is shared with the + // payments array) and re-derive the transition. + for (const k of Object.keys(target)) delete target[k]; + Object.assign(target, fresh); + applied = applyEvent(target, classified, eventDate); + if (applied.kind === "noop") { + outcome = "noop"; + break; + } + } + } + + if (outcome === "noop") { + summary.already_applied++; + continue; + } + if (outcome !== "written") { + summary.write_conflicts++; + console.error( + `Write conflict (gave up after retry): pk=${logSafe(target.pk)}, event=${classified.event}` + ); + continue; + } + + summary.applied++; if (applied.kind === "redeposit") summary.redeposits++; if (applied.kind === "voided_and_bounced") summary.voided_and_bounced++; - const sets = ["#history = list_append(if_not_exists(#history, :empty), :hist)"]; - const names = { "#history": "history" }; - const values = { ":empty": [], ":hist": [applied.historyEvent] }; - Object.entries(applied.updates).forEach(([field, value], i) => { - names[`#f${i}`] = field; - values[`:v${i}`] = value; - sets.push(`#f${i} = :v${i}`); - }); - - await ddb.send( - new UpdateCommand({ - TableName: TABLE_NAME, - Key: { pk: match.payment.pk }, - UpdateExpression: `SET ${sets.join(", ")}`, - ExpressionAttributeNames: names, - ExpressionAttributeValues: values, - }) - ); - summary.applied++; - // Mirror the write onto the in-memory copy so later transactions in the // same run (return after paid, redeposit after return) see current state. - Object.assign(match.payment, applied.updates); - match.payment.history = [...(match.payment.history || []), applied.historyEvent]; + Object.assign(target, applied.updates); + target.history = [...(target.history || []), applied.historyEvent]; console.log( - `${applied.kind}: check ${logSafe(match.payment.check_number)} via ${match.matchedBy} ` + + `${applied.kind}: check ${logSafe(target.check_number)} via ${match.matchedBy} ` + `(bankRef: ${logSafe(classified.bankReference)})` ); } - // Run summary record for auditing/alerting (Slack wiring is #71). - const runDate = fromDate === toDate ? fromDate : `${fromDate}_${toDate}`; + // Staleness sweep: payments the bank has never confirmed (#69). + const todayISO = new Date().toISOString().split("T")[0]; + const stale = sweepStalePayments(payments, todayISO); + if (stale.staleAchCount) { + console.error( + `Stale ACH (no bank settlement, sent > 16 days ago): ${stale.staleAchCount} records, ` + + `checks=${logSafe(stale.staleAch.map((s) => s.check_number).join(","))}` + ); + } + if (stale.staleChecksCount) { + console.error( + `Stale checks (no bank activity, issued > 60 days ago): ${stale.staleChecksCount} records` + ); + } + + // Run summary record for auditing/alerting (Slack wiring is #71). The pk + // is append-only (run timestamp suffix) so re-runs never overwrite a + // prior run's record. + const runAt = new Date().toISOString(); + const runKey = `boa_recon#${fromDate}_${toDate}#${runAt}`; await ddb.send( new PutCommand({ TableName: TABLE_NAME, Item: { - pk: `boa_recon#${runDate}`, - run_at: new Date().toISOString(), + pk: runKey, + run_at: runAt, from_date: fromDate, to_date: toDate, transactions_seen: summary.transactions_seen, @@ -213,19 +307,24 @@ export const handler = async (event) => { already_applied: summary.already_applied, redeposits: summary.redeposits, voided_and_bounced: summary.voided_and_bounced, - unmatched_count: summary.unmatched.length, + write_conflicts: summary.write_conflicts, + unmatched_count: summary.unmatched_count, unmatched: summary.unmatched, + unknown_count: summary.unknown_count, unknown_codes: summary.unknown_codes, + stale_ach: stale.staleAch, + stale_ach_count: stale.staleAchCount, + stale_checks_count: stale.staleChecksCount, ttl: Math.floor(Date.now() / 1000) + 90 * 24 * 60 * 60, }, }) ); - const unknownCount = Object.values(summary.unknown_codes).reduce((a, b) => a + b, 0); - if (summary.unmatched.length || unknownCount) { + if (summary.unmatched_count || summary.unknown_count || summary.write_conflicts) { console.error( - `Reconciliation ${fromDate}..${toDate}: ${summary.unmatched.length} unmatched, ` + - `${unknownCount} unknown-code transactions (see boa_recon#${runDate})` + `Reconciliation ${fromDate}..${toDate}: ${summary.unmatched_count} unmatched, ` + + `${summary.unknown_count} unknown-code transactions, ` + + `${summary.write_conflicts} write conflicts (see ${logSafe(runKey)})` ); } @@ -233,13 +332,13 @@ export const handler = async (event) => { `Processed ${summary.transactions_seen} transactions ${fromDate}..${toDate}: ` + `${summary.matched} matched, ${summary.applied} applied, ` + `${summary.already_applied} already applied, ${summary.redeposits} redeposits, ` + - `${summary.voided_and_bounced} voided-and-bounced, ${summary.unmatched.length} unmatched` + `${summary.voided_and_bounced} voided-and-bounced, ${summary.unmatched_count} unmatched` ); return { statusCode: 200, body: `${summary.matched} of ${summary.transactions_seen} transactions matched ` + - `(${summary.applied} applied, ${summary.unmatched.length} unmatched)`, + `(${summary.applied} applied, ${summary.unmatched_count} unmatched)`, }; }; diff --git a/tests/boaRecon.test.js b/tests/boaRecon.test.js index 26a07b3..67b9fba 100644 --- a/tests/boaRecon.test.js +++ b/tests/boaRecon.test.js @@ -2,15 +2,19 @@ import { test } from "node:test"; import assert from "node:assert/strict"; import { BAI_CODE_EVENTS, + STALE_LIST_CAP, amountsEqual, applyEvent, + buildEventUpdate, classifyTransaction, + digitsCorroborate, isCancelStatus, isValidISODate, matchAchTransaction, matchCheckTransaction, matchElectronicReturn, resolveDateRange, + sweepStalePayments, vendorMatches, } from "../src/boaRecon.js"; @@ -173,31 +177,67 @@ test("matcher: check number AND amount must both match", () => { assert.equal(m.matchedBy, "number+amount"); }); -test("matcher: number match with wrong amount falls back to unique exact-amount match", () => { +test("matcher: number match with wrong amount NEVER falls through — human review", () => { const payments = [ payment({ check_number: "1001", amount_usd: 500 }), payment({ check_number: "1002", amount_usd: 750.25, pk: "payment#1002" }), ]; - // Posting collapsed onto 1001's number but carries 1002's amount. + // Altered-check / collapsed-posting signal: alert, no guessing. const m = matchCheckTransaction({ checkNumber: "1001", amount: 750.25 }, payments, "2026-07-20"); - assert.equal(m.payment.check_number, "1002"); + assert.equal(m.payment, undefined); + assert.equal(m.unmatched, "number matched, amount mismatch"); +}); + +test("matcher: digit-dropped number we never issued recovers via corroborated amount fallback", () => { + // 1222000012 is a digit-subsequence of issued 11222000012. + const payments = [payment({ check_number: "11222000012", amount_usd: 6413 })]; + const m = matchCheckTransaction({ checkNumber: "1222000012", amount: 6413 }, payments, "2026-07-20"); + assert.equal(m.payment.check_number, "11222000012"); assert.equal(m.matchedBy, "amount"); }); -test("matcher: collapsed number we never issued still recovers by amount", () => { - const payments = [payment({ check_number: "11222000123", amount_usd: 6413 })]; - const m = matchCheckTransaction({ checkNumber: "1222000012", amount: 6413 }, payments, "2026-07-20"); - assert.equal(m.payment.check_number, "11222000123"); - assert.equal(m.matchedBy, "amount"); +test("matcher: amount fallback without digit-subsequence corroboration is unmatched", () => { + // Same amount, but 9876 shares no digit-subsequence relation with 11222000012. + const payments = [payment({ check_number: "11222000012", amount_usd: 6413 })]; + const m = matchCheckTransaction({ checkNumber: "9876", amount: 6413 }, payments, "2026-07-20"); + assert.equal(m.payment, undefined); + assert.match(m.unmatched, /no corroborated amount fallback/); +}); + +test("matcher: check_return amount fallback requires a bank-confirmed candidate", () => { + const unconfirmed = [payment({ check_number: "11222000012", amount_usd: 6413 })]; + const r1 = matchCheckTransaction( + { event: "check_return", checkNumber: "1222000012", amount: 6413 }, + unconfirmed, + "2026-07-20" + ); + assert.equal(r1.payment, undefined); + + const confirmed = [ + payment({ check_number: "11222000012", amount_usd: 6413, clear_status: "Cleared" }), + ]; + const r2 = matchCheckTransaction( + { event: "check_return", checkNumber: "1222000012", amount: 6413 }, + confirmed, + "2026-07-20" + ); + assert.equal(r2.payment.check_number, "11222000012"); +}); + +test("digitsCorroborate: digit-subsequence in either direction, nothing else", () => { + assert.equal(digitsCorroborate("1222000012", "11222000012"), true); // dropped digit + assert.equal(digitsCorroborate("11222000012", "1222000012"), true); // symmetric + assert.equal(digitsCorroborate("9876", "11222000012"), false); + assert.equal(digitsCorroborate("", "1234"), false); }); test("matcher: ambiguous amount fallback is unmatched — no write", () => { + // "101" digit-corroborates both 1001 and 1011; same amount -> ambiguous. const payments = [ - payment({ check_number: "1001", amount_usd: 500 }), - payment({ check_number: "1002", amount_usd: 350, pk: "payment#1002" }), - payment({ check_number: "1003", amount_usd: 350, pk: "payment#1003" }), + payment({ check_number: "1001", amount_usd: 350 }), + payment({ check_number: "1011", amount_usd: 350, pk: "payment#1011" }), ]; - const m = matchCheckTransaction({ checkNumber: "1001", amount: 350 }, payments, "2026-07-20"); + const m = matchCheckTransaction({ checkNumber: "101", amount: 350 }, payments, "2026-07-20"); assert.equal(m.payment, undefined); assert.match(m.unmatched, /ambiguous/); }); @@ -208,10 +248,11 @@ test("matcher: zero candidates is unmatched", () => { }); test("matcher: amount fallback only considers checks issued in the last 120 days", () => { + // Corroborated ("101" is a subsequence of "1001") but issued too long ago. const payments = [ payment({ check_number: "1001", amount_usd: 350, send_payment_on: "01/02/2026" }), ]; - const m = matchCheckTransaction({ checkNumber: "2001", amount: 350 }, payments, "2026-07-20"); + const m = matchCheckTransaction({ checkNumber: "101", amount: 350 }, payments, "2026-07-20"); assert.equal(m.payment, undefined); }); @@ -342,6 +383,36 @@ test("transitions: identical replayed event is a noop (idempotent replays)", () assert.equal(r.updates, null); }); +test("transitions: dedup identity is {event, date, amount} — bankRef differences are ignored", () => { + const p = payment({ + status: "Cleared", + clear_status: "Cleared", + history: [{ event: "check_paid", date: "2026-07-19", bankRef: "OLD-FORMAT-REF", amount: 500 }], + }); + // Same event replayed with a reformatted/omitted bankRef must still dedup. + const r = applyEvent(p, paidEvent({ bankReference: "813399999" }), "2026-07-19"); + assert.equal(r.kind, "noop"); +}); + +test("transitions: a paid event dated on/before the latest return is a replayed original, not a redeposit", () => { + const p = payment({ + status: "Cleared", + clear_status: "Returned", + history: [ + { event: "check_paid", date: "2026-07-18", bankRef: "a", amount: 500 }, + { event: "check_return", date: "2026-07-19", bankRef: "b", amount: 500 }, + ], + }); + // Replayed original paid (different bankRef, date <= return date): noop. + const equalDate = applyEvent(p, paidEvent({ bankReference: "c" }), "2026-07-19"); + assert.equal(equalDate.kind, "noop"); + const beforeDate = applyEvent(p, paidEvent({ bankReference: "c" }), "2026-07-17"); + assert.equal(beforeDate.kind, "noop"); + // Strictly after the return: genuine redeposit. + const after = applyEvent(p, paidEvent({ bankReference: "c" }), "2026-07-20"); + assert.equal(after.kind, "redeposit"); +}); + test("transitions: every non-noop result sets both status and clear_status", () => { const cases = [ [payment(), paidEvent()], @@ -433,26 +504,95 @@ test("ach matcher: vendor+amount only within send_payment_on -2..+14 days", () = assert.equal(m2.payment, undefined); }); -test("ach matcher: same-amount return credit after the debit matches a bank-confirmed ACH", () => { +test("ach matcher: PMT-id-less same-amount return credit matches a recently cleared ACH", () => { const payments = [ - achPayment({ payee: "Uline", clear_status: "Cleared", status: "Cleared", amount_usd: 19281.12 }), + achPayment({ + payee: "Uline", + clear_status: "Cleared", + status: "Cleared", + cleared_date: "2026-05-22", + amount_usd: 19281.12, + }), ]; - // Reversal credit: originator is SEA HAVEN INDUST, so vendor match fails. const m = matchAchTransaction( - { event: "ach_return", pmtId: "999", embeddedPaymentNumber: null, vendorText: "Sea Haven Industries, Inc", amount: 19281.12 }, + { event: "ach_return", pmtId: null, embeddedPaymentNumber: null, vendorText: null, amount: 19281.12 }, payments, "2026-05-26" ); assert.equal(m.matchedBy, "amount+cleared"); }); -test("ach matcher: ambiguity is unmatched — no write", () => { +test("ach matcher: amount+cleared fallback requires cleared_date within 45 days before the credit", () => { + const base = { + payee: "Uline", + clear_status: "Cleared", + status: "Cleared", + amount_usd: 19281.12, + }; + const tooOld = [achPayment({ ...base, cleared_date: "2026-03-01" })]; + const m1 = matchAchTransaction( + { event: "ach_return", pmtId: null, embeddedPaymentNumber: null, vendorText: null, amount: 19281.12 }, + tooOld, + "2026-05-26" + ); + assert.equal(m1.payment, undefined); + + const noDate = [achPayment({ ...base })]; + const m2 = matchAchTransaction( + { event: "ach_return", pmtId: null, embeddedPaymentNumber: null, vendorText: null, amount: 19281.12 }, + noDate, + "2026-05-26" + ); + assert.equal(m2.payment, undefined); +}); + +test("ach matcher: return credit with an unattributable PMT id is unmatched, never amount-guessed", () => { const payments = [ - achPayment({ pk: "payment#a", check_number: "a", clear_status: "Cleared", amount_usd: 500 }), - achPayment({ pk: "payment#b", check_number: "b", clear_status: "Cleared", amount_usd: 500 }), + achPayment({ clear_status: "Cleared", status: "Cleared", cleared_date: "2026-05-22", amount_usd: 8300 }), ]; const m = matchAchTransaction( - { event: "ach_return", pmtId: "999", embeddedPaymentNumber: null, vendorText: null, amount: 500 }, + { event: "ach_return", pmtId: "999", embeddedPaymentNumber: null, vendorText: null, amount: 8300 }, + payments, + "2026-05-26" + ); + assert.equal(m.payment, undefined); + assert.equal(m.unmatched, "unknown PMT id"); +}); + +test("ach matcher: pmt_id match with amount mismatch is unmatched — partial-reversal human case", () => { + const payments = [achPayment({ pmt_id: "7584198", amount_usd: 8300 })]; + const m = matchAchTransaction( + { event: "ach_return", pmtId: "7584198", embeddedPaymentNumber: null, vendorText: null, amount: 4150 }, + payments, + "2026-06-10" + ); + assert.equal(m.payment, undefined); + assert.equal(m.unmatched, "pmt_id matched, amount mismatch"); +}); + +test("ach matcher: candidates with a DIFFERENT stored pmt_id are excluded from vendor+amount", () => { + const payments = [ + achPayment({ pmt_id: "1111111", amount_usd: 8300 }), + achPayment({ check_number: "21222000270", pk: "payment#21222000270", amount_usd: 8300 }), + ]; + // Both are Cobra Septic @ 8300 in-window; the pmt_id conflict on the + // first disambiguates to the second instead of going ambiguous. + const m = matchAchTransaction( + { event: "ach_debit", pmtId: "2222222", embeddedPaymentNumber: null, vendorText: "Cobra Septic", amount: 8300 }, + payments, + "2026-06-08" + ); + assert.equal(m.payment.check_number, "21222000270"); + assert.equal(m.matchedBy, "vendor+amount"); +}); + +test("ach matcher: ambiguity is unmatched — no write", () => { + const payments = [ + achPayment({ pk: "payment#a", check_number: "a", clear_status: "Cleared", cleared_date: "2026-06-01", amount_usd: 500 }), + achPayment({ pk: "payment#b", check_number: "b", clear_status: "Cleared", cleared_date: "2026-06-02", amount_usd: 500 }), + ]; + const m = matchAchTransaction( + { event: "ach_return", pmtId: null, embeddedPaymentNumber: null, vendorText: null, amount: 500 }, payments, "2026-06-08" ); @@ -467,6 +607,15 @@ test("vendorMatches: truncated bank originator vs full payee, punctuation-insens assert.equal(vendorMatches("", "Cobra Septic"), false); }); +test("vendorMatches: prefix only counts at >= 10 normalized chars; short names need exact equality", () => { + // "ACME" (4 chars) must not claim "ACME Plumbing Co" by prefix... + assert.equal(vendorMatches("ACME Plumbing Co", "ACME"), false); + // ...but exact short-name equality still matches. + assert.equal(vendorMatches("ACME", "A.C.M.E."), true); + // 10+ char prefix still matches (truncated originators). + assert.equal(vendorMatches("Alliance Sanitation LLC", "ALLIANCESANITAT"), true); +}); + test("ach transitions: settled debit clears both fields and persists pmt_id", () => { const p = achPayment(); const r = applyEvent( @@ -504,11 +653,89 @@ test("ach transitions: electronic return on a voided-but-settled ACH is voided-a assert.equal(r.updates.clear_status, "Returned"); }); +// -------------------------------------------------- staleness sweep (F6) + +test("stale sweep: unconfirmed ACH older than 16 days is listed; checks older than 60 days counted", () => { + const payments = [ + achPayment({ check_number: "21222000300", send_payment_on: "06/20/2026" }), // 31d, stale + achPayment({ check_number: "21222000301", pk: "payment#21222000301", send_payment_on: "07/10/2026" }), // 11d, fresh + achPayment({ check_number: "21222000302", pk: "payment#21222000302", send_payment_on: "06/01/2026", clear_status: "Cleared" }), // confirmed + achPayment({ check_number: "21222000303", pk: "payment#21222000303", send_payment_on: "06/01/2026", status: "Voided" }), // cancel-exempt + payment({ check_number: "3001", send_payment_on: "04/01/2026" }), // check, 111d, stale + payment({ check_number: "3002", pk: "payment#3002", send_payment_on: "07/01/2026" }), // check, fresh + ]; + const s = sweepStalePayments(payments, "2026-07-21"); + assert.equal(s.staleAchCount, 1); + assert.deepEqual(s.staleAch, [ + { check_number: "21222000300", amount: 8300, send_payment_on: "06/20/2026" }, + ]); + assert.equal(s.staleChecksCount, 1); +}); + +test("stale sweep: ACH list is capped, count is not", () => { + const payments = []; + for (let i = 0; i < STALE_LIST_CAP + 5; i++) { + payments.push( + achPayment({ check_number: `2122200${1000 + i}`, pk: `payment#s${i}`, send_payment_on: "06/01/2026" }) + ); + } + const s = sweepStalePayments(payments, "2026-07-21"); + assert.equal(s.staleAchCount, STALE_LIST_CAP + 5); + assert.equal(s.staleAch.length, STALE_LIST_CAP); +}); + +// ---------------------------------------------- conditioned writes (F7) + +test("buildEventUpdate: condition asserts the snapshot's status and clear_status", () => { + const p = payment({ status: "Outstanding", clear_status: "Cleared" }); + const applied = applyEvent(p, returnEvent(), "2026-07-20"); + const cmd = buildEventUpdate("Table", p, applied); + assert.equal(cmd.TableName, "Table"); + assert.deepEqual(cmd.Key, { pk: p.pk }); + assert.equal(cmd.ConditionExpression, "#c0 = :c0 AND #c1 = :c1"); + assert.equal(cmd.ExpressionAttributeNames["#c0"], "status"); + assert.equal(cmd.ExpressionAttributeNames["#c1"], "clear_status"); + assert.equal(cmd.ExpressionAttributeValues[":c0"], "Outstanding"); + assert.equal(cmd.ExpressionAttributeValues[":c1"], "Cleared"); + assert.match(cmd.UpdateExpression, /^SET #history = list_append/); + // Every applied field is present in the SET clause. + const setFields = Object.entries(cmd.ExpressionAttributeNames) + .filter(([k]) => k.startsWith("#f")) + .map(([, v]) => v); + assert.deepEqual(setFields.sort(), Object.keys(applied.updates).sort()); +}); + +test("buildEventUpdate: missing snapshot clear_status becomes attribute_not_exists", () => { + const p = payment(); // no clear_status yet + const applied = applyEvent(p, paidEvent(), "2026-07-19"); + const cmd = buildEventUpdate("Table", p, applied); + assert.equal(cmd.ConditionExpression, "#c0 = :c0 AND attribute_not_exists(#c1)"); + assert.equal(":c1" in cmd.ExpressionAttributeValues, false); +}); + +// ----------------------------------------------------- ReDoS hardening + +test("classifier: bounded regexes still parse zero-padded and normal check descriptions", () => { + const c = classifyTransaction({ text: "CHECK # 0003176", amount: "-350" }); + assert.equal(c.event, "check_paid"); + assert.equal(c.checkNumber, "3176"); +}); + +test("classifier: pathological long input is bounded, classified without hanging", () => { + const junk = `CHECK ${" ".repeat(2000)}${"9".repeat(2000)}`; + const start = Date.now(); + const c = classifyTransaction({ text: junk, customerReference: "42", amount: "-1" }); + assert.ok(Date.now() - start < 1000); + // Not parseable as a paid check -> unknown (check-shaped), never dropped. + assert.equal(c.event, "unknown"); + assert.ok(c.description.length <= 500); +}); + // ------------------------------------------------------------- date range -test("resolveDateRange: default is yesterday", () => { +test("resolveDateRange: default is the trailing 3-day window (Mon covers Fri-Sun)", () => { const r = resolveDateRange({}, new Date("2026-07-21T13:00:00Z")); - assert.deepEqual(r, { fromDate: "2026-07-20", toDate: "2026-07-20" }); + assert.deepEqual(r, { fromDate: "2026-07-18", toDate: "2026-07-20" }); }); test("resolveDateRange: explicit valid range passes through", () => { @@ -521,10 +748,13 @@ test("resolveDateRange: fromDate only replays a single day", () => { assert.deepEqual(r, { fromDate: "2026-06-08", toDate: "2026-06-08" }); }); -test("resolveDateRange: malformed and impossible dates throw", () => { - assert.throws(() => resolveDateRange({ fromDate: "06/08/2026" })); - assert.throws(() => resolveDateRange({ fromDate: "2026-02-30" })); - assert.throws(() => resolveDateRange({ fromDate: "2026-07-02", toDate: "2026-07-01" })); +test("resolveDateRange: malformed and impossible dates throw, naming the offending value", () => { + assert.throws(() => resolveDateRange({ fromDate: "06/08/2026" }), /06\/08\/2026/); + assert.throws(() => resolveDateRange({ fromDate: "2026-02-30" }), /2026-02-30/); + assert.throws( + () => resolveDateRange({ fromDate: "2026-07-02", toDate: "2026-07-01" }), + /fromDate="2026-07-02", toDate="2026-07-01"/ + ); assert.throws(() => resolveDateRange({ fromDate: "2026-07-01; DROP", toDate: "2026-07-02" })); });