mirror of
https://github.com/Sea-Haven-Industries/payments-dashboard.git
synced 2026-09-30 04:13:12 +00:00
fix(fetchboa): close review findings — coverage gap, matcher corroboration, replay identity (#66)
Security-review gate (detector fan-out + verifier + GPT-4.1 cross-family)
blocked on F6 and confirmed six lower findings; all are closed here.
- F6 (HIGH): the default run now queries a trailing 3-day window
(today-3..today-1) so the Monday run covers Friday-Sunday postings the
previous-day cadence silently skipped. A per-run staleness sweep flags
never-bank-confirmed payments (ACH > 16 days, checks > 60 days) in the
summary and via console.error.
- F2: replay identity is {event, date, amount} — bankRef excluded (feeds
omit/reformat it between runs); a paid event dated on/before the
latest return in history is a replayed original, never a redeposit;
rows without a valid valueDate are routed to unmatched instead of
being applied under a substituted date; within a day, debits sort
before credits.
- F1: a check-number match with the wrong amount no longer falls
through to the amount fallback (altered-check/collapsed-posting is a
human-review case); the amount fallback requires digit-subsequence
corroboration between posting and candidate numbers; check_return
fallback requires a bank-confirmed candidate.
- F4: the ach_return amount fallback requires cleared_date within 45
days before the credit; an unattributable PMT id is an unmatched
alert, never an amount guess.
- F5: the pmt_id rung requires amount equality; mismatch is the
partial-reversal human case.
- F3: vendor prefix matching requires >= 10 normalized chars (short
names must match exactly); candidates with a conflicting stored
pmt_id are excluded; vendor+amount matches are audit-logged.
- F7: payment writes are conditioned on the snapshot's
status/clear_status and retried once against a fresh read; residual
conflicts are counted, not silently interleaved.
- F9/summary bounds: run summary pk is append-only
(boa_recon#<from>_<to>#<runAt>); unmatched list capped at 50, unknown
codes at 20 keys/16 chars, stale list at 50 (full counts kept).
- ReDoS: description bounded to 500 chars before classification;
CHECK/embedded-number regexes use bounded quantifiers.
- FBOA2-1: both BoA res.json() parses are wrapped so a malformed 200
throws a status-only error and cannot leak a body snippet.
Handler event contract extended (optional fromDate/toDate); cross-family
review required before merge.
This commit is contained in:
parent
a49b91a25f
commit
b0f72f2b8c
4 changed files with 606 additions and 124 deletions
10
README.md
10
README.md
|
|
@ -73,9 +73,11 @@ The scheduled Lambda reconciles the Previous Day feed onto `payment#` records (p
|
|||
|
||||
**Classification.** Transactions are classified via a BAI transaction-code map (`BAI_CODE_EVENTS`; only 475 = check paid debit is empirically confirmed so far — the remaining codes are pending an enumeration replay over known event dates) with a fallback classifier over the statement description formats (ARP refer-to-maker, return-of-posted-check check-numbered and electronic variants, ACH CCD `DES:PAYMENTS ID:PMT` lines). Unmapped codes on check-shaped transactions are logged (`console.error`) and counted in the run summary — never silently dropped.
|
||||
|
||||
**Matching.** Check events match on check number AND amount, evaluating all candidates (bank postings can drop/collapse digits on long check numbers). A number match with the wrong amount — or an unknown number — falls back to an exact-amount match within checks issued in the last 120 days; zero or multiple fallback candidates means unmatched, and unmatched transactions are recorded in the run summary with no write. Electronic returns (no check number) match by exact amount among bank-confirmed payments.
|
||||
**Matching.** Check events match on check number AND amount, evaluating all candidates (bank postings can drop/collapse digits on long check numbers). A number match with the WRONG amount never auto-resolves — it is the altered-check/collapsed-posting signal and goes to unmatched for human review. An unknown number falls back to an exact-amount match within checks issued in the last 120 days, and only when the posting's digits are a subsequence of the candidate's check number (or vice versa); return credits additionally require a bank-confirmed candidate. Zero or multiple fallback candidates means unmatched, recorded in the run summary with no write. Electronic returns (no check number) match by exact amount among bank-confirmed payments.
|
||||
|
||||
**ACH.** ACH is bank-confirmed too (#69): `processPaymentCsv` no longer auto-clears ACH on the send date (rows keep their Stampli status until the bank settles). ACH CCD lines match by the stored `pmt_id` first (reversals reuse the original `PMT <id>`), then by the Stampli payment number embedded at the end of `PMT INFO` (internal spaces stripped, amount must agree), then by vendor + exact amount within `send_payment_on` −2..+14 days. A settled debit clears the record and persists `pmt_id`; a credit reusing the `pmt_id` (or a unique same-amount return credit against a bank-confirmed ACH) marks it Returned.
|
||||
**ACH.** ACH is bank-confirmed too (#69): `processPaymentCsv` no longer auto-clears ACH on the send date (rows keep their Stampli status until the bank settles). ACH CCD lines match by the stored `pmt_id` first (reversals reuse the original `PMT <id>`; a `pmt_id` match with the wrong amount is the partial-reversal human case and goes to unmatched), then by the Stampli payment number embedded at the end of `PMT INFO` (internal spaces stripped, amount must agree), then by vendor + exact amount within `send_payment_on` −2..+14 days (candidates whose stored `pmt_id` differs are excluded; vendor prefix matching requires ≥ 10 normalized chars). A settled debit clears the record and persists `pmt_id`. A return credit whose `PMT <id>` attributes to no stored `pmt_id` is an unmatched alert ("unknown PMT id"); credits without a `PMT <id>` may fall back to a unique same-amount match against a bank-confirmed ACH whose `cleared_date` is within the prior 45 days.
|
||||
|
||||
**Staleness sweep.** Every run also flags never-bank-confirmed payments: ACH with no `clear_status` sent more than 16 days ago (listed in the summary, capped at 50, plus a total count) and checks issued more than 60 days ago with no `clear_status` (count only). Both alert via `console.error`.
|
||||
|
||||
**State transitions.** Every write sets BOTH `status` and `clear_status` (`clear_status` is bank truth; "Cleared without a subsequent return is permanent" keys off it):
|
||||
|
||||
|
|
@ -88,9 +90,9 @@ The scheduled Lambda reconciles the Previous Day feed onto `payment#` records (p
|
|||
|
||||
Each applied event is appended to a `history` list attribute (`{event, date, bankRef, amount}`); identical replayed events are idempotent noops.
|
||||
|
||||
**Replay.** The handler accepts an optional payload `{"fromDate": "YYYY-MM-DD", "toDate": "YYYY-MM-DD"}` (strictly validated; `toDate` defaults to `fromDate`; no payload = yesterday) for weekend/outage gap replays and BAI-code enumeration runs.
|
||||
**Replay.** The handler accepts an optional payload `{"fromDate": "YYYY-MM-DD", "toDate": "YYYY-MM-DD"}` (strictly validated; `toDate` defaults to `fromDate`) for weekend/outage gap replays and BAI-code enumeration runs. With no payload it queries the trailing 3-day window (today−3 .. today−1), so the Monday run covers Friday–Sunday; overlapping days are idempotent (event identity `{event, date, amount}`). Feed rows without a valid `valueDate` are never applied with a substituted date — they go to unmatched for review.
|
||||
|
||||
**Run summary.** Each run writes a `boa_recon#<runDate>` item (90-day TTL) with counts per classified event type, matched/applied/redeposit/voided-and-bounced totals, the unmatched check numbers and amounts, and any unknown BAI codes encountered. Unmatched and unknown-code transactions also `console.error` (Slack alerting is tracked in #71).
|
||||
**Run summary.** Each run writes an append-only `boa_recon#<fromDate>_<toDate>#<runAt>` item (90-day TTL) with counts per classified event type, matched/applied/redeposit/voided-and-bounced/write-conflict totals, the unmatched check numbers and amounts (list capped at 50; full count kept), unknown BAI codes (capped at 20 distinct keys), and the staleness sweep results. Unmatched and unknown-code transactions also `console.error` (Slack alerting is tracked in #71). Payment writes are conditioned on the read snapshot's `status`/`clear_status` and retried once against a fresh read on conflict.
|
||||
|
||||
## Documentation
|
||||
|
||||
|
|
|
|||
221
src/boaRecon.js
221
src/boaRecon.js
|
|
@ -67,11 +67,15 @@ const POSTED_RETURN_ELECTRONIC_RE =
|
|||
/^RETURN OF POSTED CHECK \/ ITEM \(RECEIVED ON \d{2}-\d{2}\)\s*ELECTRONIC TRANSACTION\b/i;
|
||||
const ACH_PMT_RE = /DES:PAYMENTS\s+ID:PMT\s*(\d+)/i;
|
||||
const PMT_INFO_RE = /PMT INFO:\s*(.*)$/i;
|
||||
const CHECK_PAID_RE = /^CHECK\s*#?\s*(\d+)$/i;
|
||||
const CHECK_PAID_RE = /^CHECK\s{0,10}#?\s{0,10}0*(\d{1,12})$/i;
|
||||
// Trailing digit run at the end of PMT INFO — Stampli recently started
|
||||
// embedding the payment number there, and the bank wraps it with arbitrary
|
||||
// internal spaces ("21 222000108", "2122200 0256").
|
||||
const EMBEDDED_NUMBER_RE = /(\d[\d ]{6,}\d)\s*$/;
|
||||
// internal spaces ("21 222000108", "2122200 0256"). Run length is bounded
|
||||
// against pathological input.
|
||||
const EMBEDDED_NUMBER_RE = /(\d[\d ]{6,40}\d)\s*$/;
|
||||
|
||||
// Bound untrusted text before any regex work (ReDoS hardening).
|
||||
const MAX_DESCRIPTION_LEN = 500;
|
||||
|
||||
const stripLeadingZeros = (s) => String(s ?? "").replace(/^0+(?=\d)/, "");
|
||||
|
||||
|
|
@ -84,7 +88,9 @@ export function classifyTransaction(txn) {
|
|||
const code = String(txn.transactionCode ?? "").trim();
|
||||
const description = String(
|
||||
txn.text ?? txn.description ?? txn.transactionDescription ?? ""
|
||||
).trim();
|
||||
)
|
||||
.slice(0, MAX_DESCRIPTION_LEN)
|
||||
.trim();
|
||||
const customerReference = stripLeadingZeros(String(txn.customerReference ?? "").trim());
|
||||
const bankReference = String(txn.bankReference ?? "").trim();
|
||||
const amount = Math.abs(parseAmount(txn.amount));
|
||||
|
|
@ -165,16 +171,36 @@ const issuedWithinDays = (payment, refISO, days) => {
|
|||
return d !== null && d >= 0 && d <= days;
|
||||
};
|
||||
|
||||
// True when a's digits appear, in order, inside b (digit-dropped mangling:
|
||||
// posting 1222000012 came from issued 11222000012).
|
||||
function isDigitSubsequence(a, b) {
|
||||
if (a.length > b.length) return false;
|
||||
let i = 0;
|
||||
for (let j = 0; j < b.length && i < a.length; j++) {
|
||||
if (a[i] === b[j]) i++;
|
||||
}
|
||||
return i === a.length;
|
||||
}
|
||||
|
||||
export const digitsCorroborate = (a, b) =>
|
||||
Boolean(a && b) && (isDigitSubsequence(a, b) || isDigitSubsequence(b, a));
|
||||
|
||||
// Match a check event (paid debit or return credit) against payment records.
|
||||
// Match on check number AND amount, evaluating all candidates — never
|
||||
// first-match on number alone: bank postings drop/collapse digits on long
|
||||
// check numbers, so a posting under one number can belong to another check
|
||||
// (or to a number we never issued). When the number matches but the amount
|
||||
// does not — or the number is unknown — fall back to an exact-amount match
|
||||
// within checks issued in the last 120 days. Anything but a single fallback
|
||||
// candidate is unmatched: no write on ambiguity.
|
||||
// (or to a number we never issued).
|
||||
//
|
||||
// A number match with the WRONG amount never falls through to the amount
|
||||
// fallback: that pattern is the altered-check / collapsed-posting signal a
|
||||
// human must review, so it goes to unmatched. The amount-only fallback
|
||||
// (number unknown) is limited to checks issued in the last 120 days AND
|
||||
// requires digit-subsequence corroboration between the posting's number and
|
||||
// the candidate's; return credits additionally require a bank-confirmed
|
||||
// candidate. Anything but a single fallback candidate is unmatched: no
|
||||
// write on ambiguity.
|
||||
export function matchCheckTransaction(classified, payments, refDateISO) {
|
||||
const { checkNumber, amount } = classified;
|
||||
const { checkNumber, amount, event } = classified;
|
||||
const checks = payments.filter((p) => p.method === "Check" && p.check_number);
|
||||
|
||||
const numberMatches = checkNumber
|
||||
|
|
@ -183,19 +209,24 @@ export function matchCheckTransaction(classified, payments, refDateISO) {
|
|||
const exact = numberMatches.filter((p) => amountsEqual(p.amount_usd, amount));
|
||||
if (exact.length === 1) return { payment: exact[0], matchedBy: "number+amount" };
|
||||
if (exact.length > 1) return { unmatched: "multiple number+amount matches" };
|
||||
if (numberMatches.length) {
|
||||
return { unmatched: "number matched, amount mismatch" };
|
||||
}
|
||||
|
||||
const fallback = checks.filter(
|
||||
(p) => amountsEqual(p.amount_usd, amount) && issuedWithinDays(p, refDateISO, 120)
|
||||
(p) =>
|
||||
amountsEqual(p.amount_usd, amount) &&
|
||||
issuedWithinDays(p, refDateISO, 120) &&
|
||||
digitsCorroborate(checkNumber, p.check_number) &&
|
||||
(event !== "check_return" ||
|
||||
p.clear_status === "Cleared" ||
|
||||
p.clear_status === "Returned")
|
||||
);
|
||||
if (fallback.length === 1) return { payment: fallback[0], matchedBy: "amount" };
|
||||
if (fallback.length > 1) {
|
||||
return { unmatched: `amount fallback ambiguous (${fallback.length} candidates)` };
|
||||
}
|
||||
return {
|
||||
unmatched: numberMatches.length
|
||||
? "number matched with wrong amount; no unique amount fallback"
|
||||
: "no number match; no unique amount fallback",
|
||||
};
|
||||
return { unmatched: "no number match; no corroborated amount fallback" };
|
||||
}
|
||||
|
||||
// Electronic return credits carry no check number at all. Match by exact
|
||||
|
|
@ -223,10 +254,15 @@ const normalizeVendor = (s) =>
|
|||
.toUpperCase()
|
||||
.replace(/[^A-Z0-9]/g, "");
|
||||
|
||||
// Prefix matching only counts when the shorter normalized string is at
|
||||
// least 10 chars; short names must match exactly ("ACME" must not claim
|
||||
// "ACME Plumbing Co").
|
||||
export function vendorMatches(payee, vendorText) {
|
||||
const a = normalizeVendor(payee);
|
||||
const b = normalizeVendor(vendorText);
|
||||
if (!a || !b) return false;
|
||||
if (a === b) return true;
|
||||
if (Math.min(a.length, b.length) < 10) return false;
|
||||
return a.startsWith(b) || b.startsWith(a);
|
||||
}
|
||||
|
||||
|
|
@ -238,21 +274,41 @@ const withinSendWindow = (payment, postingISO) => {
|
|||
return d !== null && d >= -2 && d <= 14;
|
||||
};
|
||||
|
||||
// Candidate cleared_date must sit within `days` before the credit posting;
|
||||
// records without a valid cleared_date are excluded.
|
||||
const clearedWithinDaysBefore = (payment, postingISO, days) => {
|
||||
if (!isValidISODate(payment.cleared_date)) return false;
|
||||
const d = Math.round((Date.parse(postingISO) - Date.parse(payment.cleared_date)) / 86400000);
|
||||
return d >= 0 && d <= days;
|
||||
};
|
||||
|
||||
// Match an ACH CCD debit or return credit (payments-dashboard#69).
|
||||
// Order: stored pmt_id (reversals reuse the original PMT id), then the
|
||||
// Stampli payment number embedded in PMT INFO (spaces stripped; amount must
|
||||
// also agree — a mismatch falls through), then vendor + exact amount within
|
||||
// the send window. Return credits additionally fall back to a unique
|
||||
// same-amount match among bank-confirmed ACH payments, since reversal
|
||||
// credits carry the originator's name, not the vendor's. Ambiguity is
|
||||
// always unmatched — no write.
|
||||
// Rungs, every one amount-corroborated:
|
||||
// 1. stored pmt_id (reversals reuse the original PMT id); a pmt_id match
|
||||
// with the wrong amount is the partial-reversal human case — unmatched.
|
||||
// 2. Stampli payment number embedded in PMT INFO (spaces stripped); a
|
||||
// wrong-amount embedded match falls through.
|
||||
// 3. vendor + exact amount within the send window (candidates whose stored
|
||||
// pmt_id differs from the transaction's are excluded).
|
||||
// 4. return credits only: if the credit's PMT id attributes to no stored
|
||||
// pmt_id, that is an unmatched alert ("unknown PMT id") — never an
|
||||
// amount guess. Credits without a PMT id may fall back to a unique
|
||||
// same-amount match among bank-confirmed ACH payments whose
|
||||
// cleared_date is within 45 days before the credit.
|
||||
// Ambiguity is always unmatched — no write.
|
||||
export function matchAchTransaction(classified, payments, postingISO) {
|
||||
const { event, pmtId, embeddedPaymentNumber, vendorText, amount } = classified;
|
||||
const achs = payments.filter((p) => p.method === "ACH");
|
||||
|
||||
if (pmtId) {
|
||||
const byPmtId = achs.filter((p) => p.pmt_id === pmtId);
|
||||
if (byPmtId.length === 1) return { payment: byPmtId[0], matchedBy: "pmt_id" };
|
||||
if (byPmtId.length === 1) {
|
||||
if (amountsEqual(byPmtId[0].amount_usd, amount)) {
|
||||
return { payment: byPmtId[0], matchedBy: "pmt_id" };
|
||||
}
|
||||
return { unmatched: "pmt_id matched, amount mismatch" };
|
||||
}
|
||||
if (byPmtId.length > 1) return { unmatched: "multiple pmt_id matches" };
|
||||
}
|
||||
|
||||
if (embeddedPaymentNumber) {
|
||||
|
|
@ -262,8 +318,11 @@ export function matchAchTransaction(classified, payments, postingISO) {
|
|||
if (byNumber.length === 1) return { payment: byNumber[0], matchedBy: "payment-number+amount" };
|
||||
}
|
||||
|
||||
const pmtIdConflicts = (p) => Boolean(pmtId && p.pmt_id && p.pmt_id !== pmtId);
|
||||
|
||||
const byVendor = achs.filter(
|
||||
(p) =>
|
||||
!pmtIdConflicts(p) &&
|
||||
amountsEqual(p.amount_usd, amount) &&
|
||||
vendorMatches(p.payee, vendorText) &&
|
||||
withinSendWindow(p, postingISO)
|
||||
|
|
@ -274,10 +333,13 @@ export function matchAchTransaction(classified, payments, postingISO) {
|
|||
}
|
||||
|
||||
if (event === "ach_return") {
|
||||
if (pmtId) return { unmatched: "unknown PMT id" };
|
||||
const byAmount = achs.filter(
|
||||
(p) =>
|
||||
!pmtIdConflicts(p) &&
|
||||
(p.clear_status === "Cleared" || p.clear_status === "Returned") &&
|
||||
amountsEqual(p.amount_usd, amount)
|
||||
amountsEqual(p.amount_usd, amount) &&
|
||||
clearedWithinDaysBefore(p, postingISO, 45)
|
||||
);
|
||||
if (byAmount.length === 1) return { payment: byAmount[0], matchedBy: "amount+cleared" };
|
||||
if (byAmount.length > 1) {
|
||||
|
|
@ -291,6 +353,8 @@ export function matchAchTransaction(classified, payments, postingISO) {
|
|||
const CANCEL_STATUSES = ["voided", "cancelled", "canceled", "marked as void"];
|
||||
export const isCancelStatus = (s) => CANCEL_STATUSES.includes(String(s ?? "").toLowerCase());
|
||||
|
||||
const RETURN_EVENTS = new Set(["check_return", "electronic_return", "ach_return"]);
|
||||
|
||||
// Decide the DDB write for a matched bank event. Pure: returns the fields
|
||||
// to set plus the history entry; the handler turns it into an UpdateCommand
|
||||
// and mirrors the fields onto its in-memory copy.
|
||||
|
|
@ -317,18 +381,30 @@ export function applyEvent(payment, classified, eventDateISO) {
|
|||
amount,
|
||||
};
|
||||
|
||||
// Replay idempotence: an identical event already recorded in history is a
|
||||
// noop, so re-running a date range never double-applies transitions.
|
||||
// Replay idempotence: event identity is {event, date, amount} — bankRef
|
||||
// is deliberately excluded because feeds omit/reformat it between runs.
|
||||
// Assumption: the bank never posts two DISTINCT same-type events for the
|
||||
// same payment on the same date with the same amount; an identical
|
||||
// identity already in history is therefore the same event, and a noop.
|
||||
const alreadyApplied = (payment.history || []).some(
|
||||
(h) =>
|
||||
h.event === historyEvent.event &&
|
||||
h.date === historyEvent.date &&
|
||||
h.bankRef === historyEvent.bankRef &&
|
||||
amountsEqual(h.amount, amount)
|
||||
);
|
||||
if (alreadyApplied) return { kind: "noop", updates: null, historyEvent: null };
|
||||
|
||||
if (event === "check_paid" || event === "ach_debit") {
|
||||
// A paid event dated on/before the latest known return is a replayed
|
||||
// original paid debit, not a redeposit — never re-clear from it.
|
||||
const latestReturnDate = (payment.history || [])
|
||||
.filter((h) => RETURN_EVENTS.has(h.event) && typeof h.date === "string")
|
||||
.map((h) => h.date)
|
||||
.sort()
|
||||
.pop();
|
||||
if (latestReturnDate && eventDateISO <= latestReturnDate) {
|
||||
return { kind: "noop", updates: null, historyEvent: null };
|
||||
}
|
||||
const redeposit = payment.clear_status === "Returned";
|
||||
const updates = {
|
||||
status: canceled ? payment.status : "Cleared",
|
||||
|
|
@ -376,24 +452,99 @@ export function isValidISODate(s) {
|
|||
);
|
||||
}
|
||||
|
||||
// Optional {fromDate, toDate} replay payload; the default stays yesterday,
|
||||
// matching the previous-day inquiry cadence. Strings are validated strictly
|
||||
// so a malformed payload fails loudly instead of querying a garbage range.
|
||||
// Optional {fromDate, toDate} replay payload. The default is a trailing
|
||||
// 3-day window (today-3 .. today-1): the previous-day feed only runs on
|
||||
// weekdays, so the Monday run must cover Friday through Sunday. Overlapping
|
||||
// days are safe — event identity makes replays idempotent. Strings are
|
||||
// validated strictly so a malformed payload fails loudly instead of
|
||||
// querying a garbage range.
|
||||
export function resolveDateRange(event, now = new Date()) {
|
||||
const hasFrom = event?.fromDate != null;
|
||||
const hasTo = event?.toDate != null;
|
||||
if (!hasFrom && !hasTo) {
|
||||
const yesterday = new Date(now.getTime() - 24 * 60 * 60 * 1000);
|
||||
const d = yesterday.toISOString().split("T")[0];
|
||||
return { fromDate: d, toDate: d };
|
||||
const day = 24 * 60 * 60 * 1000;
|
||||
return {
|
||||
fromDate: new Date(now.getTime() - 3 * day).toISOString().split("T")[0],
|
||||
toDate: new Date(now.getTime() - day).toISOString().split("T")[0],
|
||||
};
|
||||
}
|
||||
const fromDate = hasFrom ? event.fromDate : event.toDate;
|
||||
const toDate = hasTo ? event.toDate : event.fromDate;
|
||||
if (!isValidISODate(fromDate) || !isValidISODate(toDate)) {
|
||||
throw new Error("fromDate/toDate must be valid YYYY-MM-DD strings");
|
||||
throw new Error(
|
||||
`fromDate/toDate must be valid YYYY-MM-DD strings: ` +
|
||||
`fromDate=${logSafe(fromDate)}, toDate=${logSafe(toDate)}`
|
||||
);
|
||||
}
|
||||
if (fromDate > toDate) {
|
||||
throw new Error("fromDate must be <= toDate");
|
||||
throw new Error(
|
||||
`fromDate must be <= toDate: fromDate=${logSafe(fromDate)}, toDate=${logSafe(toDate)}`
|
||||
);
|
||||
}
|
||||
return { fromDate, toDate };
|
||||
}
|
||||
|
||||
// Staleness sweep (#66 review F6): flag records the bank has never
|
||||
// confirmed. ACH with no clear_status and a send date older than 16 days
|
||||
// (settlement lags at most ~8 business days) and checks older than 60 days
|
||||
// are surfaced in the run summary; cancel-status records are exempt.
|
||||
export const STALE_LIST_CAP = 50;
|
||||
export function sweepStalePayments(payments, todayISO) {
|
||||
const staleAch = [];
|
||||
let staleAchCount = 0;
|
||||
let staleChecksCount = 0;
|
||||
for (const p of payments) {
|
||||
if (p.clear_status || isCancelStatus(p.status)) continue;
|
||||
const age = daysSinceIssue(p, todayISO);
|
||||
if (age === null) continue;
|
||||
if (p.method === "ACH" && age > 16) {
|
||||
staleAchCount++;
|
||||
if (staleAch.length < STALE_LIST_CAP) {
|
||||
staleAch.push({
|
||||
check_number: p.check_number,
|
||||
amount: p.amount_usd,
|
||||
send_payment_on: p.send_payment_on,
|
||||
});
|
||||
}
|
||||
} else if (p.method === "Check" && age > 60) {
|
||||
staleChecksCount++;
|
||||
}
|
||||
}
|
||||
return { staleAch, staleAchCount, staleChecksCount };
|
||||
}
|
||||
|
||||
// Conditioned write for an applied event (#66 review F7): the update only
|
||||
// lands if the snapshot's status/clear_status are still current, so a
|
||||
// concurrent CSV upsert can't be silently interleaved. The handler retries
|
||||
// once against a fresh read on ConditionalCheckFailedException.
|
||||
export function buildEventUpdate(tableName, payment, applied) {
|
||||
const sets = ["#history = list_append(if_not_exists(#history, :empty), :hist)"];
|
||||
const names = { "#history": "history" };
|
||||
const values = { ":empty": [], ":hist": [applied.historyEvent] };
|
||||
Object.entries(applied.updates).forEach(([field, value], i) => {
|
||||
names[`#f${i}`] = field;
|
||||
values[`:v${i}`] = value;
|
||||
sets.push(`#f${i} = :v${i}`);
|
||||
});
|
||||
const conditions = [];
|
||||
[
|
||||
["status", payment.status],
|
||||
["clear_status", payment.clear_status],
|
||||
].forEach(([field, snapshot], i) => {
|
||||
names[`#c${i}`] = field;
|
||||
if (snapshot == null) {
|
||||
conditions.push(`attribute_not_exists(#c${i})`);
|
||||
} else {
|
||||
values[`:c${i}`] = snapshot;
|
||||
conditions.push(`#c${i} = :c${i}`);
|
||||
}
|
||||
});
|
||||
return {
|
||||
TableName: tableName,
|
||||
Key: { pk: payment.pk },
|
||||
UpdateExpression: `SET ${sets.join(", ")}`,
|
||||
ConditionExpression: conditions.join(" AND "),
|
||||
ExpressionAttributeNames: names,
|
||||
ExpressionAttributeValues: values,
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,8 +1,9 @@
|
|||
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
|
||||
import { DynamoDBDocumentClient, PutCommand, ScanCommand, UpdateCommand } from "@aws-sdk/lib-dynamodb";
|
||||
import { DynamoDBDocumentClient, GetCommand, PutCommand, ScanCommand, UpdateCommand } from "@aws-sdk/lib-dynamodb";
|
||||
import { SecretsManagerClient, GetSecretValueCommand } from "@aws-sdk/client-secrets-manager";
|
||||
import {
|
||||
applyEvent,
|
||||
buildEventUpdate,
|
||||
classifyTransaction,
|
||||
isValidISODate,
|
||||
logSafe,
|
||||
|
|
@ -10,6 +11,7 @@ import {
|
|||
matchCheckTransaction,
|
||||
matchElectronicReturn,
|
||||
resolveDateRange,
|
||||
sweepStalePayments,
|
||||
} from "./boaRecon.js";
|
||||
|
||||
const ddb = DynamoDBDocumentClient.from(new DynamoDBClient());
|
||||
|
|
@ -17,6 +19,9 @@ const secrets = new SecretsManagerClient();
|
|||
const TABLE_NAME = process.env.TABLE_NAME;
|
||||
const BOA_BASE_URL = process.env.BOA_BASE_URL;
|
||||
|
||||
const UNMATCHED_LIST_CAP = 50;
|
||||
const UNKNOWN_CODES_CAP = 20;
|
||||
|
||||
let cachedCreds;
|
||||
async function getReportingCreds() {
|
||||
if (cachedCreds) return cachedCreds;
|
||||
|
|
@ -27,6 +32,16 @@ async function getReportingCreds() {
|
|||
return cachedCreds;
|
||||
}
|
||||
|
||||
// Parse a BoA response body without ever surfacing a body snippet — a
|
||||
// malformed 200 must not leak account data into logs/errors.
|
||||
async function parseJsonResponse(res, label) {
|
||||
try {
|
||||
return await res.json();
|
||||
} catch {
|
||||
throw new Error(`BoA ${label} response not JSON: HTTP ${res.status}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function getAccessToken(applicationID, clientId, clientSecret) {
|
||||
const res = await fetch(`${BOA_BASE_URL}/authn/v1/client-authentication`, {
|
||||
method: "POST",
|
||||
|
|
@ -41,12 +56,15 @@ async function getAccessToken(applicationID, clientId, clientSecret) {
|
|||
throw new Error(`OAuth token exchange failed: HTTP ${res.status}`);
|
||||
}
|
||||
|
||||
const data = await res.json();
|
||||
const data = await parseJsonResponse(res, "authn");
|
||||
return data.access_token;
|
||||
}
|
||||
|
||||
const directionRank = (d) => (d === "debit" ? 0 : d === "credit" ? 1 : 2);
|
||||
|
||||
export const handler = async (event) => {
|
||||
// Optional replay payload {fromDate, toDate}; default is yesterday.
|
||||
// Optional replay payload {fromDate, toDate}; default is the trailing
|
||||
// 3-day window so the Monday run covers Friday through Sunday.
|
||||
const { fromDate, toDate } = resolveDateRange(event ?? {});
|
||||
|
||||
const { appId, clientId, token: clientSecret, accountNumber, bankId } = await getReportingCreds();
|
||||
|
|
@ -71,7 +89,7 @@ export const handler = async (event) => {
|
|||
throw new Error(`BoA API error: HTTP ${res.status}`);
|
||||
}
|
||||
|
||||
const data = await res.json();
|
||||
const data = await parseJsonResponse(res, "reporting");
|
||||
|
||||
// Response shape: { accountTransactions: [{ accountNumber, bankId, currency, transactions: [...] }] }
|
||||
const allTransactions = (data.accountTransactions || []).flatMap(
|
||||
|
|
@ -79,9 +97,16 @@ export const handler = async (event) => {
|
|||
);
|
||||
|
||||
// Process in posting-date order so multi-day replays apply paid -> return
|
||||
// -> redeposit sequences in the order the bank did.
|
||||
allTransactions.sort((a, b) =>
|
||||
String(a.valueDate ?? "").localeCompare(String(b.valueDate ?? ""))
|
||||
// -> redeposit sequences in the order the bank did; within a day, debits
|
||||
// before credits (deterministic, and a same-day return follows its debit).
|
||||
const classifiedTxns = allTransactions.map((txn) => ({
|
||||
txn,
|
||||
classified: classifyTransaction(txn),
|
||||
}));
|
||||
classifiedTxns.sort(
|
||||
(a, b) =>
|
||||
String(a.txn.valueDate ?? "").localeCompare(String(b.txn.valueDate ?? "")) ||
|
||||
directionRank(a.classified.direction) - directionRank(b.classified.direction)
|
||||
);
|
||||
|
||||
// Load all payment records (Check AND ACH — ACH is bank-confirmed here
|
||||
|
|
@ -109,19 +134,44 @@ export const handler = async (event) => {
|
|||
already_applied: 0,
|
||||
redeposits: 0,
|
||||
voided_and_bounced: 0,
|
||||
write_conflicts: 0,
|
||||
unmatched_count: 0,
|
||||
unmatched: [],
|
||||
unknown_count: 0,
|
||||
unknown_codes: {},
|
||||
};
|
||||
|
||||
for (const txn of allTransactions) {
|
||||
const classified = classifyTransaction(txn);
|
||||
const recordUnmatched = (classified, reason) => {
|
||||
summary.unmatched_count++;
|
||||
if (summary.unmatched.length < UNMATCHED_LIST_CAP) {
|
||||
summary.unmatched.push({
|
||||
event: classified.event,
|
||||
check_number: classified.checkNumber || classified.embeddedPaymentNumber,
|
||||
amount: classified.amount,
|
||||
reason,
|
||||
});
|
||||
}
|
||||
console.error(
|
||||
`Unmatched ${classified.event}: check=${logSafe(classified.checkNumber || classified.embeddedPaymentNumber)}, ` +
|
||||
`amount=${classified.amount}, reason=${reason} ` +
|
||||
`(bankRef: ${logSafe(classified.bankReference)})`
|
||||
);
|
||||
};
|
||||
|
||||
for (const { txn, classified } of classifiedTxns) {
|
||||
summary.classified[classified.event] = (summary.classified[classified.event] || 0) + 1;
|
||||
|
||||
if (classified.event === "ignored") continue;
|
||||
|
||||
if (classified.event === "unknown") {
|
||||
summary.unknown_codes[classified.code || "?"] =
|
||||
(summary.unknown_codes[classified.code || "?"] || 0) + 1;
|
||||
summary.unknown_count++;
|
||||
const codeKey = (classified.code || "?").slice(0, 16);
|
||||
if (
|
||||
codeKey in summary.unknown_codes ||
|
||||
Object.keys(summary.unknown_codes).length < UNKNOWN_CODES_CAP
|
||||
) {
|
||||
summary.unknown_codes[codeKey] = (summary.unknown_codes[codeKey] || 0) + 1;
|
||||
}
|
||||
console.error(
|
||||
`Unknown check-shaped transaction: code=${logSafe(classified.code)}, ` +
|
||||
`description=${logSafe(classified.description)}, ` +
|
||||
|
|
@ -130,7 +180,13 @@ export const handler = async (event) => {
|
|||
continue;
|
||||
}
|
||||
|
||||
const eventDate = isValidISODate(txn.valueDate) ? txn.valueDate : toDate;
|
||||
// Never substitute a date: event identity and transition ordering both
|
||||
// key on the posting date, so a feed row without one is a human case.
|
||||
if (!isValidISODate(txn.valueDate)) {
|
||||
recordUnmatched(classified, "missing valueDate");
|
||||
continue;
|
||||
}
|
||||
const eventDate = txn.valueDate;
|
||||
|
||||
let match;
|
||||
if (classified.event === "ach_debit" || classified.event === "ach_return") {
|
||||
|
|
@ -142,68 +198,106 @@ export const handler = async (event) => {
|
|||
}
|
||||
|
||||
if (!match.payment) {
|
||||
summary.unmatched.push({
|
||||
event: classified.event,
|
||||
check_number: classified.checkNumber || classified.embeddedPaymentNumber,
|
||||
amount: classified.amount,
|
||||
reason: match.unmatched,
|
||||
});
|
||||
console.error(
|
||||
`Unmatched ${classified.event}: check=${logSafe(classified.checkNumber || classified.embeddedPaymentNumber)}, ` +
|
||||
`amount=${classified.amount}, reason=${match.unmatched} ` +
|
||||
`(bankRef: ${logSafe(classified.bankReference)})`
|
||||
);
|
||||
recordUnmatched(classified, match.unmatched);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (match.matchedBy === "vendor+amount") {
|
||||
// Audit trail for the loosest ACH rung.
|
||||
console.log(
|
||||
`ACH vendor+amount match: payee=${logSafe(match.payment.payee)}, ` +
|
||||
`bankVendor=${logSafe(classified.vendorText)}, pmt=${logSafe(classified.pmtId)}`
|
||||
);
|
||||
}
|
||||
|
||||
summary.matched++;
|
||||
const applied = applyEvent(match.payment, classified, eventDate);
|
||||
const target = match.payment;
|
||||
let applied = applyEvent(target, classified, eventDate);
|
||||
if (applied.kind === "noop") {
|
||||
summary.already_applied++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Conditioned write: assert the snapshot's status/clear_status are
|
||||
// still current; on conflict, re-read, re-derive, retry once.
|
||||
let outcome = "conflict";
|
||||
for (let attempt = 0; attempt < 2; attempt++) {
|
||||
try {
|
||||
await ddb.send(new UpdateCommand(buildEventUpdate(TABLE_NAME, target, applied)));
|
||||
outcome = "written";
|
||||
break;
|
||||
} catch (err) {
|
||||
if (err.name !== "ConditionalCheckFailedException") throw err;
|
||||
if (attempt === 1) break;
|
||||
const { Item: fresh } = await ddb.send(
|
||||
new GetCommand({ TableName: TABLE_NAME, Key: { pk: target.pk } })
|
||||
);
|
||||
if (!fresh) break;
|
||||
// Refresh the in-memory record in place (it is shared with the
|
||||
// payments array) and re-derive the transition.
|
||||
for (const k of Object.keys(target)) delete target[k];
|
||||
Object.assign(target, fresh);
|
||||
applied = applyEvent(target, classified, eventDate);
|
||||
if (applied.kind === "noop") {
|
||||
outcome = "noop";
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (outcome === "noop") {
|
||||
summary.already_applied++;
|
||||
continue;
|
||||
}
|
||||
if (outcome !== "written") {
|
||||
summary.write_conflicts++;
|
||||
console.error(
|
||||
`Write conflict (gave up after retry): pk=${logSafe(target.pk)}, event=${classified.event}`
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
summary.applied++;
|
||||
if (applied.kind === "redeposit") summary.redeposits++;
|
||||
if (applied.kind === "voided_and_bounced") summary.voided_and_bounced++;
|
||||
|
||||
const sets = ["#history = list_append(if_not_exists(#history, :empty), :hist)"];
|
||||
const names = { "#history": "history" };
|
||||
const values = { ":empty": [], ":hist": [applied.historyEvent] };
|
||||
Object.entries(applied.updates).forEach(([field, value], i) => {
|
||||
names[`#f${i}`] = field;
|
||||
values[`:v${i}`] = value;
|
||||
sets.push(`#f${i} = :v${i}`);
|
||||
});
|
||||
|
||||
await ddb.send(
|
||||
new UpdateCommand({
|
||||
TableName: TABLE_NAME,
|
||||
Key: { pk: match.payment.pk },
|
||||
UpdateExpression: `SET ${sets.join(", ")}`,
|
||||
ExpressionAttributeNames: names,
|
||||
ExpressionAttributeValues: values,
|
||||
})
|
||||
);
|
||||
summary.applied++;
|
||||
|
||||
// Mirror the write onto the in-memory copy so later transactions in the
|
||||
// same run (return after paid, redeposit after return) see current state.
|
||||
Object.assign(match.payment, applied.updates);
|
||||
match.payment.history = [...(match.payment.history || []), applied.historyEvent];
|
||||
Object.assign(target, applied.updates);
|
||||
target.history = [...(target.history || []), applied.historyEvent];
|
||||
|
||||
console.log(
|
||||
`${applied.kind}: check ${logSafe(match.payment.check_number)} via ${match.matchedBy} ` +
|
||||
`${applied.kind}: check ${logSafe(target.check_number)} via ${match.matchedBy} ` +
|
||||
`(bankRef: ${logSafe(classified.bankReference)})`
|
||||
);
|
||||
}
|
||||
|
||||
// Run summary record for auditing/alerting (Slack wiring is #71).
|
||||
const runDate = fromDate === toDate ? fromDate : `${fromDate}_${toDate}`;
|
||||
// Staleness sweep: payments the bank has never confirmed (#69).
|
||||
const todayISO = new Date().toISOString().split("T")[0];
|
||||
const stale = sweepStalePayments(payments, todayISO);
|
||||
if (stale.staleAchCount) {
|
||||
console.error(
|
||||
`Stale ACH (no bank settlement, sent > 16 days ago): ${stale.staleAchCount} records, ` +
|
||||
`checks=${logSafe(stale.staleAch.map((s) => s.check_number).join(","))}`
|
||||
);
|
||||
}
|
||||
if (stale.staleChecksCount) {
|
||||
console.error(
|
||||
`Stale checks (no bank activity, issued > 60 days ago): ${stale.staleChecksCount} records`
|
||||
);
|
||||
}
|
||||
|
||||
// Run summary record for auditing/alerting (Slack wiring is #71). The pk
|
||||
// is append-only (run timestamp suffix) so re-runs never overwrite a
|
||||
// prior run's record.
|
||||
const runAt = new Date().toISOString();
|
||||
const runKey = `boa_recon#${fromDate}_${toDate}#${runAt}`;
|
||||
await ddb.send(
|
||||
new PutCommand({
|
||||
TableName: TABLE_NAME,
|
||||
Item: {
|
||||
pk: `boa_recon#${runDate}`,
|
||||
run_at: new Date().toISOString(),
|
||||
pk: runKey,
|
||||
run_at: runAt,
|
||||
from_date: fromDate,
|
||||
to_date: toDate,
|
||||
transactions_seen: summary.transactions_seen,
|
||||
|
|
@ -213,19 +307,24 @@ export const handler = async (event) => {
|
|||
already_applied: summary.already_applied,
|
||||
redeposits: summary.redeposits,
|
||||
voided_and_bounced: summary.voided_and_bounced,
|
||||
unmatched_count: summary.unmatched.length,
|
||||
write_conflicts: summary.write_conflicts,
|
||||
unmatched_count: summary.unmatched_count,
|
||||
unmatched: summary.unmatched,
|
||||
unknown_count: summary.unknown_count,
|
||||
unknown_codes: summary.unknown_codes,
|
||||
stale_ach: stale.staleAch,
|
||||
stale_ach_count: stale.staleAchCount,
|
||||
stale_checks_count: stale.staleChecksCount,
|
||||
ttl: Math.floor(Date.now() / 1000) + 90 * 24 * 60 * 60,
|
||||
},
|
||||
})
|
||||
);
|
||||
|
||||
const unknownCount = Object.values(summary.unknown_codes).reduce((a, b) => a + b, 0);
|
||||
if (summary.unmatched.length || unknownCount) {
|
||||
if (summary.unmatched_count || summary.unknown_count || summary.write_conflicts) {
|
||||
console.error(
|
||||
`Reconciliation ${fromDate}..${toDate}: ${summary.unmatched.length} unmatched, ` +
|
||||
`${unknownCount} unknown-code transactions (see boa_recon#${runDate})`
|
||||
`Reconciliation ${fromDate}..${toDate}: ${summary.unmatched_count} unmatched, ` +
|
||||
`${summary.unknown_count} unknown-code transactions, ` +
|
||||
`${summary.write_conflicts} write conflicts (see ${logSafe(runKey)})`
|
||||
);
|
||||
}
|
||||
|
||||
|
|
@ -233,13 +332,13 @@ export const handler = async (event) => {
|
|||
`Processed ${summary.transactions_seen} transactions ${fromDate}..${toDate}: ` +
|
||||
`${summary.matched} matched, ${summary.applied} applied, ` +
|
||||
`${summary.already_applied} already applied, ${summary.redeposits} redeposits, ` +
|
||||
`${summary.voided_and_bounced} voided-and-bounced, ${summary.unmatched.length} unmatched`
|
||||
`${summary.voided_and_bounced} voided-and-bounced, ${summary.unmatched_count} unmatched`
|
||||
);
|
||||
|
||||
return {
|
||||
statusCode: 200,
|
||||
body:
|
||||
`${summary.matched} of ${summary.transactions_seen} transactions matched ` +
|
||||
`(${summary.applied} applied, ${summary.unmatched.length} unmatched)`,
|
||||
`(${summary.applied} applied, ${summary.unmatched_count} unmatched)`,
|
||||
};
|
||||
};
|
||||
|
|
|
|||
|
|
@ -2,15 +2,19 @@ import { test } from "node:test";
|
|||
import assert from "node:assert/strict";
|
||||
import {
|
||||
BAI_CODE_EVENTS,
|
||||
STALE_LIST_CAP,
|
||||
amountsEqual,
|
||||
applyEvent,
|
||||
buildEventUpdate,
|
||||
classifyTransaction,
|
||||
digitsCorroborate,
|
||||
isCancelStatus,
|
||||
isValidISODate,
|
||||
matchAchTransaction,
|
||||
matchCheckTransaction,
|
||||
matchElectronicReturn,
|
||||
resolveDateRange,
|
||||
sweepStalePayments,
|
||||
vendorMatches,
|
||||
} from "../src/boaRecon.js";
|
||||
|
||||
|
|
@ -173,31 +177,67 @@ test("matcher: check number AND amount must both match", () => {
|
|||
assert.equal(m.matchedBy, "number+amount");
|
||||
});
|
||||
|
||||
test("matcher: number match with wrong amount falls back to unique exact-amount match", () => {
|
||||
test("matcher: number match with wrong amount NEVER falls through — human review", () => {
|
||||
const payments = [
|
||||
payment({ check_number: "1001", amount_usd: 500 }),
|
||||
payment({ check_number: "1002", amount_usd: 750.25, pk: "payment#1002" }),
|
||||
];
|
||||
// Posting collapsed onto 1001's number but carries 1002's amount.
|
||||
// Altered-check / collapsed-posting signal: alert, no guessing.
|
||||
const m = matchCheckTransaction({ checkNumber: "1001", amount: 750.25 }, payments, "2026-07-20");
|
||||
assert.equal(m.payment.check_number, "1002");
|
||||
assert.equal(m.payment, undefined);
|
||||
assert.equal(m.unmatched, "number matched, amount mismatch");
|
||||
});
|
||||
|
||||
test("matcher: digit-dropped number we never issued recovers via corroborated amount fallback", () => {
|
||||
// 1222000012 is a digit-subsequence of issued 11222000012.
|
||||
const payments = [payment({ check_number: "11222000012", amount_usd: 6413 })];
|
||||
const m = matchCheckTransaction({ checkNumber: "1222000012", amount: 6413 }, payments, "2026-07-20");
|
||||
assert.equal(m.payment.check_number, "11222000012");
|
||||
assert.equal(m.matchedBy, "amount");
|
||||
});
|
||||
|
||||
test("matcher: collapsed number we never issued still recovers by amount", () => {
|
||||
const payments = [payment({ check_number: "11222000123", amount_usd: 6413 })];
|
||||
const m = matchCheckTransaction({ checkNumber: "1222000012", amount: 6413 }, payments, "2026-07-20");
|
||||
assert.equal(m.payment.check_number, "11222000123");
|
||||
assert.equal(m.matchedBy, "amount");
|
||||
test("matcher: amount fallback without digit-subsequence corroboration is unmatched", () => {
|
||||
// Same amount, but 9876 shares no digit-subsequence relation with 11222000012.
|
||||
const payments = [payment({ check_number: "11222000012", amount_usd: 6413 })];
|
||||
const m = matchCheckTransaction({ checkNumber: "9876", amount: 6413 }, payments, "2026-07-20");
|
||||
assert.equal(m.payment, undefined);
|
||||
assert.match(m.unmatched, /no corroborated amount fallback/);
|
||||
});
|
||||
|
||||
test("matcher: check_return amount fallback requires a bank-confirmed candidate", () => {
|
||||
const unconfirmed = [payment({ check_number: "11222000012", amount_usd: 6413 })];
|
||||
const r1 = matchCheckTransaction(
|
||||
{ event: "check_return", checkNumber: "1222000012", amount: 6413 },
|
||||
unconfirmed,
|
||||
"2026-07-20"
|
||||
);
|
||||
assert.equal(r1.payment, undefined);
|
||||
|
||||
const confirmed = [
|
||||
payment({ check_number: "11222000012", amount_usd: 6413, clear_status: "Cleared" }),
|
||||
];
|
||||
const r2 = matchCheckTransaction(
|
||||
{ event: "check_return", checkNumber: "1222000012", amount: 6413 },
|
||||
confirmed,
|
||||
"2026-07-20"
|
||||
);
|
||||
assert.equal(r2.payment.check_number, "11222000012");
|
||||
});
|
||||
|
||||
test("digitsCorroborate: digit-subsequence in either direction, nothing else", () => {
|
||||
assert.equal(digitsCorroborate("1222000012", "11222000012"), true); // dropped digit
|
||||
assert.equal(digitsCorroborate("11222000012", "1222000012"), true); // symmetric
|
||||
assert.equal(digitsCorroborate("9876", "11222000012"), false);
|
||||
assert.equal(digitsCorroborate("", "1234"), false);
|
||||
});
|
||||
|
||||
test("matcher: ambiguous amount fallback is unmatched — no write", () => {
|
||||
// "101" digit-corroborates both 1001 and 1011; same amount -> ambiguous.
|
||||
const payments = [
|
||||
payment({ check_number: "1001", amount_usd: 500 }),
|
||||
payment({ check_number: "1002", amount_usd: 350, pk: "payment#1002" }),
|
||||
payment({ check_number: "1003", amount_usd: 350, pk: "payment#1003" }),
|
||||
payment({ check_number: "1001", amount_usd: 350 }),
|
||||
payment({ check_number: "1011", amount_usd: 350, pk: "payment#1011" }),
|
||||
];
|
||||
const m = matchCheckTransaction({ checkNumber: "1001", amount: 350 }, payments, "2026-07-20");
|
||||
const m = matchCheckTransaction({ checkNumber: "101", amount: 350 }, payments, "2026-07-20");
|
||||
assert.equal(m.payment, undefined);
|
||||
assert.match(m.unmatched, /ambiguous/);
|
||||
});
|
||||
|
|
@ -208,10 +248,11 @@ test("matcher: zero candidates is unmatched", () => {
|
|||
});
|
||||
|
||||
test("matcher: amount fallback only considers checks issued in the last 120 days", () => {
|
||||
// Corroborated ("101" is a subsequence of "1001") but issued too long ago.
|
||||
const payments = [
|
||||
payment({ check_number: "1001", amount_usd: 350, send_payment_on: "01/02/2026" }),
|
||||
];
|
||||
const m = matchCheckTransaction({ checkNumber: "2001", amount: 350 }, payments, "2026-07-20");
|
||||
const m = matchCheckTransaction({ checkNumber: "101", amount: 350 }, payments, "2026-07-20");
|
||||
assert.equal(m.payment, undefined);
|
||||
});
|
||||
|
||||
|
|
@ -342,6 +383,36 @@ test("transitions: identical replayed event is a noop (idempotent replays)", ()
|
|||
assert.equal(r.updates, null);
|
||||
});
|
||||
|
||||
test("transitions: dedup identity is {event, date, amount} — bankRef differences are ignored", () => {
|
||||
const p = payment({
|
||||
status: "Cleared",
|
||||
clear_status: "Cleared",
|
||||
history: [{ event: "check_paid", date: "2026-07-19", bankRef: "OLD-FORMAT-REF", amount: 500 }],
|
||||
});
|
||||
// Same event replayed with a reformatted/omitted bankRef must still dedup.
|
||||
const r = applyEvent(p, paidEvent({ bankReference: "813399999" }), "2026-07-19");
|
||||
assert.equal(r.kind, "noop");
|
||||
});
|
||||
|
||||
test("transitions: a paid event dated on/before the latest return is a replayed original, not a redeposit", () => {
|
||||
const p = payment({
|
||||
status: "Cleared",
|
||||
clear_status: "Returned",
|
||||
history: [
|
||||
{ event: "check_paid", date: "2026-07-18", bankRef: "a", amount: 500 },
|
||||
{ event: "check_return", date: "2026-07-19", bankRef: "b", amount: 500 },
|
||||
],
|
||||
});
|
||||
// Replayed original paid (different bankRef, date <= return date): noop.
|
||||
const equalDate = applyEvent(p, paidEvent({ bankReference: "c" }), "2026-07-19");
|
||||
assert.equal(equalDate.kind, "noop");
|
||||
const beforeDate = applyEvent(p, paidEvent({ bankReference: "c" }), "2026-07-17");
|
||||
assert.equal(beforeDate.kind, "noop");
|
||||
// Strictly after the return: genuine redeposit.
|
||||
const after = applyEvent(p, paidEvent({ bankReference: "c" }), "2026-07-20");
|
||||
assert.equal(after.kind, "redeposit");
|
||||
});
|
||||
|
||||
test("transitions: every non-noop result sets both status and clear_status", () => {
|
||||
const cases = [
|
||||
[payment(), paidEvent()],
|
||||
|
|
@ -433,26 +504,95 @@ test("ach matcher: vendor+amount only within send_payment_on -2..+14 days", () =
|
|||
assert.equal(m2.payment, undefined);
|
||||
});
|
||||
|
||||
test("ach matcher: same-amount return credit after the debit matches a bank-confirmed ACH", () => {
|
||||
test("ach matcher: PMT-id-less same-amount return credit matches a recently cleared ACH", () => {
|
||||
const payments = [
|
||||
achPayment({ payee: "Uline", clear_status: "Cleared", status: "Cleared", amount_usd: 19281.12 }),
|
||||
achPayment({
|
||||
payee: "Uline",
|
||||
clear_status: "Cleared",
|
||||
status: "Cleared",
|
||||
cleared_date: "2026-05-22",
|
||||
amount_usd: 19281.12,
|
||||
}),
|
||||
];
|
||||
// Reversal credit: originator is SEA HAVEN INDUST, so vendor match fails.
|
||||
const m = matchAchTransaction(
|
||||
{ event: "ach_return", pmtId: "999", embeddedPaymentNumber: null, vendorText: "Sea Haven Industries, Inc", amount: 19281.12 },
|
||||
{ event: "ach_return", pmtId: null, embeddedPaymentNumber: null, vendorText: null, amount: 19281.12 },
|
||||
payments,
|
||||
"2026-05-26"
|
||||
);
|
||||
assert.equal(m.matchedBy, "amount+cleared");
|
||||
});
|
||||
|
||||
test("ach matcher: ambiguity is unmatched — no write", () => {
|
||||
test("ach matcher: amount+cleared fallback requires cleared_date within 45 days before the credit", () => {
|
||||
const base = {
|
||||
payee: "Uline",
|
||||
clear_status: "Cleared",
|
||||
status: "Cleared",
|
||||
amount_usd: 19281.12,
|
||||
};
|
||||
const tooOld = [achPayment({ ...base, cleared_date: "2026-03-01" })];
|
||||
const m1 = matchAchTransaction(
|
||||
{ event: "ach_return", pmtId: null, embeddedPaymentNumber: null, vendorText: null, amount: 19281.12 },
|
||||
tooOld,
|
||||
"2026-05-26"
|
||||
);
|
||||
assert.equal(m1.payment, undefined);
|
||||
|
||||
const noDate = [achPayment({ ...base })];
|
||||
const m2 = matchAchTransaction(
|
||||
{ event: "ach_return", pmtId: null, embeddedPaymentNumber: null, vendorText: null, amount: 19281.12 },
|
||||
noDate,
|
||||
"2026-05-26"
|
||||
);
|
||||
assert.equal(m2.payment, undefined);
|
||||
});
|
||||
|
||||
test("ach matcher: return credit with an unattributable PMT id is unmatched, never amount-guessed", () => {
|
||||
const payments = [
|
||||
achPayment({ pk: "payment#a", check_number: "a", clear_status: "Cleared", amount_usd: 500 }),
|
||||
achPayment({ pk: "payment#b", check_number: "b", clear_status: "Cleared", amount_usd: 500 }),
|
||||
achPayment({ clear_status: "Cleared", status: "Cleared", cleared_date: "2026-05-22", amount_usd: 8300 }),
|
||||
];
|
||||
const m = matchAchTransaction(
|
||||
{ event: "ach_return", pmtId: "999", embeddedPaymentNumber: null, vendorText: null, amount: 500 },
|
||||
{ event: "ach_return", pmtId: "999", embeddedPaymentNumber: null, vendorText: null, amount: 8300 },
|
||||
payments,
|
||||
"2026-05-26"
|
||||
);
|
||||
assert.equal(m.payment, undefined);
|
||||
assert.equal(m.unmatched, "unknown PMT id");
|
||||
});
|
||||
|
||||
test("ach matcher: pmt_id match with amount mismatch is unmatched — partial-reversal human case", () => {
|
||||
const payments = [achPayment({ pmt_id: "7584198", amount_usd: 8300 })];
|
||||
const m = matchAchTransaction(
|
||||
{ event: "ach_return", pmtId: "7584198", embeddedPaymentNumber: null, vendorText: null, amount: 4150 },
|
||||
payments,
|
||||
"2026-06-10"
|
||||
);
|
||||
assert.equal(m.payment, undefined);
|
||||
assert.equal(m.unmatched, "pmt_id matched, amount mismatch");
|
||||
});
|
||||
|
||||
test("ach matcher: candidates with a DIFFERENT stored pmt_id are excluded from vendor+amount", () => {
|
||||
const payments = [
|
||||
achPayment({ pmt_id: "1111111", amount_usd: 8300 }),
|
||||
achPayment({ check_number: "21222000270", pk: "payment#21222000270", amount_usd: 8300 }),
|
||||
];
|
||||
// Both are Cobra Septic @ 8300 in-window; the pmt_id conflict on the
|
||||
// first disambiguates to the second instead of going ambiguous.
|
||||
const m = matchAchTransaction(
|
||||
{ event: "ach_debit", pmtId: "2222222", embeddedPaymentNumber: null, vendorText: "Cobra Septic", amount: 8300 },
|
||||
payments,
|
||||
"2026-06-08"
|
||||
);
|
||||
assert.equal(m.payment.check_number, "21222000270");
|
||||
assert.equal(m.matchedBy, "vendor+amount");
|
||||
});
|
||||
|
||||
test("ach matcher: ambiguity is unmatched — no write", () => {
|
||||
const payments = [
|
||||
achPayment({ pk: "payment#a", check_number: "a", clear_status: "Cleared", cleared_date: "2026-06-01", amount_usd: 500 }),
|
||||
achPayment({ pk: "payment#b", check_number: "b", clear_status: "Cleared", cleared_date: "2026-06-02", amount_usd: 500 }),
|
||||
];
|
||||
const m = matchAchTransaction(
|
||||
{ event: "ach_return", pmtId: null, embeddedPaymentNumber: null, vendorText: null, amount: 500 },
|
||||
payments,
|
||||
"2026-06-08"
|
||||
);
|
||||
|
|
@ -467,6 +607,15 @@ test("vendorMatches: truncated bank originator vs full payee, punctuation-insens
|
|||
assert.equal(vendorMatches("", "Cobra Septic"), false);
|
||||
});
|
||||
|
||||
test("vendorMatches: prefix only counts at >= 10 normalized chars; short names need exact equality", () => {
|
||||
// "ACME" (4 chars) must not claim "ACME Plumbing Co" by prefix...
|
||||
assert.equal(vendorMatches("ACME Plumbing Co", "ACME"), false);
|
||||
// ...but exact short-name equality still matches.
|
||||
assert.equal(vendorMatches("ACME", "A.C.M.E."), true);
|
||||
// 10+ char prefix still matches (truncated originators).
|
||||
assert.equal(vendorMatches("Alliance Sanitation LLC", "ALLIANCESANITAT"), true);
|
||||
});
|
||||
|
||||
test("ach transitions: settled debit clears both fields and persists pmt_id", () => {
|
||||
const p = achPayment();
|
||||
const r = applyEvent(
|
||||
|
|
@ -504,11 +653,89 @@ test("ach transitions: electronic return on a voided-but-settled ACH is voided-a
|
|||
assert.equal(r.updates.clear_status, "Returned");
|
||||
});
|
||||
|
||||
// -------------------------------------------------- staleness sweep (F6)
|
||||
|
||||
test("stale sweep: unconfirmed ACH older than 16 days is listed; checks older than 60 days counted", () => {
|
||||
const payments = [
|
||||
achPayment({ check_number: "21222000300", send_payment_on: "06/20/2026" }), // 31d, stale
|
||||
achPayment({ check_number: "21222000301", pk: "payment#21222000301", send_payment_on: "07/10/2026" }), // 11d, fresh
|
||||
achPayment({ check_number: "21222000302", pk: "payment#21222000302", send_payment_on: "06/01/2026", clear_status: "Cleared" }), // confirmed
|
||||
achPayment({ check_number: "21222000303", pk: "payment#21222000303", send_payment_on: "06/01/2026", status: "Voided" }), // cancel-exempt
|
||||
payment({ check_number: "3001", send_payment_on: "04/01/2026" }), // check, 111d, stale
|
||||
payment({ check_number: "3002", pk: "payment#3002", send_payment_on: "07/01/2026" }), // check, fresh
|
||||
];
|
||||
const s = sweepStalePayments(payments, "2026-07-21");
|
||||
assert.equal(s.staleAchCount, 1);
|
||||
assert.deepEqual(s.staleAch, [
|
||||
{ check_number: "21222000300", amount: 8300, send_payment_on: "06/20/2026" },
|
||||
]);
|
||||
assert.equal(s.staleChecksCount, 1);
|
||||
});
|
||||
|
||||
test("stale sweep: ACH list is capped, count is not", () => {
|
||||
const payments = [];
|
||||
for (let i = 0; i < STALE_LIST_CAP + 5; i++) {
|
||||
payments.push(
|
||||
achPayment({ check_number: `2122200${1000 + i}`, pk: `payment#s${i}`, send_payment_on: "06/01/2026" })
|
||||
);
|
||||
}
|
||||
const s = sweepStalePayments(payments, "2026-07-21");
|
||||
assert.equal(s.staleAchCount, STALE_LIST_CAP + 5);
|
||||
assert.equal(s.staleAch.length, STALE_LIST_CAP);
|
||||
});
|
||||
|
||||
// ---------------------------------------------- conditioned writes (F7)
|
||||
|
||||
test("buildEventUpdate: condition asserts the snapshot's status and clear_status", () => {
|
||||
const p = payment({ status: "Outstanding", clear_status: "Cleared" });
|
||||
const applied = applyEvent(p, returnEvent(), "2026-07-20");
|
||||
const cmd = buildEventUpdate("Table", p, applied);
|
||||
assert.equal(cmd.TableName, "Table");
|
||||
assert.deepEqual(cmd.Key, { pk: p.pk });
|
||||
assert.equal(cmd.ConditionExpression, "#c0 = :c0 AND #c1 = :c1");
|
||||
assert.equal(cmd.ExpressionAttributeNames["#c0"], "status");
|
||||
assert.equal(cmd.ExpressionAttributeNames["#c1"], "clear_status");
|
||||
assert.equal(cmd.ExpressionAttributeValues[":c0"], "Outstanding");
|
||||
assert.equal(cmd.ExpressionAttributeValues[":c1"], "Cleared");
|
||||
assert.match(cmd.UpdateExpression, /^SET #history = list_append/);
|
||||
// Every applied field is present in the SET clause.
|
||||
const setFields = Object.entries(cmd.ExpressionAttributeNames)
|
||||
.filter(([k]) => k.startsWith("#f"))
|
||||
.map(([, v]) => v);
|
||||
assert.deepEqual(setFields.sort(), Object.keys(applied.updates).sort());
|
||||
});
|
||||
|
||||
test("buildEventUpdate: missing snapshot clear_status becomes attribute_not_exists", () => {
|
||||
const p = payment(); // no clear_status yet
|
||||
const applied = applyEvent(p, paidEvent(), "2026-07-19");
|
||||
const cmd = buildEventUpdate("Table", p, applied);
|
||||
assert.equal(cmd.ConditionExpression, "#c0 = :c0 AND attribute_not_exists(#c1)");
|
||||
assert.equal(":c1" in cmd.ExpressionAttributeValues, false);
|
||||
});
|
||||
|
||||
// ----------------------------------------------------- ReDoS hardening
|
||||
|
||||
test("classifier: bounded regexes still parse zero-padded and normal check descriptions", () => {
|
||||
const c = classifyTransaction({ text: "CHECK # 0003176", amount: "-350" });
|
||||
assert.equal(c.event, "check_paid");
|
||||
assert.equal(c.checkNumber, "3176");
|
||||
});
|
||||
|
||||
test("classifier: pathological long input is bounded, classified without hanging", () => {
|
||||
const junk = `CHECK ${" ".repeat(2000)}${"9".repeat(2000)}`;
|
||||
const start = Date.now();
|
||||
const c = classifyTransaction({ text: junk, customerReference: "42", amount: "-1" });
|
||||
assert.ok(Date.now() - start < 1000);
|
||||
// Not parseable as a paid check -> unknown (check-shaped), never dropped.
|
||||
assert.equal(c.event, "unknown");
|
||||
assert.ok(c.description.length <= 500);
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------- date range
|
||||
|
||||
test("resolveDateRange: default is yesterday", () => {
|
||||
test("resolveDateRange: default is the trailing 3-day window (Mon covers Fri-Sun)", () => {
|
||||
const r = resolveDateRange({}, new Date("2026-07-21T13:00:00Z"));
|
||||
assert.deepEqual(r, { fromDate: "2026-07-20", toDate: "2026-07-20" });
|
||||
assert.deepEqual(r, { fromDate: "2026-07-18", toDate: "2026-07-20" });
|
||||
});
|
||||
|
||||
test("resolveDateRange: explicit valid range passes through", () => {
|
||||
|
|
@ -521,10 +748,13 @@ test("resolveDateRange: fromDate only replays a single day", () => {
|
|||
assert.deepEqual(r, { fromDate: "2026-06-08", toDate: "2026-06-08" });
|
||||
});
|
||||
|
||||
test("resolveDateRange: malformed and impossible dates throw", () => {
|
||||
assert.throws(() => resolveDateRange({ fromDate: "06/08/2026" }));
|
||||
assert.throws(() => resolveDateRange({ fromDate: "2026-02-30" }));
|
||||
assert.throws(() => resolveDateRange({ fromDate: "2026-07-02", toDate: "2026-07-01" }));
|
||||
test("resolveDateRange: malformed and impossible dates throw, naming the offending value", () => {
|
||||
assert.throws(() => resolveDateRange({ fromDate: "06/08/2026" }), /06\/08\/2026/);
|
||||
assert.throws(() => resolveDateRange({ fromDate: "2026-02-30" }), /2026-02-30/);
|
||||
assert.throws(
|
||||
() => resolveDateRange({ fromDate: "2026-07-02", toDate: "2026-07-01" }),
|
||||
/fromDate="2026-07-02", toDate="2026-07-01"/
|
||||
);
|
||||
assert.throws(() => resolveDateRange({ fromDate: "2026-07-01; DROP", toDate: "2026-07-02" }));
|
||||
});
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue