2 commits

Author	SHA1	Message	Date
Adam Moussa	d8a2412d75	fix(fetchboa): read real CashPro field names; extend BAI code map; 7-day window (#75) ... The deployed v2 pipeline was fully inert: the classifier never read detailText (where the live API carries the ACH DES:PAYMENTS text) and the handler keyed event dates off valueDate, which the API does not send (it sends asOfDate) — so ACH could not classify and no event of any kind could apply. Both proven against real captured responses. - classifyTransaction: detailText first in the description chain; Summary-row guard (new "summary" event); all-zeros customerReference normalizes to empty instead of fabricating check number 0. - BAI_CODE_EVENTS: empirical enumeration lands — 255 + 252 are both check-numbered return credits, 266 is the no-number return credit (text disambiguates ach_return vs electronic_return via new fallbackEvent semantics), 455 is ach_debit via DES text or ignored third-party autopay, 170/201/470/481 are noise. - postingDate helper (asOfDate ?? valueDate) drives both the sort and event dates; unmatched reason is now "missing posting date". - Default replay window widened to trailing 7 days ending yesterday: self-healing across missed runs; responses verified pagination-free. Refs: #66, #69	2026-07-22 15:40:49 -04:00
Adam Moussa	f7adb6e8b8	feat(fetchboa): bank-truth reconciliation v2 — returns, redeposits, ACH confirmation (#73) ... * feat(fetchboa): v2 return/redeposit-aware reconciliation (#66) The two-code 475/255 filter missed every return on the Jan-Jul statement (29 ARP refer-to-maker credits, 24 electronic return credits, and the redeposit cycles), leaving 5 paid-then-returned checks stuck at Cleared ($5,256.62, vendors unpaid). Rewrite fetchBoaTransactions around a pure reconciliation module (src/boaRecon.js) covered by node:test: - BAI transaction-code event map (only 475 = check paid is confirmed; remaining codes pend the enumeration replay) with a description-text fallback classifier for ARP refer-to-maker, return-of-posted-check (check-numbered and electronic) and ACH DES:PAYMENTS formats. Unknown check-shaped transactions are logged and counted, never dropped. - Matching on check number AND amount over all candidates; wrong-amount or collapsed-number postings fall back to a unique exact-amount match within checks issued in the last 120 days; ambiguity means unmatched with no write. - Transitions always set BOTH status and clear_status (the missed returns slipped through the divergence between them). clear_status is bank truth: returns apply even to Cleared records, a second paid debit on a Returned check is a redeposit back to Cleared, and a return on a voided check is terminal voided-and-bounced. Every applied event is appended to a history list; identical replayed events are noops. - Optional {fromDate, toDate} replay payload (strictly validated) for gap replays and the BAI-code enumeration runs; default stays yesterday. - Each run writes a boa_recon#<runDate> summary item (90-day TTL) with per-event counts, unmatched check numbers/amounts, and unknown codes; unmatched/unknown also console.error (Slack alerting is #71). ACH transactions are classified but not yet acted on; bank-confirming ACH lands with #69. * feat(ach): bank-confirm ACH, drop CSV-date auto-clear (#69) processPaymentCsv auto-cleared ACH the moment send_payment_on passed, but the bank disagrees often enough to matter: settlement lags the send date by up to 8 days, settled ACH can bounce and re-debit (Uline $19,281.12, Heights $10,000.00 on the 5/26 overdrawn week), and voided ACH that settled anyway returned via electronic credits invisible to a Check-only feed. Move ACH to bank confirmation: - processPaymentCsv: ACH rows keep their Stampli status; the send-date auto-clear is removed. The bankConfirmed protection is unchanged, so bank-cleared records still cannot be moved backward by the CSV. - fetchBoaTransactions scans all payments (method filter dropped) and processes ACH CCD lines: match by stored pmt_id first (reversals reuse the original PMT id), then by the Stampli payment number embedded in PMT INFO (internal spaces stripped, amount must agree), then vendor + exact amount within send_payment_on -2..+14 days. Settled debit -> Cleared/Cleared with dates, pmt_id persisted on the record; credit reusing the pmt_id (or a unique same-amount return credit against a bank-confirmed ACH) -> clear_status=Returned + returned_date + history event. Ambiguity is unmatched, no write. Existing DDB ACH records already Cleared by the old auto-clear are unaffected: bank confirmation is additive and the CSV path never moves a record backward. Handler event contract extended (optional fromDate/toDate); cross-family review required before merge. * fix(fetchboa): close review findings — coverage gap, matcher corroboration, replay identity (#66) Security-review gate (detector fan-out + verifier + GPT-4.1 cross-family) blocked on F6 and confirmed six lower findings; all are closed here. - F6 (HIGH): the default run now queries a trailing 3-day window (today-3..today-1) so the Monday run covers Friday-Sunday postings the previous-day cadence silently skipped. A per-run staleness sweep flags never-bank-confirmed payments (ACH > 16 days, checks > 60 days) in the summary and via console.error. - F2: replay identity is {event, date, amount} — bankRef excluded (feeds omit/reformat it between runs); a paid event dated on/before the latest return in history is a replayed original, never a redeposit; rows without a valid valueDate are routed to unmatched instead of being applied under a substituted date; within a day, debits sort before credits. - F1: a check-number match with the wrong amount no longer falls through to the amount fallback (altered-check/collapsed-posting is a human-review case); the amount fallback requires digit-subsequence corroboration between posting and candidate numbers; check_return fallback requires a bank-confirmed candidate. - F4: the ach_return amount fallback requires cleared_date within 45 days before the credit; an unattributable PMT id is an unmatched alert, never an amount guess. - F5: the pmt_id rung requires amount equality; mismatch is the partial-reversal human case. - F3: vendor prefix matching requires >= 10 normalized chars (short names must match exactly); candidates with a conflicting stored pmt_id are excluded; vendor+amount matches are audit-logged. - F7: payment writes are conditioned on the snapshot's status/clear_status and retried once against a fresh read; residual conflicts are counted, not silently interleaved. - F9/summary bounds: run summary pk is append-only (boa_recon#<from>_<to>#<runAt>); unmatched list capped at 50, unknown codes at 20 keys/16 chars, stale list at 50 (full counts kept). - ReDoS: description bounded to 500 chars before classification; CHECK/embedded-number regexes use bounded quantifiers. - FBOA2-1: both BoA res.json() parses are wrapped so a malformed 200 throws a status-only error and cannot leak a body snippet. Handler event contract extended (optional fromDate/toDate); cross-family review required before merge. * fix(boaRecon): add method=Check filter to matchElectronicReturn Electronic returns only apply to checks, but the matcher was not filtering by method, so an electronic return could incorrectly match against a same-amount, bank-confirmed ACH record.	2026-07-21 22:07:37 -04:00