sh-mcp/packages/shared/src/redact.ts
Adam Moussa 22c09e99fe
Some checks are pending
deploy / deploy (push) Waiting to run
Phase 1: runnable MCP + OpenAPI servers (ops + finance) (#3)
* Add shared transport: dispatch, MCP + OpenAPI adapters, local auth

Add the single authoritative tool-execution path (executeTool) plus the two
universal interfaces over it (design.md §2.5, §7.3):
- dispatch.ts: scope enforcement, ajv input validation, rate limiting, finance
  egress redaction (redactDeep), and structured audit emission on one path.
- audit.ts / rate-limit.ts: injected AuditLogger + RateLimiter abstractions.
- mcp.ts: low-level MCP Server with scope-filtered tools/list (tool-hiding) and
  tools/call routed through executeTool.
- http.ts: Express host mounting /mcp, /openapi.json, POST /tools/:name, /healthz.
- openapi.ts: buildOpenApiDocument wraps the existing path generator into a full
  OpenAPI 3.1 document.
- local-auth.ts: LocalAuthProvider (dev bearer tokens) that refuses to construct
  outside SH_MCP_ENV=local and enforces audience binding (design.md §3, §6).

* Add in-memory dev clients; make package tool exports lazy

Add an in-memory Client implementation per integration package (seeded fake
data, no network) selected when SH_MCP_ENV=local (build-plan §4). Gmail/calendar/
tasks dev clients partition by ctx.sub; payments/qbo seed sensitive-looking
fields so the redaction egress path has real targets to mask.

Make the eager default-tool exports in tasks/reminders/qbo LAZY (getDefaultTools)
so importing a package barrel no longer constructs an AWS client at module load
(build-plan §7 'no I/O at import time') — the previous eager construction broke
server startup. Fix payments tsconfig rootDir (src, was '.') so its declarations
resolve under dist/index.d.ts like the other 8 packages.

* Add runnable sh-mcp-ops and sh-mcp-finance servers

Two thin composition-root servers over the shared transport (design.md §3):
- ops: internal-data, knowledge-base, google-maps, gmail, calendar, tasks,
  reminders. finance: qbo, payments (audited + redacted on egress).
- config from env only (no hardcoded ids/issuer/tables); SH_MCP_ENV selects
  LocalAuthProvider + dev clients (local) vs CognitoAuthProvider + real stubs
  (aws). Finance applies the 15-min finance-token TTL ceiling (design.md §2.5).
- index.ts is the only place .listen() is called; a Lambda handler placeholder
  is exported but not depended on.
- synth-only CDK stubs (no real IAM/Cognito/WAF) so 'cdk synth' has a valid app
  (build-plan §6); READMEs document local run, dev tokens, curl, MCP Inspector.

* Add security-weighted test suite + coverage gate; wire tooling

Add tests for the highest-risk surface (build-plan §5, design.md §7.3):
tool-hiding, server-side scope enforcement (incl. forced hidden calls),
audience binding, input-schema validation, finance redaction on egress, audit
emission with hashed args, prompt-injection regression (tool output is data),
rate limiting, MCP conformance (in-memory transport round-trip), OpenAPI 3.1
validity, and local-auth safety. Add HTTP integration tests (supertest) for both
servers and per-package dev-client tests. 405 tests pass.

Wire the coverage gate into vitest.config.ts: 80% overall, with per-file
thresholds on the auth + dispatch crown jewels; exclude deferred real client
stubs, entrypoints, cdk apps, and aws-only config from the gate (documented).
Extend eslint flat config + add .prettierignore to cover servers/. Commit the
updated package-lock.json.

* Suppress pre-existing dev-tooling + out-of-scope scanner findings

Add written-justification suppressions for the 4 confirmed crit/high pre-push
scanner findings, none of which are in this PR's Phase 1 production code:
- npmaudit vitest / @vitest/coverage-v8 / vite: dev/test-only deps that never
  run in the deployed server/Lambda runtime (pins carried from Phase 0b;
  Dependabot will bump).
- gitleaks docs/agentforce-plan.md secret: that file is not on this branch and
  not in this changeset; flagged for the maintainer to scrub on its own branch.

The deep agentic /sh-security-review (required for this auth/authz-touching PR)
was NOT run by the agent and is flagged outstanding in the PR body.

* Address CodeQL findings: bound ajv error work + edge rate limiting

GHAS code-scanning alerts on this PR:
- dispatch.ts (js/resource-exhaustion): ajv ran with allErrors:true on
  untrusted input, letting a crafted payload force unbounded error
  enumeration. Switch to allErrors:false (default) so validation
  short-circuits on the first failure; the 400 still names that path.
- http.ts (js/missing-rate-limiting): the authenticated routes (/mcp,
  /tools/:name) had no edge throttle — auth/JWT verification ran on every
  request before the per-sub dispatch limiter could apply. Add an IP-keyed
  express-rate-limit in front of authenticate (120/60s default, configurable),
  returning the standard 429 shape. Defense-in-depth over the per-sub +
  per-tool limiter in executeTool; API GW/WAF remains the production edge.

Tests: +2 cases proving the edge limiter throttles before auth (429, not
401) on /tools and /mcp. 407 pass; tsc/eslint/prettier clean.

* Fix polynomial ReDoS in Bearer-token extraction (CodeQL js/polynomial-redos)

extractBearerToken matched /^Bearer\s+(.+)$/ — \s and . both match a space,
so the two quantifiers overlap and a crafted header can drive polynomial
backtracking. Require the capture to start with a non-whitespace char
(/^Bearer\s+(\S.*)$/), removing the ambiguity → linear match. Behavior is
unchanged for real tokens; +2 regression tests.

* Harden auth + finance redaction (sh-security-review confirmed mediums)

Two confirmed medium findings from the agentic security review:

- Fail-open SH_MCP_ENV: config defaulted to 'local' when the var was unset,
  so a deploy that forgot SH_MCP_ENV=aws would silently run LocalAuthProvider
  and accept static dev bearer tokens (dev-finance-admin -> finance:admin).
  Now fail-closed: SH_MCP_ENV must be explicitly 'local' or 'aws' or the
  server refuses to start. Plus an independent guard in LocalAuthProvider
  that refuses to construct in an AWS runtime (AWS_LAMBDA_FUNCTION_NAME /
  AWS_EXECUTION_ENV present), regardless of the env flag.

- Finance egress redaction gap: redactDeep only wholesale-masked a sensitive
  key when its value was a scalar; an object/array under a sensitive key was
  recursed into, letting a bare nested value (e.g. {account:{number:...}})
  escape the keyword-gated pattern matcher. Now the entire subtree under a
  sensitive key is masked. No current finance tool emitted such shapes (all
  flat strings), so this closes a latent hole in the universal safety net.

+4 tests (subtree redaction, AWS-runtime guard). 411 pass; coverage gate green.

Review also produced lows (memo free-text digits, unsalted argsHash,
unauth /openapi.json by-design, session-cap no-reset by-design) tracked
separately; 0 confirmed critical/high — review verdict PASS.
2026-06-26 13:33:21 -04:00

164 lines
5.9 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

/**
* MCP-layer PII redaction.
*
* Masks sensitive financial identifiers in tool response strings before they
* leave the server and reach the LLM agent or the user.
*
* WHAT IS MASKED:
* - US bank routing numbers (9 digits, ABA)
* - US bank account numbers (4–17 digits following routing or account keywords)
* - Payment card numbers (13–19 digits, Luhn-matching encouraged; basic pattern here)
* - US Social Security Numbers (NNN-NN-NNNN / NNN NN NNNN / NNNNNNNNN)
*
* WHAT IS LEFT INTACT:
* - Vendor/company names
* - Contact information (phone numbers, email addresses, street addresses)
* - Dollar amounts and invoice/PO numbers
* - Any other non-financial-identity data
*
* See docs/design.md §2.5 for the requirement context.
* See src/redact.test.ts for the full contract.
*/
// ---------------------------------------------------------------------------
// Mask helper
// ---------------------------------------------------------------------------
/** The string used to replace masked values. */
export const REDACTED = '[REDACTED]';
// ---------------------------------------------------------------------------
// Individual pattern redactors
// ---------------------------------------------------------------------------
/**
* Mask US Social Security Numbers.
*
* Patterns matched:
* - NNN-NN-NNNN (canonical)
* - NNN NN NNNN (spaced)
* - NNNNNNNNN (bare 9 digits) — only when preceded by an SSN keyword
* to avoid colliding with routing/account numbers handled below.
*/
function redactSSN(value: string): string {
// Canonical and spaced formats (unambiguous)
let result = value.replace(/\b(\d{3})[- ](\d{2})[- ](\d{4})\b/g, REDACTED);
// Bare 9-digit SSN preceded by an SSN keyword
result = result.replace(
/\b(ssn|social\s+security(?:\s+number)?|tax\s+id)\s*[:#]?\s*(\d{9})\b/gi,
(_match, keyword) => `${keyword} ${REDACTED}`,
);
return result;
}
/**
* Mask US ABA routing numbers.
*
* ABA routing numbers are exactly 9 digits. We match them when:
* a) preceded by a routing-number keyword, OR
* b) followed by a routing-number keyword
*
* We do NOT mask bare 9-digit strings without a keyword to avoid clobbering
* zip+4 combos, phone fragments, etc.
*/
function redactRouting(value: string): string {
// Keyword BEFORE the number: "routing number: 021000021"
let result = value.replace(
/\b(routing\s*(?:number|#|no\.?)?|aba\s*(?:number|#|no\.?)?)\s*[:#]?\s*(\d{9})\b/gi,
(_match, keyword) => `${keyword.trim()} ${REDACTED}`,
);
// Keyword AFTER the number: "021000021 (routing)"
result = result.replace(/\b(\d{9})\s*\((routing|aba)\)/gi, `${REDACTED} ($2)`);
return result;
}
/**
* Mask US bank account numbers.
*
* Bank account numbers are 4–17 digits. We mask them only when a keyword
* context makes them unambiguous, to avoid clobbering invoice/PO numbers,
* phone numbers, etc.
*/
function redactAccountNumber(value: string): string {
return value.replace(
/\b(account\s*(?:number|#|no\.?)?|acct\.?\s*(?:#|no\.?)?)\s*[:#]?\s*(\d{4,17})\b/gi,
(_match, keyword) => `${keyword.trim()} ${REDACTED}`,
);
}
/**
* Mask payment card numbers (credit / debit).
*
* Matches 13–19 consecutive digits (with optional spaces or hyphens between
* groups of 4) that look like a PAN. We apply a basic Luhn check so common
* non-card numeric strings (invoice IDs, phone numbers) do not get masked.
*/
function passesLuhn(digits: string): boolean {
let sum = 0;
let alternate = false;
for (let i = digits.length - 1; i >= 0; i--) {
let n = parseInt(digits[i]!, 10);
if (alternate) {
n *= 2;
if (n > 9) n -= 9;
}
sum += n;
alternate = !alternate;
}
return sum % 10 === 0;
}
function redactCard(value: string): string {
// Match 13–19 digits, optionally separated by spaces or hyphens in groups of 4.
return value.replace(/\b(\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{1,7}|\d{13,19})\b/g, (match) => {
const digits = match.replace(/[\s-]/g, '');
if (digits.length < 13 || digits.length > 19) return match;
if (!passesLuhn(digits)) return match;
return REDACTED;
});
}
// ---------------------------------------------------------------------------
// Public API
// ---------------------------------------------------------------------------
/**
* Redact PII from a string that may appear in a tool response.
*
* Applies all pattern redactors in a safe order (SSN first to avoid the bare
* 9-digit pattern conflicting with the routing-number check).
*
* The function is PURE and has no side effects. It never makes network calls.
*
* @param value The raw string (may be JSON, plain text, CSV, etc.)
* @returns A copy of `value` with sensitive fields replaced by `[REDACTED]`.
*/
export function redact(value: string): string {
let result = value;
result = redactSSN(result);
result = redactRouting(result);
result = redactAccountNumber(result);
result = redactCard(result);
return result;
}
/**
* Mask a sensitive field value at the field level.
*
* Use this when you have an already-isolated sensitive field value (e.g. a
* bank account number stored in its own database column) and need to produce
* a display-safe string. Unlike `redact()`, which is designed for inline
* pattern-matching inside free-form text, `maskValue` blindly replaces the
* entire value with masking characters.
*
* Finance-tier tools MUST still call `redact()` on the field as well —
* `maskValue` is a complementary, not a replacement, operation.
*
* @param value The raw sensitive string (e.g. "123456789", "4111-1111-1111-1111").
* @returns A string of `●` characters the same length as `value` (max 16),
* safe to include in tool output or logs.
*/
export function maskValue(value: string): string {
// Show at most 16 mask characters so excessively long values don't bloat output.
return '●'.repeat(Math.min(value.length, 16));
}