sh-mcp/packages/internal-data/src/client.ts
Adam Moussa 22c09e99fe
Some checks are pending
deploy / deploy (push) Waiting to run
Phase 1: runnable MCP + OpenAPI servers (ops + finance) (#3)
* Add shared transport: dispatch, MCP + OpenAPI adapters, local auth

Add the single authoritative tool-execution path (executeTool) plus the two
universal interfaces over it (design.md §2.5, §7.3):
- dispatch.ts: scope enforcement, ajv input validation, rate limiting, finance
  egress redaction (redactDeep), and structured audit emission on one path.
- audit.ts / rate-limit.ts: injected AuditLogger + RateLimiter abstractions.
- mcp.ts: low-level MCP Server with scope-filtered tools/list (tool-hiding) and
  tools/call routed through executeTool.
- http.ts: Express host mounting /mcp, /openapi.json, POST /tools/:name, /healthz.
- openapi.ts: buildOpenApiDocument wraps the existing path generator into a full
  OpenAPI 3.1 document.
- local-auth.ts: LocalAuthProvider (dev bearer tokens) that refuses to construct
  outside SH_MCP_ENV=local and enforces audience binding (design.md §3, §6).

* Add in-memory dev clients; make package tool exports lazy

Add an in-memory Client implementation per integration package (seeded fake
data, no network) selected when SH_MCP_ENV=local (build-plan §4). Gmail/calendar/
tasks dev clients partition by ctx.sub; payments/qbo seed sensitive-looking
fields so the redaction egress path has real targets to mask.

Make the eager default-tool exports in tasks/reminders/qbo LAZY (getDefaultTools)
so importing a package barrel no longer constructs an AWS client at module load
(build-plan §7 'no I/O at import time') — the previous eager construction broke
server startup. Fix payments tsconfig rootDir (src, was '.') so its declarations
resolve under dist/index.d.ts like the other 8 packages.

* Add runnable sh-mcp-ops and sh-mcp-finance servers

Two thin composition-root servers over the shared transport (design.md §3):
- ops: internal-data, knowledge-base, google-maps, gmail, calendar, tasks,
  reminders. finance: qbo, payments (audited + redacted on egress).
- config from env only (no hardcoded ids/issuer/tables); SH_MCP_ENV selects
  LocalAuthProvider + dev clients (local) vs CognitoAuthProvider + real stubs
  (aws). Finance applies the 15-min finance-token TTL ceiling (design.md §2.5).
- index.ts is the only place .listen() is called; a Lambda handler placeholder
  is exported but not depended on.
- synth-only CDK stubs (no real IAM/Cognito/WAF) so 'cdk synth' has a valid app
  (build-plan §6); READMEs document local run, dev tokens, curl, MCP Inspector.

* Add security-weighted test suite + coverage gate; wire tooling

Add tests for the highest-risk surface (build-plan §5, design.md §7.3):
tool-hiding, server-side scope enforcement (incl. forced hidden calls),
audience binding, input-schema validation, finance redaction on egress, audit
emission with hashed args, prompt-injection regression (tool output is data),
rate limiting, MCP conformance (in-memory transport round-trip), OpenAPI 3.1
validity, and local-auth safety. Add HTTP integration tests (supertest) for both
servers and per-package dev-client tests. 405 tests pass.

Wire the coverage gate into vitest.config.ts: 80% overall, with per-file
thresholds on the auth + dispatch crown jewels; exclude deferred real client
stubs, entrypoints, cdk apps, and aws-only config from the gate (documented).
Extend eslint flat config + add .prettierignore to cover servers/. Commit the
updated package-lock.json.

* Suppress pre-existing dev-tooling + out-of-scope scanner findings

Add written-justification suppressions for the 4 confirmed crit/high pre-push
scanner findings, none of which are in this PR's Phase 1 production code:
- npmaudit vitest / @vitest/coverage-v8 / vite: dev/test-only deps that never
  run in the deployed server/Lambda runtime (pins carried from Phase 0b;
  Dependabot will bump).
- gitleaks docs/agentforce-plan.md secret: that file is not on this branch and
  not in this changeset; flagged for the maintainer to scrub on its own branch.

The deep agentic /sh-security-review (required for this auth/authz-touching PR)
was NOT run by the agent and is flagged outstanding in the PR body.

* Address CodeQL findings: bound ajv error work + edge rate limiting

GHAS code-scanning alerts on this PR:
- dispatch.ts (js/resource-exhaustion): ajv ran with allErrors:true on
  untrusted input, letting a crafted payload force unbounded error
  enumeration. Switch to allErrors:false (default) so validation
  short-circuits on the first failure; the 400 still names that path.
- http.ts (js/missing-rate-limiting): the authenticated routes (/mcp,
  /tools/:name) had no edge throttle — auth/JWT verification ran on every
  request before the per-sub dispatch limiter could apply. Add an IP-keyed
  express-rate-limit in front of authenticate (120/60s default, configurable),
  returning the standard 429 shape. Defense-in-depth over the per-sub +
  per-tool limiter in executeTool; API GW/WAF remains the production edge.

Tests: +2 cases proving the edge limiter throttles before auth (429, not
401) on /tools and /mcp. 407 pass; tsc/eslint/prettier clean.

* Fix polynomial ReDoS in Bearer-token extraction (CodeQL js/polynomial-redos)

extractBearerToken matched /^Bearer\s+(.+)$/ — \s and . both match a space,
so the two quantifiers overlap and a crafted header can drive polynomial
backtracking. Require the capture to start with a non-whitespace char
(/^Bearer\s+(\S.*)$/), removing the ambiguity → linear match. Behavior is
unchanged for real tokens; +2 regression tests.

* Harden auth + finance redaction (sh-security-review confirmed mediums)

Two confirmed medium findings from the agentic security review:

- Fail-open SH_MCP_ENV: config defaulted to 'local' when the var was unset,
  so a deploy that forgot SH_MCP_ENV=aws would silently run LocalAuthProvider
  and accept static dev bearer tokens (dev-finance-admin -> finance:admin).
  Now fail-closed: SH_MCP_ENV must be explicitly 'local' or 'aws' or the
  server refuses to start. Plus an independent guard in LocalAuthProvider
  that refuses to construct in an AWS runtime (AWS_LAMBDA_FUNCTION_NAME /
  AWS_EXECUTION_ENV present), regardless of the env flag.

- Finance egress redaction gap: redactDeep only wholesale-masked a sensitive
  key when its value was a scalar; an object/array under a sensitive key was
  recursed into, letting a bare nested value (e.g. {account:{number:...}})
  escape the keyword-gated pattern matcher. Now the entire subtree under a
  sensitive key is masked. No current finance tool emitted such shapes (all
  flat strings), so this closes a latent hole in the universal safety net.

+4 tests (subtree redaction, AWS-runtime guard). 411 pass; coverage gate green.

Review also produced lows (memo free-text digits, unsalted argsHash,
unauth /openapi.json by-design, session-cap no-reset by-design) tracked
separately; 0 confirmed critical/high — review verdict PASS.
2026-06-26 13:33:21 -04:00

186 lines
8.1 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

/**
* DynamoDB client interface + thin implementation.
*
* The interface is what every tool handler receives — callers (including tests)
* inject any object that satisfies it. The real implementation wraps the AWS SDK
* DynamoDB DocumentClient, but the SDK is only instantiated when
* RealDynamoClient.create() is explicitly called; nothing happens at import time
* and no AWS calls are made unless you call a method.
*
* Tables used by this package:
* WorkOrders – work-order records, keyed on `workOrderId` (PK)
* purchase-orders – purchase-order records, keyed on `purchaseOrderId` (PK)
* SiteAssignments – site records, keyed on `siteId` (PK)
*/
// ---------------------------------------------------------------------------
// DynamoDB record shapes returned from each table
// ---------------------------------------------------------------------------
export interface WorkOrderRecord {
workOrderId: string;
title: string;
status: string;
siteId?: string;
assignedTo?: string;
createdAt: string;
updatedAt: string;
description?: string;
[key: string]: unknown;
}
export interface PurchaseOrderRecord {
purchaseOrderId: string;
vendor: string;
status: string;
totalAmount?: number;
currency?: string;
issuedAt: string;
updatedAt: string;
lineItems?: Array<{ description: string; quantity: number; unitPrice: number }>;
[key: string]: unknown;
}
export interface SiteRecord {
siteId: string;
name: string;
address?: string;
region?: string;
status: string;
assignedTechnicians?: string[];
[key: string]: unknown;
}
// ---------------------------------------------------------------------------
// Client interface — inject this everywhere; never import the AWS SDK directly
// ---------------------------------------------------------------------------
export interface InternalDataClient {
getWorkOrder(workOrderId: string): Promise<WorkOrderRecord | null>;
getPurchaseOrder(purchaseOrderId: string): Promise<PurchaseOrderRecord | null>;
getSite(siteId: string): Promise<SiteRecord | null>;
}
// ---------------------------------------------------------------------------
// Real (AWS SDK-backed) implementation
//
// The AWS SDK import lives here — behind this class — so that:
// a) Nothing happens at module load time (no credential resolution, no env reads).
// b) Tests never reach this code; they inject a mock that satisfies the interface.
//
// TODO (DEFERRED auth layer): When the Gateway layer is built, the Lambda execution
// role will supply credentials via the standard AWS environment variables. At that
// point ensure the DocumentClient is constructed with the correct region and that
// the table names are injected via environment variables (WORK_ORDERS_TABLE,
// PURCHASE_ORDERS_TABLE, SITE_ASSIGNMENTS_TABLE) rather than hard-coded.
// ---------------------------------------------------------------------------
export class RealDynamoClient implements InternalDataClient {
// Table names — override via environment variables at Lambda deploy time.
private readonly workOrdersTable: string;
private readonly purchaseOrdersTable: string;
private readonly siteAssignmentsTable: string;
// The DocumentClient is typed as `unknown` here to avoid importing the AWS SDK
// at module scope. It is cast when needed inside each method.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
private readonly ddb: any;
private constructor(
// eslint-disable-next-line @typescript-eslint/no-explicit-any
ddb: any,
workOrdersTable: string,
purchaseOrdersTable: string,
siteAssignmentsTable: string,
) {
this.ddb = ddb;
this.workOrdersTable = workOrdersTable;
this.purchaseOrdersTable = purchaseOrdersTable;
this.siteAssignmentsTable = siteAssignmentsTable;
}
/**
* Factory — the only place the AWS SDK DocumentClient is instantiated.
* Calling this from a Lambda handler (not at module scope) is the correct pattern.
*
* NOTE: @aws-sdk/client-dynamodb and @aws-sdk/lib-dynamodb are intentionally absent
* from package.json until the Lambda runtime bundle is assembled (see TODO above).
* The module specifiers are stored in runtime variables so TypeScript does not attempt
* static module-resolution at build time.
*/
static async create(): Promise<RealDynamoClient> {
// Store specifiers in variables to prevent TypeScript static module resolution.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const dynImport = (s: string): Promise<any> => import(/* @vite-ignore */ s);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const { DynamoDBClient } = (await dynImport('@aws-sdk/client-dynamodb')) as {
DynamoDBClient: new (cfg: { region: string }) => any;
};
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const { DynamoDBDocumentClient } = (await dynImport('@aws-sdk/lib-dynamodb')) as {
DynamoDBDocumentClient: { from: (c: any) => any };
};
const region = process.env['AWS_REGION'] ?? 'us-east-1';
// eslint-disable-next-line @typescript-eslint/no-unsafe-call, @typescript-eslint/no-unsafe-assignment
const raw = new DynamoDBClient({ region });
// eslint-disable-next-line @typescript-eslint/no-unsafe-call, @typescript-eslint/no-unsafe-member-access, @typescript-eslint/no-unsafe-assignment
const ddb = DynamoDBDocumentClient.from(raw);
return new RealDynamoClient(
ddb,
process.env['WORK_ORDERS_TABLE'] ?? 'WorkOrders',
process.env['PURCHASE_ORDERS_TABLE'] ?? 'purchase-orders',
process.env['SITE_ASSIGNMENTS_TABLE'] ?? 'SiteAssignments',
);
}
async getWorkOrder(workOrderId: string): Promise<WorkOrderRecord | null> {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const dynImport = (s: string): Promise<any> => import(/* @vite-ignore */ s);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const { GetCommand } = (await dynImport('@aws-sdk/lib-dynamodb')) as {
GetCommand: new (i: any) => any;
};
// eslint-disable-next-line @typescript-eslint/no-unsafe-call, @typescript-eslint/no-unsafe-assignment
const result = await this.ddb.send(
// eslint-disable-next-line @typescript-eslint/no-unsafe-call
new GetCommand({ TableName: this.workOrdersTable, Key: { workOrderId } }),
);
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
return (result.Item as WorkOrderRecord) ?? null;
}
async getPurchaseOrder(purchaseOrderId: string): Promise<PurchaseOrderRecord | null> {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const dynImport = (s: string): Promise<any> => import(/* @vite-ignore */ s);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const { GetCommand } = (await dynImport('@aws-sdk/lib-dynamodb')) as {
GetCommand: new (i: any) => any;
};
// eslint-disable-next-line @typescript-eslint/no-unsafe-call, @typescript-eslint/no-unsafe-assignment
const result = await this.ddb.send(
// eslint-disable-next-line @typescript-eslint/no-unsafe-call
new GetCommand({ TableName: this.purchaseOrdersTable, Key: { purchaseOrderId } }),
);
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
return (result.Item as PurchaseOrderRecord) ?? null;
}
async getSite(siteId: string): Promise<SiteRecord | null> {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const dynImport = (s: string): Promise<any> => import(/* @vite-ignore */ s);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const { GetCommand } = (await dynImport('@aws-sdk/lib-dynamodb')) as {
GetCommand: new (i: any) => any;
};
// eslint-disable-next-line @typescript-eslint/no-unsafe-call, @typescript-eslint/no-unsafe-assignment
const result = await this.ddb.send(
// eslint-disable-next-line @typescript-eslint/no-unsafe-call
new GetCommand({ TableName: this.siteAssignmentsTable, Key: { siteId } }),
);
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
return (result.Item as SiteRecord) ?? null;
}
}