sh-mcp/packages/shared/src/http.ts
Adam Moussa 22c09e99fe
Some checks are pending
deploy / deploy (push) Waiting to run
Phase 1: runnable MCP + OpenAPI servers (ops + finance) (#3)
* Add shared transport: dispatch, MCP + OpenAPI adapters, local auth

Add the single authoritative tool-execution path (executeTool) plus the two
universal interfaces over it (design.md §2.5, §7.3):
- dispatch.ts: scope enforcement, ajv input validation, rate limiting, finance
  egress redaction (redactDeep), and structured audit emission on one path.
- audit.ts / rate-limit.ts: injected AuditLogger + RateLimiter abstractions.
- mcp.ts: low-level MCP Server with scope-filtered tools/list (tool-hiding) and
  tools/call routed through executeTool.
- http.ts: Express host mounting /mcp, /openapi.json, POST /tools/:name, /healthz.
- openapi.ts: buildOpenApiDocument wraps the existing path generator into a full
  OpenAPI 3.1 document.
- local-auth.ts: LocalAuthProvider (dev bearer tokens) that refuses to construct
  outside SH_MCP_ENV=local and enforces audience binding (design.md §3, §6).

* Add in-memory dev clients; make package tool exports lazy

Add an in-memory Client implementation per integration package (seeded fake
data, no network) selected when SH_MCP_ENV=local (build-plan §4). Gmail/calendar/
tasks dev clients partition by ctx.sub; payments/qbo seed sensitive-looking
fields so the redaction egress path has real targets to mask.

Make the eager default-tool exports in tasks/reminders/qbo LAZY (getDefaultTools)
so importing a package barrel no longer constructs an AWS client at module load
(build-plan §7 'no I/O at import time') — the previous eager construction broke
server startup. Fix payments tsconfig rootDir (src, was '.') so its declarations
resolve under dist/index.d.ts like the other 8 packages.

* Add runnable sh-mcp-ops and sh-mcp-finance servers

Two thin composition-root servers over the shared transport (design.md §3):
- ops: internal-data, knowledge-base, google-maps, gmail, calendar, tasks,
  reminders. finance: qbo, payments (audited + redacted on egress).
- config from env only (no hardcoded ids/issuer/tables); SH_MCP_ENV selects
  LocalAuthProvider + dev clients (local) vs CognitoAuthProvider + real stubs
  (aws). Finance applies the 15-min finance-token TTL ceiling (design.md §2.5).
- index.ts is the only place .listen() is called; a Lambda handler placeholder
  is exported but not depended on.
- synth-only CDK stubs (no real IAM/Cognito/WAF) so 'cdk synth' has a valid app
  (build-plan §6); READMEs document local run, dev tokens, curl, MCP Inspector.

* Add security-weighted test suite + coverage gate; wire tooling

Add tests for the highest-risk surface (build-plan §5, design.md §7.3):
tool-hiding, server-side scope enforcement (incl. forced hidden calls),
audience binding, input-schema validation, finance redaction on egress, audit
emission with hashed args, prompt-injection regression (tool output is data),
rate limiting, MCP conformance (in-memory transport round-trip), OpenAPI 3.1
validity, and local-auth safety. Add HTTP integration tests (supertest) for both
servers and per-package dev-client tests. 405 tests pass.

Wire the coverage gate into vitest.config.ts: 80% overall, with per-file
thresholds on the auth + dispatch crown jewels; exclude deferred real client
stubs, entrypoints, cdk apps, and aws-only config from the gate (documented).
Extend eslint flat config + add .prettierignore to cover servers/. Commit the
updated package-lock.json.

* Suppress pre-existing dev-tooling + out-of-scope scanner findings

Add written-justification suppressions for the 4 confirmed crit/high pre-push
scanner findings, none of which are in this PR's Phase 1 production code:
- npmaudit vitest / @vitest/coverage-v8 / vite: dev/test-only deps that never
  run in the deployed server/Lambda runtime (pins carried from Phase 0b;
  Dependabot will bump).
- gitleaks docs/agentforce-plan.md secret: that file is not on this branch and
  not in this changeset; flagged for the maintainer to scrub on its own branch.

The deep agentic /sh-security-review (required for this auth/authz-touching PR)
was NOT run by the agent and is flagged outstanding in the PR body.

* Address CodeQL findings: bound ajv error work + edge rate limiting

GHAS code-scanning alerts on this PR:
- dispatch.ts (js/resource-exhaustion): ajv ran with allErrors:true on
  untrusted input, letting a crafted payload force unbounded error
  enumeration. Switch to allErrors:false (default) so validation
  short-circuits on the first failure; the 400 still names that path.
- http.ts (js/missing-rate-limiting): the authenticated routes (/mcp,
  /tools/:name) had no edge throttle — auth/JWT verification ran on every
  request before the per-sub dispatch limiter could apply. Add an IP-keyed
  express-rate-limit in front of authenticate (120/60s default, configurable),
  returning the standard 429 shape. Defense-in-depth over the per-sub +
  per-tool limiter in executeTool; API GW/WAF remains the production edge.

Tests: +2 cases proving the edge limiter throttles before auth (429, not
401) on /tools and /mcp. 407 pass; tsc/eslint/prettier clean.

* Fix polynomial ReDoS in Bearer-token extraction (CodeQL js/polynomial-redos)

extractBearerToken matched /^Bearer\s+(.+)$/ — \s and . both match a space,
so the two quantifiers overlap and a crafted header can drive polynomial
backtracking. Require the capture to start with a non-whitespace char
(/^Bearer\s+(\S.*)$/), removing the ambiguity → linear match. Behavior is
unchanged for real tokens; +2 regression tests.

* Harden auth + finance redaction (sh-security-review confirmed mediums)

Two confirmed medium findings from the agentic security review:

- Fail-open SH_MCP_ENV: config defaulted to 'local' when the var was unset,
  so a deploy that forgot SH_MCP_ENV=aws would silently run LocalAuthProvider
  and accept static dev bearer tokens (dev-finance-admin -> finance:admin).
  Now fail-closed: SH_MCP_ENV must be explicitly 'local' or 'aws' or the
  server refuses to start. Plus an independent guard in LocalAuthProvider
  that refuses to construct in an AWS runtime (AWS_LAMBDA_FUNCTION_NAME /
  AWS_EXECUTION_ENV present), regardless of the env flag.

- Finance egress redaction gap: redactDeep only wholesale-masked a sensitive
  key when its value was a scalar; an object/array under a sensitive key was
  recursed into, letting a bare nested value (e.g. {account:{number:...}})
  escape the keyword-gated pattern matcher. Now the entire subtree under a
  sensitive key is masked. No current finance tool emitted such shapes (all
  flat strings), so this closes a latent hole in the universal safety net.

+4 tests (subtree redaction, AWS-runtime guard). 411 pass; coverage gate green.

Review also produced lows (memo free-text digits, unsalted argsHash,
unauth /openapi.json by-design, session-cap no-reset by-design) tracked
separately; 0 confirmed critical/high — review verdict PASS.
2026-06-26 13:33:21 -04:00

175 lines
7 KiB
TypeScript

/**
* Express host mounting both universal interfaces over one tool registry.
*
* Routes (build-plan §2.1, design.md §2.5):
* - `POST /mcp` (+ GET/DELETE) — MCP over Streamable HTTP. Authenticated.
* - `GET /openapi.json` — full OpenAPI 3.1 document. Unauthenticated.
* - `POST /tools/:name` — one-shot tool call. Authenticated.
* - `GET /healthz` — liveness. Unauthenticated.
*
* Both `/mcp` and `/tools/:name` require a valid token — there is never an
* unauthenticated tool path (design.md §2.5). Auth runs `authProvider.
* authenticate(req)`; on `AuthError` → 401, otherwise the resolved `AuthContext`
* flows into the SAME `executeTool` dispatch path for both interfaces.
*
* No listener is created here: `createApp` only builds. The server entrypoint
* calls `.listen()` — keeping `@sh-mcp/shared` import-side-effect free
* (build-plan §2.1, §7 "No I/O at import time").
*/
import express, { type Express, type Request, type Response, type NextFunction } from 'express';
import { rateLimit, type RateLimitRequestHandler } from 'express-rate-limit';
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
import { AuthError } from './cognito-auth.js';
import { ScopeError } from './auth.js';
import { RateLimitError } from './rate-limit.js';
import {
executeTool,
InputValidationError,
UnknownToolError,
type DispatchDeps,
} from './dispatch.js';
import { createMcpServer, type McpServerInfo } from './mcp.js';
import { buildOpenApiDocument, type BuildOpenApiOptions } from './openapi.js';
import { visibleTools } from './visibility.js';
import { ToolRegistry } from './registry.js';
import type { AuthProvider } from './auth.js';
import type { AuthContext } from './types.js';
export interface CreateAppOptions {
registry: ToolRegistry;
authProvider: AuthProvider;
deps: DispatchDeps;
/** MCP handshake identity + OpenAPI `info`/`servers`. */
mcpInfo: McpServerInfo;
openApi: BuildOpenApiOptions;
/**
* Per-IP edge rate limit for the authenticated routes (`/mcp`, `/tools/:name`).
* This is a coarse abuse/DoS guard that runs BEFORE `authProvider.authenticate`
* — so it throttles unauthenticated and invalid-token floods before the (more
* expensive) JWT verification and dispatch. It complements, and does not
* replace, the per-`sub` + per-tool limiter inside `executeTool` (design.md
* §7.3). Defaults: 120 requests / 60s / IP. In production the API Gateway/WAF
* is the first edge; this is defense-in-depth.
*/
edgeRateLimit?: { windowMs: number; limit: number };
}
/** Express `Request` augmented with the authenticated context. */
interface AuthedRequest extends Request {
authContext?: AuthContext;
}
/**
* Build (but do not start) the Express app hosting MCP + OpenAPI for one server.
*/
export function createApp(options: CreateAppOptions): Express {
const { registry, authProvider, deps, mcpInfo, openApi } = options;
const app = express();
app.use(express.json());
// --- Per-IP edge rate limiter for the authenticated routes ---
// Runs ahead of auth so request floods are throttled before JWT verification
// and dispatch. Keyed by client IP (express-rate-limit default). Returns the
// same 429 shape adapters use elsewhere; standard RateLimit headers, no legacy.
const { windowMs = 60_000, limit = 120 } = options.edgeRateLimit ?? {};
const edgeRateLimit: RateLimitRequestHandler = rateLimit({
windowMs,
limit,
standardHeaders: 'draft-7',
legacyHeaders: false,
handler: (_req: Request, res: Response) => {
res.status(429).json({ error: 'rate_limited' });
},
});
// --- Unauthenticated liveness ---
app.get('/healthz', (_req: Request, res: Response) => {
res.status(200).json({ status: 'ok' });
});
// --- Unauthenticated full spec ---
// The static document advertises ALL tools (it is the published contract);
// per-call scope enforcement still gates execution server-side.
app.get('/openapi.json', (_req: Request, res: Response) => {
res.status(200).json(buildOpenApiDocument(registry, openApi));
});
// --- Auth middleware for tool paths + MCP ---
const authenticate = async (
req: AuthedRequest,
res: Response,
next: NextFunction,
): Promise<void> => {
try {
req.authContext = await authProvider.authenticate(req);
next();
} catch (err) {
if (err instanceof AuthError) {
res.status(401).json({ error: 'unauthorized', code: err.code });
return;
}
// Unexpected auth failure — do not leak details.
res.status(401).json({ error: 'unauthorized' });
}
};
// --- MCP over Streamable HTTP ---
const handleMcp = async (req: AuthedRequest, res: Response): Promise<void> => {
const ctx = req.authContext!;
// Stateless transport: a fresh server+transport per request (no session
// store needed for this PR). sessionIdGenerator: undefined = stateless mode.
const server = createMcpServer(registry, ctx, deps, mcpInfo);
const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: undefined });
res.on('close', () => {
void transport.close();
void server.close();
});
await server.connect(transport);
await transport.handleRequest(req, res, req.body);
};
app.post('/mcp', edgeRateLimit, authenticate, (req, res) => void handleMcp(req, res));
app.get('/mcp', edgeRateLimit, authenticate, (req, res) => void handleMcp(req, res));
app.delete('/mcp', edgeRateLimit, authenticate, (req, res) => void handleMcp(req, res));
// --- One-shot OpenAPI tool call ---
app.post('/tools/:name', edgeRateLimit, authenticate, (req: AuthedRequest, res: Response) => {
const ctx = req.authContext!;
const rawName = req.params['name'];
const name = Array.isArray(rawName) ? (rawName[0] ?? '') : (rawName ?? '');
void executeTool(registry, ctx, name, req.body ?? {}, deps)
.then((output) => res.status(200).json(output))
.catch((err: unknown) => sendToolError(res, err));
});
return app;
}
/** Translate dispatch errors to HTTP status codes (no stack traces / secrets). */
function sendToolError(res: Response, err: unknown): void {
if (err instanceof UnknownToolError) {
res.status(404).json({ error: 'unknown_tool', tool: err.tool });
return;
}
if (err instanceof ScopeError) {
res.status(403).json({ error: 'forbidden', requiredScope: err.requiredScope });
return;
}
if (err instanceof InputValidationError) {
res.status(400).json({ error: 'invalid_input', issues: err.issues });
return;
}
if (err instanceof RateLimitError) {
if (err.retryAfterMs != null) {
res.setHeader('Retry-After', Math.ceil(err.retryAfterMs / 1000).toString());
}
res.status(429).json({ error: 'rate_limited' });
return;
}
// Generic handler failure — never echo the underlying message.
res.status(500).json({ error: 'internal_error' });
}
/** Re-export so server entrypoints can construct registries/visibility. */
export { ToolRegistry, visibleTools };