Branch re-cut on main 2026-07-23 (old base carried stale PR #99 commits). Contract Rev 2026-07-23: - Producer account corrected: seahaven-prod (011934824531); mgmt frozen - Reconciliation backstop is the new procurement read API, not SyncController - wo_status "unknown" is real; SHOC must map it (checklist item added) - write_origin forward-compat note for phase-2 write-back echo suppression - SyncVendorReplies retirement flagged (dead table, no vendor_reply event) Plan updates: - Account gate: seahaven-prod only; never enable streams on mgmt tables - Emitter ships DARK (ESMs enabled=False); activation is a deliberate flip after the SHOC receiver passes shared HMAC vectors - Post-refactor conventions: common.py helpers, bundle-consistency AST pins, pytest.ini --cov additions, consolidated test roots - Dedicated-CMK rationale, secret-ARN handooff step, consumer audit refreshed (slack-bot decommissioned), enum golden test, write_origin skip-branch test
10 KiB
SHOC Work-Order Webhook — Delivery Contract (v1)
Status: DRAFT — for SHOC team (Luby) review. Rev 2026-07-23 (supersedes the 2026-07-16 draft: producer account corrected to seahaven-prod; reconciliation backstop changed to the new read API; §4.1 unknown status note; §3 write_origin forward-compat note. Sections 2, 5, 6, 7, and 10 are unchanged from the 07-16 draft).
Producer: workorder-shoc-emitter Lambda, Sea Haven seahaven-prod AWS account (011934824531, us-east-1). The former management account (328440206208) is frozen/rollback-only and will never host this feed, its secret, or its streams.
Consumer: SHOC backend (.NET 8), initially https://api.dev.seahaven.com. Endpoint path is SHOC's choice — suggested POST /api/webhooks/work-orders; map entities onto the work-orders domain from shoc-backend PR #10.
1. Overview
Every work-order mutation the procurement-ingest pipeline writes to DynamoDB is pushed to SHOC as an HTTPS POST within seconds. The feed is driven by DynamoDB Streams, so events are emitted in the exact order the pipeline committed them, per work order. DynamoDB remains the source of truth; this webhook is a realtime feed. The reconciliation backstop (and the initial-history load — the feed starts at activation time, not from history) is the procurement read API (GET /work-orders, GET /work-orders/{id}/comments, IAM SigV4 — see its OpenAPI doc). SHOC's existing SyncController DynamoDB scan is transitional: it points at the old management account and retires when those stacks are decommissioned. SyncVendorReplies should be dropped on the SHOC side — the VendorReplies table is dead (its writer was deleted), and no vendor_reply webhook event exists or is planned.
2. Transport
- HTTPS POST,
Content-Type: application/json; charset=utf-8, body ≤ 256 KB. - Producer timeout is 10 seconds. Respond
2xxas fast as possible; if your processing is slow, accept-and-enqueue internally rather than processing inline.
3. Event envelope
{
"schema_version": 1,
"delivery_id": "4b7c2f0e-...",
"event_type": "work_order.updated",
"occurred_at": "2026-07-16T14:03:22.114208+00:00",
"source": "procurement-ingest/workorder-shoc-emitter",
"replay": false,
"data": { ... }
}
| Field | Meaning |
|---|---|
schema_version |
Integer. Breaking payload changes increment it; SHOC should reject versions it doesn't know. |
delivery_id |
Unique per source event, stable across producer retries — your idempotency key. Persist it; ignore any delivery whose id you've already processed. |
event_type |
See §4. |
occurred_at |
ISO 8601 with UTC offset (+00:00), when the pipeline committed the write. |
replay |
true when re-sent by the operator replay tool after an outage. Same idempotency rules apply. |
data |
Event-type-specific body, §4. |
Forward-compat (phase-2 write-back): when SHOC later gains write endpoints on the read API, records SHOC itself wrote will carry a
write_originattribute (e.g."shoc-write-api"), and the emitter will skip them so SHOC never receives an echo of its own write. Nothing to build now — just don't reject envelopes if adata.write_originfield appears later.
4. Event types
4.1 work_order.created / work_order.updated / work_order.cancelled
Emitted from WorkOrders table writes: created on first insert, updated on any subsequent change, cancelled when wo_status transitions to cancelled (a cancelled event is a specialization of updated — same body). data is the full current work-order state (not a diff); fields absent from the source email are null:
{
"work_order_id": "11144580730",
"wo_status": "assigned",
"description": "Dock door 14 won't close",
"customer": "AMAZON",
"site_code": "JFK8",
"building": "JFK8",
"address": "546 Gulf Ave, Staten Island, NY 10314",
"severity": "3-Normal",
"priority": "Medium",
"assigned_to": "Sea Haven Industries",
"date_reported": "2026-07-14T09:12:00",
"scheduled_start": "2026-07-17T08:00:00",
"due_date": "2026-07-21T17:00:00",
"record_type": "new_work_order",
"created_at": "2026-07-16T14:03:22.114208+00:00",
"updated_at": "2026-07-16T14:03:22.114208+00:00"
}
wo_status ∈ new | assigned | in_progress | on_hold | completed | cancelled | unknown. record_type (the triggering email's type) ∈ new_work_order | update | comment | cancellation.
unknownis a real value the extractor emits when the source email doesn't state a status — SHOC must define an explicit mapping for it (and defensively for any unrecognized future value) rather than letting it fall through a switch. Field names and enums in this section mirror the producer's write path,lambdas/wo/email_processor/persistence.py(+prompts.pyfor enums) onmain— that code is the authoritative source if this doc ever drifts.
4.2 work_order.comment_added
Emitted from WorkOrderComments inserts — one per source email (comments, but also update/cancellation event records):
{
"work_order_id": "11144580730",
"comment_id": "11144580730#2026-04-27T23:51:48#a1b2c3d4e5f6",
"record_type": "comment",
"commenter": "APM Technician",
"text": "Vendor dispatched, ETA tomorrow AM.",
"created_at": "2026-04-27T23:51:48",
"ingested_at": "2026-07-16T14:03:22.114208+00:00"
}
comment_id is unique per source email and stable across retries — use it (or delivery_id) for dedupe.
5. Ordering and delivery semantics
- At-least-once. Duplicates are possible on retry; dedupe on
delivery_id. - Per-work-order, per-event-family ordering is guaranteed: all
work_order.*state events for a givenwork_order_idarrive in commit order (acancellednever precedes itscreated). Comment events are likewise ordered among themselves per work order. - Cross-family ordering is NOT guaranteed: a
comment_addedmay occasionally arrive before thecreatedfor its work order (separate streams). Requirement: on a comment for an unknownwork_order_id, upsert a skeleton work order and let the state event backfill it — the same semantics the pipeline itself uses for out-of-order source emails. - Expected volume ≈ 22,900 events/month (~760/day); >90% are comment events. Bursts of a few events/second are possible.
6. Authentication — HMAC signature
Every request carries:
X-SH-Timestamp: 1784642602 (unix seconds, producer clock)
X-SH-Key-Id: 2026-07-20T00
X-SH-Signature: v1=hex(HMAC_SHA256(secret, "{timestamp}.{raw_body}"))
Verification requirements (fail closed — reject with 401 on any failure):
- Reject if
|now - X-SH-Timestamp| > 300s(replay window). - Look up the secret for
X-SH-Key-Idfrom the shared secret material (§6.1). Reject unknown key ids. - Recompute
HMAC-SHA256(secret, timestamp + "." + raw_body)over the raw request bytes (before any JSON parsing/re-serialization) and compare constant-time against thev1=value.
6.1 Key material and rotation
The secret lives in seahaven-prod (011934824531) Secrets Manager: workorder-ingest/shoc-webhook-hmac, encrypted with a dedicated customer-managed KMS key so it is readable cross-account. The SHOC backend role (arn:aws:iam::396287094661:role/shoc-backend-dev) is granted secretsmanager:GetSecretValue + kms:Decrypt via resource policies — exact role ARN only; future shoc-backend-staging/-prod roles are each a deliberate, individually-reviewed policy addition (no wildcard/prefix trust). Secret value shape:
{
"keys": [
{ "kid": "2026-07-20T00", "secret": "<64 hex chars>" },
{ "kid": "2026-06-20T00", "secret": "<64 hex chars>" }
]
}
- Rotation is automated (30-day schedule): a new key is prepended, the previous key is kept for one overlap cycle.
- The producer always signs with
keys[0]. The receiver must accept any listedkid. - Receiver must re-fetch the secret at most every 5 minutes (cache TTL ≤ 300s) so rotation propagates with zero downtime. Also re-fetch immediately on an unknown-
kidbefore rejecting.
7. Response contract
| Receiver response | Producer behavior |
|---|---|
2xx |
Delivered. Body ignored. |
429, 5xx, timeout, connection error |
Retried with in-order blocking (that work order's events queue behind it) for up to 24 hours; then parked for operator replay. |
Any other 4xx |
Not retried — parked immediately and alarmed on our side. A 4xx means a contract bug; expect us to call you. |
8. Backstop / replay
If SHOC is down longer than the retry window or deliveries are parked, an operator replay tool re-sends events rebuilt from DynamoDB (marked "replay": true). Because DynamoDB stays the source of truth, the procurement read API can also fully reconcile at any time (paginated GET /work-orders + per-WO comments). Missed webhooks are therefore an availability inconvenience, never data loss.
9. SHOC-side checklist
- Endpoint path confirmed + implemented against PR #10 entity model
- Raw-body HMAC verification, constant-time compare, ±300s timestamp window, fail-closed
- Cross-account secret fetch with ≤5-min cache + refresh-on-unknown-kid
- Dedupe store on
delivery_id(and/orcomment_id) - Skeleton-upsert on comment-before-create
- Fast
2xxack (<10s), internal queue if processing is slow - Timestamp parsing accepts
+00:00offsets - Reject unknown
schema_version - Explicit mapping for
wo_status: unknown(and a safe default for unrecognized future values) - Initial history loaded via the read API before relying on the feed (feed starts at activation, not from history)
SyncVendorRepliesdropped (dead table; no vendor-reply event exists)
10. Environments
| Env | URL | Status |
|---|---|---|
| dev | https://api.dev.seahaven.com/<path TBD> |
first target (prod data in dev accepted 2026-07-16) |
| staging | https://api.staging.seahaven.com/<path TBD> |
when staging deploys |
| prod | TBD | when SHOC prod exists |
The target URL is producer-side config (per-env), so promotion is a config change, not a code change.