engineering-handbook/confluence/08-sentry.md

75 lines
4.2 KiB
Markdown
Raw Normal View History

# Sentry
Sentry is our observability platform. Errors, logs, traces, and application metrics from every deployable go to Sentry.
## Projects
- One Sentry project per deployable. A frontend and its backend are separate projects.
- Name the project after the deployable, in kebab-case, for example `expense-approval-bot` and `expense-approval-bot-api`.
- Sea Haven creates projects and DSNs. Ask your technical point of contact for access.
## SDKs
| Deployable | SDK |
|---|---|
| Python Lambda | `sentry-sdk` with `AwsLambdaIntegration(timeout_warning=True)` |
| Node.js Lambda | `@sentry/aws-serverless`, with every handler wrapped in `Sentry.wrapHandler` |
| React SPA | `@sentry/react`, with `Sentry.ErrorBoundary` around the app |
Initialize Sentry once in a shared module that every handler imports, not separately in each function.
## Configuration
- The DSN comes from configuration for each environment. Never hardcode or commit it.
- When the DSN is empty, the SDK does nothing. Local runs, unit tests, and CI run with no DSN and make no calls to Sentry.
- Set `environment` to the deploy environment (`dev` or `prod`), taken from the pipeline, not from a hand-set variable.
- Set `release` to the commit SHA, added at build time in the pipeline.
- The Sentry auth token used to upload source maps is a secret. It lives in GitHub Environment secrets.
## What to send
| Signal | Standard |
|---|---|
| Errors | Unhandled exceptions and failed calls to external services. Tag them with `route`, `upstream`, and a correlation ID where they apply. |
| Logs | `enable_logs=True` (Python) or `enableLogs: true` (JavaScript). Send existing INFO-level and higher logs to Sentry Logs. |
| Tracing | Trace incoming requests and add spans for external calls and database access. Sample 100% in dev; in prod, sample 20% for user-facing apps, or 100% for low-volume jobs. |
| Metrics | Count business outcomes, such as requests created, upstream errors, and cache hits, and record latency distributions. |
| Session replay | Frontends only. Mask all text and inputs and block all media. Sample 10% of sessions in prod, and 100% of sessions that hit an error. |
Do not report user-input 4xx responses as errors. Do not report business alerts that still return success as errors.
## Privacy
This section is mandatory. Sentry must never receive personal data, secrets, or request payloads.
- Set `send_default_pii=False` (Python) or `sendDefaultPii: false` (JavaScript).
- Turn off stack-frame local variables (`include_local_variables=False`) and request bodies (`max_request_body_size="never"`).
- Scrub in every hook: `before_send`, `before_send_transaction`, `before_send_log`, and `before_send_metric`.
- Always strip `Authorization`, `Cookie`, and `Set-Cookie` headers, tokens, API keys, and query strings on authentication routes.
- Never send SSNs, legal IDs, or other personal identifiers in events, logs, traces, or metric attributes. Never use an email address as a metric attribute.
- Replay network capture records URL, status, and timing only. No request or response bodies.
Write unit tests that prove the scrubbing hooks drop these fields.
## Source maps
- The release workflow uploads source maps to Sentry, tagged with the release SHA.
- Do not deploy source maps to the public hosting bucket.
## Alerts and dashboards
- Every project has alerts for new issues and regressions, plus metric alerts for error rate and p95 latency, for each environment. Alerts go to the team's Slack alerts channel.
- Build dashboards in Sentry. Do not build CloudWatch dashboards, CloudWatch custom metrics, or X-Ray tracing for new work.
- Lambda log groups still exist, with 60-day retention, but nothing new is built on top of them.
## Definition of done
A new deployable is not production-ready until:
- [ ] A forced error in dev shows in Sentry with `environment`, `release`, and a readable stack trace
- [ ] Logs for a request appear in Sentry and link to its trace
- [ ] The scrubbing tests pass, and captured dev events contain no headers, tokens, or personal data
- [ ] Local runs and CI make no Sentry calls
- [ ] Alerts fire to Slack for a test issue
- [ ] The README documents the Sentry project, sampling rates, and what is scrubbed