engineering-handbook/secrets-and-config.md

Secrets and Configuration

The Boundary

There is a strict separation between sensitive and non-sensitive configuration. No gray areas.

AWS Secrets Manager

Use Secrets Manager for all sensitive values:

• API access tokens and keys
• Signing values used for request verification
• Webhook URLs that act as implicit authentication
• Database connection strings with embedded passwords
• Any value that would be dangerous if leaked

When in doubt about whether something qualifies as sensitive, treat it as sensitive.

Naming Convention

stack-name/value-name

Examples:

• my-stack/slack-signing
• my-stack/stripe-key

SSM Parameter Store

Use Parameter Store only for non-sensitive configuration:

• Feature flags
• Endpoint URLs (public, non-authenticated)
• Schedule expressions
• Channel IDs and non-sensitive identifiers
• The HCP deploy contract: names Terraform writes and GitHub Actions reads (/<repo>/deploy/* in each account). See hcp-terraform.md.

Do not put build identity (GIT_SHA, release labels) in Terraform-managed Lambda environment variables. Inline it at build time in GitHub Actions so an apply cannot regress the reported version.

Lambda Pattern

1. Store the sensitive value in Secrets Manager
2. Grant the function's IAM role secretsmanager:GetSecretValue scoped to only the values it needs
3. Read the value on cold start via the AWS SDK
4. Cache it in a module-level variable so subsequent invocations reuse it

import boto3
import json

_client = boto3.client('secretsmanager')
_cached = None

def get_config():
    global _cached
    if _cached is None:
        resp = _client.get_secret_value(SecretId='my-stack/config')
        _cached = json.loads(resp['SecretString'])
    return _cached

def handler(event, context):
    config = get_config()
    # use config values

What NOT to Do

• Never use Lambda environment variables for sensitive values. Even with NoEcho CloudFormation parameters, the values end up as plaintext in the Lambda console and are readable by anyone with GetFunctionConfiguration access.
• Never commit .env files containing real values to a repository.
• Never store sensitive values in Notion, Slack messages, or other plaintext documents.