feat: sh-openswe-traces — LangSmith bulk-export trace archive

Storage-only SAM stack (S3 + KMS CMK + write-only IAM writer + Secrets Manager
holder) as the S3 destination for LangSmith Bulk Export of Open SWE traces, for
long-horizon auditing and prompt improvement (Athena over Parquet).

- template.yaml: versioned SSE-KMS bucket, access-log bucket, TLS-only policy,
  DEEP_ARCHIVE lifecycle; least-privilege LangSmith writer (bucket-wide PutObject,
  ViaService-scoped KMS, no read/delete).
- bootstrap.yaml: dedicated OIDC deploy role + least-privilege CFN exec role so CI
  never touches the shared execution role.
- CI/CD via reusable ci-python-sam / cd-sam workflows.

IAM passed GPT-4.1 cross-review + /sh-security-review (no blocking findings).
This commit is contained in:
Adam Moussa 2026-07-10 15:45:23 -04:00
commit 0dea585c1a
No known key found for this signature in database
7 changed files with 682 additions and 0 deletions

10
.github/workflows/ci.yaml vendored Normal file
View file

@ -0,0 +1,10 @@
name: CI
on:
pull_request:
branches: [main]
jobs:
ci:
# No Python source in this repo — ruff no-ops on an empty file set and the
# reusable workflow still runs `sam validate --lint` on template.yaml.
uses: Sea-Haven-Industries/.github/.github/workflows/ci-python-sam.yaml@fd60e4c9041784f666ac0fdefb9bec3c7fbf5143 # main

22
.github/workflows/deploy.yaml vendored Normal file
View file

@ -0,0 +1,22 @@
name: Deploy
on:
push:
branches: [main]
permissions:
id-token: write
contents: read
concurrency:
group: deploy
cancel-in-progress: false
jobs:
deploy:
uses: Sea-Haven-Industries/.github/.github/workflows/cd-sam.yaml@fd60e4c9041784f666ac0fdefb9bec3c7fbf5143 # main
with:
stack-name: sh-openswe-traces
cfn-role-arn: arn:aws:iam::328440206208:role/sh-openswe-traces-cfn-exec-role
secrets:
deploy-role-arn: ${{ secrets.AWS_DEPLOY_ROLE_ARN }}
parameter-overrides: ${{ secrets.SAM_PARAMETER_OVERRIDES }}

5
.gitignore vendored Normal file
View file

@ -0,0 +1,5 @@
.aws-sam/
samconfig.toml
*.pyc
__pycache__/
.DS_Store

194
README.md Normal file
View file

@ -0,0 +1,194 @@
# sh-openswe-traces
LangSmith Bulk Export destination for the Open SWE deployment. A **storage-only**
SAM stack — no compute. LangSmith runs the export on its own schedule and writes
Parquet run/trace data into this bucket; we retain it for periodic auditing and
prompt/instruction improvement (query with Athena).
## Why this exists
LangSmith retains traces for ~14 days. To audit agent behavior over longer
horizons and mine it for prompt improvements, we own the data in S3 rather than
paying for extended LangSmith retention. Capture is LangSmith-native Bulk Export;
this repo is just the destination and its access controls.
## Architecture
```
LangSmith (Bulk Export, scheduled LangSmith-side)
│ s3:PutObject (IAM user access key, least-privilege)
▼
s3://sh-openswe-traces/langsmith/… SSE-KMS (alias/sh-openswe-traces)
│ lifecycle: → DEEP_ARCHIVE @ 90d
▼
Athena / manual audit
```
| Resource | Name | Notes |
|---|---|---|
| S3 bucket | `sh-openswe-traces` | BPA all-on, SSE-KMS (default), versioned, TLS-only, `Retain` on delete |
| Log bucket | `sh-openswe-traces-logs` | S3 server access logs (SSE-S3) for read attribution; logs expire 365d |
| KMS CMK | `alias/sh-openswe-traces` | Rotation on; bucket default + writer encrypt through it |
| IAM user | auto-named (tag `sh-openswe-langsmith-export`) | Write-only LangSmith writer; `PutObject` bucket-wide, no read/delete |
| Secret | `sh-openswe/langsmith-export-s3` | Writer's access key (aws/secretsmanager key); populated post-deploy |
The IAM user is **not** given an explicit name so the app stack deploys under
`CAPABILITY_IAM`. It is referenced by ARN.
## Deploy
Two stacks:
- **`bootstrap.yaml`** → `sh-openswe-traces-bootstrap` — the CI IAM roles (OIDC deploy
role + a least-privilege CFN exec role). Deployed **once, manually, under admin**
(`CAPABILITY_NAMED_IAM`); rarely changes. Kept separate so CI never touches the shared
`github-cfn-execution-role`, which is roles-only and can't create this stack's KMS key,
secret, or IAM user.
- **`template.yaml`** → `sh-openswe-traces` — the app (buckets, CMK, writer, secret). CI
(`cd-sam.yaml`) deploys it on merge to `main`, assuming the OIDC deploy role and passing
`sh-openswe-traces-cfn-exec-role` as `--role-arn`.
```bash
# One-time bootstrap (admin):
aws cloudformation deploy --template-file bootstrap.yaml \
--stack-name sh-openswe-traces-bootstrap --capabilities CAPABILITY_NAMED_IAM \
--region us-east-1
# App stack — CI does this on merge to main; for a local/admin deploy:
cp samconfig.toml.example samconfig.toml
sam validate --lint && sam build
sam deploy --capabilities CAPABILITY_IAM --resolve-s3
```
### Wire CI (after bootstrap + repo exist)
- Set repo secret **`AWS_DEPLOY_ROLE_ARN`** to the bootstrap `DeployRoleArn` output
(`arn:aws:iam::328440206208:role/githubdeploy-sh-openswe-traces`).
- `deploy.yaml` already passes `cfn-role-arn = sh-openswe-traces-cfn-exec-role`; the OIDC
trust is pinned to this repo's `main` ref.
### Post-deploy: mint and store the writer access key
The stack creates the IAM user and an empty secret; the access key is minted
out-of-band so it never lands in CloudFormation state. **Run this only on a trusted
single-user workstation, never in CI** — it handles a live credential.
```bash
USER=$(aws cloudformation describe-stacks --stack-name sh-openswe-traces \
--query "Stacks[0].Outputs[?OutputKey=='ExportUserName'].OutputValue" --output text)
KEY_JSON=$(aws iam create-access-key --user-name "$USER" \
| jq '{AccessKeyId: .AccessKey.AccessKeyId, SecretAccessKey: .AccessKey.SecretAccessKey}')
# Pass the secret via stdin, not argv, so it never appears in the process table.
aws secretsmanager put-secret-value \
--secret-id sh-openswe/langsmith-export-s3 \
--secret-string file:///dev/stdin <<<"$KEY_JSON"
unset KEY_JSON
```
### Configure LangSmith Bulk Export
Driven by the LangSmith API (Plus/Enterprise only). Needs `LS_API_KEY` (LangSmith API
key) and `LS_TENANT` (workspace id). Run **after** the writer key is minted/stored — the
destination call validates by test-writing to the bucket.
**1. Create the destination** (creds pulled from Secrets Manager, never pasted):
```bash
CREDS=$(aws secretsmanager get-secret-value --secret-id sh-openswe/langsmith-export-s3 \
--query SecretString --output text)
AKID=$(jq -r .AccessKeyId <<<"$CREDS"); SAK=$(jq -r .SecretAccessKey <<<"$CREDS")
curl -sS -X POST 'https://api.smith.langchain.com/api/v1/bulk-exports/destinations' \
-H 'Content-Type: application/json' -H "X-API-Key: $LS_API_KEY" -H "X-Tenant-Id: $LS_TENANT" \
--data @- <<JSON | jq .
{ "destination_type": "s3", "display_name": "sh-openswe-traces us-east-1",
"config": { "bucket_name": "sh-openswe-traces", "prefix": "langsmith", "region": "us-east-1" },
"credentials": { "access_key_id": "$AKID", "secret_access_key": "$SAK" } }
JSON
```
`display_name` must match `^[a-zA-Z0-9\-_ ']+$` (no parens); omit `endpoint_url` (S3-native,
not GCS/MinIO). Save the returned destination `id`. LangSmith writes header-less, so
bucket-default SSE-KMS encrypts every object under the CMK (confirm via `head-object`).
**2. One scheduled export per tracing project** (this workspace has 4). Get project UUIDs
from `GET /api/v1/sessions`, then:
```bash
export LS_DEST_ID='<destination id>'
PROJECTS=( '<uuid-1>' '<uuid-2>' '<uuid-3>' '<uuid-4>' ) # all 4, or the subset you audit
for PID in "${PROJECTS[@]}"; do
curl -sS -X POST 'https://api.smith.langchain.com/api/v1/bulk-exports' \
-H 'Content-Type: application/json' -H "X-API-Key: $LS_API_KEY" -H "X-Tenant-Id: $LS_TENANT" \
--data @- <<JSON | jq '{id, session_id, status}'
{ "bulk_export_destination_id": "$LS_DEST_ID", "session_id": "$PID",
"start_time": "2026-06-26T00:00:00Z", "interval_hours": 24, "format_version": "v2_beta" }
JSON
done
```
`start_time` ~14d back backfills each project's retained window, then it continues daily.
Keep `inputs`/`outputs` (omit `export_fields`) — they're the point of the audit. Data lands
partitioned per project: `langsmith/export_id=…/…/session_id=<id>/…`.
### Monitor exports
```bash
# Every export in the workspace: id, project, schedule, status
curl -sS 'https://api.smith.langchain.com/api/v1/bulk-exports' \
-H "X-API-Key: $LS_API_KEY" -H "X-Tenant-Id: $LS_TENANT" \
| jq -r '(.bulk_exports // .exports // .)[]
| [.id,
(.session_id // "all_experiments"),
(if .interval_hours then "every \(.interval_hours)h" else "one-off" end),
.status] | @tsv' | column -t
```
- A recurring schedule's own status is `IntervalScheduled`; the daily child exports it
spawns have `CREATED` / `RUNNING` / `COMPLETED` / `FAILED` / `CANCELLED` / `TIMEDOUT`
(child exports carry `source_bulk_export_id`).
- One export's detail: `GET /api/v1/bulk-exports/<id>` — its per-run rows:
`GET /api/v1/bulk-exports/<id>/runs`.
- **Stop a schedule:** `PATCH /api/v1/bulk-exports/<id>` with `{"status":"Cancelled"}`.
Already-spawned child exports must be cancelled separately, and a cancelled job can't be
restarted — create a new one.
## Operations
- **Rotate** the writer access key quarterly: `aws iam create-access-key`, update the
secret + LangSmith destination (`PATCH`/recreate), then delete the old key. Key age is
monitored account-wide by the Security Hub `ACCESS_KEYS_ROTATED` Config rule (flags at
90d) — rotation keeps it compliant.
- **Audit**: point Athena at `s3://sh-openswe-traces/langsmith/` (Parquet). Objects older
than 90 days are in Deep Archive — restore before querying.
- **Read attribution**: object access is logged to `s3://sh-openswe-traces-logs/s3-access/`
(S3 server access logging).
- **Cost**: Deep Archive ≈ $1/TB/mo; expect the archive to dominate storage cost.
## Gotchas (from IAM cross-review)
- **Writer needs `kms:Decrypt`.** SSE-KMS multipart uploads call `kms:Decrypt` at
`CompleteMultipartUpload`; without it every multipart export fails `AccessDenied`.
It's granted and safe — the writer has no `s3:GetObject`, so nothing to exfiltrate.
- **No ACL headers.** The bucket is `BucketOwnerEnforced`; any PutObject carrying an
ACL header is rejected with `AccessControlListNotSupported` (not fixable in policy).
boto3/most SDKs send none by default — verify LangSmith's exporter likewise.
- **Downstream readers need their own CMK grant.** An Athena/Glue role reading the
Parquet needs explicit `kms:Decrypt` (+ `kms:GenerateDataKey`) on
`alias/sh-openswe-traces` — see `reference_kms_cmk_grant_migration`.
## Security
Traces can contain source code and secrets surfaced in tool I/O. Controls: SSE-KMS at
rest (customer-managed CMK, bucket default), versioning (overwrite recovery), Block Public
Access, TLS-only bucket policy, a write-only least-privilege writer (bucket-wide `PutObject`
+ `kms:Decrypt` gated to `kms:ViaService=s3`, no read/delete), the credential secret on a
separate managed key, and S3 access logging. CI deploys via a dedicated least-privilege
exec role (see `bootstrap.yaml`), not the shared execution role.
The IAM surface passed GPT-4.1 cross-review and a `/sh-security-review` fan-out +
proof-or-kill verifier (no blocking findings). Deferred, non-blocking: CloudTrail S3
data-events (org trail carries none; access logging covers attribution for now).

202
bootstrap.yaml Normal file
View file

@ -0,0 +1,202 @@
AWSTemplateFormatVersion: "2010-09-09"
Description: >
Bootstrap IAM for the sh-openswe-traces app stack. Deployed ONCE, manually, under
admin (CAPABILITY_NAMED_IAM). Creates two named roles so CI never touches the shared
github-cfn-execution-role:
- DeployRole (githubdeploy-sh-openswe-traces): assumed by this repo's GitHub
Actions via OIDC (main branch only); can drive CloudFormation for THIS stack and
pass the exec role.
- ExecRole (sh-openswe-traces-cfn-exec-role): assumed by CloudFormation to create
the app stack's resources; least-privilege to exactly this stack's resource set.
Resources:
ExecRole:
Type: AWS::IAM::Role
Properties:
RoleName: sh-openswe-traces-cfn-exec-role
Description: CloudFormation execution role for the sh-openswe-traces app stack.
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: cloudformation.amazonaws.com
Action: sts:AssumeRole
Condition:
StringEquals:
"aws:SourceAccount": !Ref "AWS::AccountId"
Policies:
- PolicyName: manage-sh-openswe-traces-resources
PolicyDocument:
Version: "2012-10-17"
Statement:
# Defense-in-depth: this role can create sh-openswe-traces-* users, so
# explicitly forbid the actions that would turn one into a usable/escalated
# principal (credentials, console login, extra policies, boundary removal).
- Sid: DenyUserCredentialAndEscalation
Effect: Deny
Action:
- "iam:CreateAccessKey"
- "iam:CreateLoginProfile"
- "iam:UpdateLoginProfile"
- "iam:AttachUserPolicy"
- "iam:CreateServiceSpecificCredential"
- "iam:PutUserPermissionsBoundary"
- "iam:DeleteUserPermissionsBoundary"
Resource: "*"
- Sid: Buckets
Effect: Allow
Action:
- "s3:CreateBucket"
- "s3:DeleteBucket"
- "s3:PutBucketPolicy"
- "s3:DeleteBucketPolicy"
- "s3:GetBucketPolicy"
- "s3:PutEncryptionConfiguration"
- "s3:GetEncryptionConfiguration"
- "s3:PutBucketVersioning"
- "s3:GetBucketVersioning"
- "s3:PutBucketPublicAccessBlock"
- "s3:GetBucketPublicAccessBlock"
- "s3:PutBucketOwnershipControls"
- "s3:GetBucketOwnershipControls"
- "s3:PutLifecycleConfiguration"
- "s3:GetLifecycleConfiguration"
- "s3:PutBucketLogging"
- "s3:GetBucketLogging"
- "s3:PutBucketTagging"
- "s3:GetBucketTagging"
- "s3:GetBucketLocation"
- "s3:GetBucketAcl"
Resource:
- "arn:aws:s3:::sh-openswe-traces"
- "arn:aws:s3:::sh-openswe-traces-logs"
# CreateKey/CreateAlias cannot be resource-scoped (the key does not yet
# exist). Only CloudFormation can assume this role, and only to deploy this
# stack, so the blast radius is bounded to this stack's deployments.
- Sid: Kms
Effect: Allow
Action:
- "kms:CreateKey"
- "kms:CreateAlias"
- "kms:DeleteAlias"
- "kms:UpdateAlias"
- "kms:PutKeyPolicy"
- "kms:GetKeyPolicy"
- "kms:EnableKeyRotation"
- "kms:DisableKeyRotation"
- "kms:GetKeyRotationStatus"
- "kms:DescribeKey"
- "kms:TagResource"
- "kms:UntagResource"
- "kms:ListResourceTags"
- "kms:ScheduleKeyDeletion"
- "kms:EnableKey"
Resource: "*"
- Sid: Secret
Effect: Allow
Action:
- "secretsmanager:CreateSecret"
- "secretsmanager:DeleteSecret"
- "secretsmanager:DescribeSecret"
- "secretsmanager:UpdateSecret"
- "secretsmanager:TagResource"
- "secretsmanager:UntagResource"
- "secretsmanager:GetResourcePolicy"
- "secretsmanager:PutResourcePolicy"
Resource: !Sub "arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:sh-openswe/*"
- Sid: ExportUser
Effect: Allow
Action:
- "iam:CreateUser"
- "iam:DeleteUser"
- "iam:GetUser"
- "iam:TagUser"
- "iam:UntagUser"
- "iam:PutUserPolicy"
- "iam:DeleteUserPolicy"
- "iam:GetUserPolicy"
- "iam:ListUserPolicies"
- "iam:ListUserTags"
- "iam:ListAttachedUserPolicies"
- "iam:ListGroupsForUser"
Resource: !Sub "arn:aws:iam::${AWS::AccountId}:user/sh-openswe-traces-*"
DeployRole:
Type: AWS::IAM::Role
Properties:
RoleName: githubdeploy-sh-openswe-traces
Description: GitHub Actions OIDC deploy role for the sh-openswe-traces app stack.
MaxSessionDuration: 3600
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Federated: !Sub "arn:aws:iam::${AWS::AccountId}:oidc-provider/token.actions.githubusercontent.com"
Action: sts:AssumeRoleWithWebIdentity
Condition:
StringEquals:
"token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
"token.actions.githubusercontent.com:sub": "repo:Sea-Haven-Industries/sh-openswe-traces:ref:refs/heads/main"
Policies:
- PolicyName: deploy-sh-openswe-traces
PolicyDocument:
Version: "2012-10-17"
Statement:
- Sid: AppStack
Effect: Allow
Action:
- "cloudformation:CreateChangeSet"
- "cloudformation:ExecuteChangeSet"
- "cloudformation:DescribeChangeSet"
- "cloudformation:DeleteChangeSet"
- "cloudformation:CreateStack"
- "cloudformation:UpdateStack"
- "cloudformation:DescribeStacks"
- "cloudformation:DescribeStackEvents"
- "cloudformation:DescribeStackResource"
- "cloudformation:DescribeStackResources"
- "cloudformation:ListStackResources"
- "cloudformation:GetTemplate"
- "cloudformation:GetTemplateSummary"
Resource:
- !Sub "arn:aws:cloudformation:${AWS::Region}:${AWS::AccountId}:stack/sh-openswe-traces/*"
- !Sub "arn:aws:cloudformation:${AWS::Region}:${AWS::AccountId}:changeSet/*/*"
# SAM's --resolve-s3 looks up (does not recreate) the pre-existing managed
# artifact stack + bucket.
- Sid: SamManagedStackRead
Effect: Allow
Action:
- "cloudformation:DescribeStacks"
Resource: !Sub "arn:aws:cloudformation:${AWS::Region}:${AWS::AccountId}:stack/aws-sam-cli-managed-default/*"
- Sid: SamArtifactBucket
Effect: Allow
Action:
- "s3:GetObject"
- "s3:PutObject"
- "s3:GetBucketLocation"
- "s3:ListBucket"
Resource:
- "arn:aws:s3:::aws-sam-cli-managed-default-*"
- "arn:aws:s3:::aws-sam-cli-managed-default-*/*"
- Sid: CfnValidate
Effect: Allow
Action:
- "cloudformation:ValidateTemplate"
Resource: "*"
- Sid: PassExecRoleToCfn
Effect: Allow
Action: "iam:PassRole"
Resource: !GetAtt ExecRole.Arn
Condition:
StringEquals:
"iam:PassedToService": cloudformation.amazonaws.com
Outputs:
DeployRoleArn:
Description: Set as the repo secret AWS_DEPLOY_ROLE_ARN.
Value: !GetAtt DeployRole.Arn
ExecRoleArn:
Description: Set as cfn-role-arn in .github/workflows/deploy.yaml.
Value: !GetAtt ExecRole.Arn

10
samconfig.toml.example Normal file
View file

@ -0,0 +1,10 @@
version = 0.1
[default.deploy.parameters]
stack_name = "sh-openswe-traces"
region = "us-east-1"
capabilities = "CAPABILITY_IAM"
resolve_s3 = true
confirm_changeset = true
# Optional: override the export prefix
# parameter_overrides = "ExportPrefix=langsmith/"

239
template.yaml Normal file
View file

@ -0,0 +1,239 @@
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
sh-openswe-traces — LangSmith Bulk Export destination. Storage-only:
KMS-encrypted S3 bucket, a least-privilege IAM writer for LangSmith's
export job, and a Secrets Manager holder for that writer's access key.
No compute — the export schedule is configured on the LangSmith side.
Parameters:
ExportPrefix:
Type: String
Default: langsmith/
Description: S3 key prefix LangSmith writes exports under (also the lifecycle scope).
Resources:
TracesKey:
Type: AWS::KMS::Key
Properties:
Description: SSE-KMS CMK for sh-openswe-traces (LangSmith export archive).
EnableKeyRotation: true
KeyPolicy:
Version: "2012-10-17"
Statement:
# Root-enable so IAM identity policies (below) govern access.
- Sid: EnableIAMPolicies
Effect: Allow
Principal:
AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root"
Action: "kms:*"
Resource: "*"
TracesKeyAlias:
Type: AWS::KMS::Alias
Properties:
AliasName: alias/sh-openswe-traces
TargetKeyId: !Ref TracesKey
TracesBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Retain
UpdateReplacePolicy: Retain
# Logging target policy must exist before S3 will accept LoggingConfiguration.
DependsOn: TracesLogBucketPolicy
Properties:
BucketName: sh-openswe-traces
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: aws:kms
KMSMasterKeyID: !Ref TracesKey
BucketKeyEnabled: true
OwnershipControls:
Rules:
- ObjectOwnership: BucketOwnerEnforced
# Tamper recovery: the writer key is write-only (no DeleteObject / no
# DeleteObjectVersion), so a malicious or buggy overwrite creates a noncurrent
# version the prior bytes are recoverable from. Exports write new partitioned
# keys, so noncurrent versions are rare — expire them after 90 days.
VersioningConfiguration:
Status: Enabled
LifecycleConfiguration:
Rules:
- Id: archive-exports-to-deep-archive
Status: Enabled
Prefix: !Ref ExportPrefix
Transitions:
- StorageClass: DEEP_ARCHIVE
TransitionInDays: 90
- Id: expire-noncurrent-versions
Status: Enabled
NoncurrentVersionExpiration:
NoncurrentDays: 90
- Id: abort-incomplete-multipart
Status: Enabled
AbortIncompleteMultipartUpload:
DaysAfterInitiation: 7
LoggingConfiguration:
DestinationBucketName: !Ref TracesLogBucket
LogFilePrefix: s3-access/
TracesBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref TracesBucket
PolicyDocument:
Version: "2012-10-17"
Statement:
- Sid: DenyInsecureTransport
Effect: Deny
Principal: "*"
Action: "s3:*"
Resource:
- !GetAtt TracesBucket.Arn
- !Sub "${TracesBucket.Arn}/*"
Condition:
Bool:
"aws:SecureTransport": "false"
# No SSE-header enforcement Deny. LangSmith's exporter does not send an
# "aws:kms" SSE header, so a "must be aws:kms" Deny blocks its writes — and
# StringNotEqualsIfExists on a Deny also blocks header-less puts (absent key
# evaluates true). Encryption is instead guaranteed by the bucket DEFAULT
# (SSE-KMS with our CMK, applied to every header-less put) plus S3's baseline
# (no object is ever stored unencrypted). If LangSmith explicitly requests
# AES256, that object lands as SSE-S3 rather than CMK — verify post-write
# (head-object) and decide CMK-vs-SSE-S3 if so.
# Server access logging target for TracesBucket — read attribution for the
# secret-bearing archive (the org trail logs no S3 data events). SSE-S3 only:
# S3 log delivery cannot write to an SSE-KMS bucket.
TracesLogBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Retain
UpdateReplacePolicy: Retain
Properties:
BucketName: sh-openswe-traces-logs
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: AES256
OwnershipControls:
Rules:
- ObjectOwnership: BucketOwnerEnforced
LifecycleConfiguration:
Rules:
- Id: expire-access-logs
Status: Enabled
ExpirationInDays: 365
TracesLogBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref TracesLogBucket
PolicyDocument:
Version: "2012-10-17"
Statement:
- Sid: S3ServerAccessLogsWrite
Effect: Allow
Principal:
Service: logging.s3.amazonaws.com
Action: "s3:PutObject"
Resource: !Sub "${TracesLogBucket.Arn}/s3-access/*"
Condition:
ArnLike:
# literal (not !GetAtt) to avoid a cycle with TracesBucket's DependsOn
"aws:SourceArn": "arn:aws:s3:::sh-openswe-traces"
StringEquals:
"aws:SourceAccount": !Ref "AWS::AccountId"
- Sid: DenyInsecureTransport
Effect: Deny
Principal: "*"
Action: "s3:*"
Resource:
- !GetAtt TracesLogBucket.Arn
- !Sub "${TracesLogBucket.Arn}/*"
Condition:
Bool:
"aws:SecureTransport": "false"
# No explicit UserName: an IAM name would require CAPABILITY_NAMED_IAM, but the
# standard cd-sam.yaml reusable workflow deploys with CAPABILITY_IAM only. The
# principal is referenced by ARN (in the secret + LangSmith config), not by name;
# the tag carries the human-facing identifier.
LangSmithExportUser:
Type: AWS::IAM::User
Properties:
Tags:
- Key: Name
Value: sh-openswe-langsmith-export
- Key: purpose
Value: langsmith-bulk-export-writer
Policies:
- PolicyName: langsmith-export-put
PolicyDocument:
Version: "2012-10-17"
Statement:
# Bucket-wide (not prefix-scoped): LangSmith's destination-creation
# validation writes a test object whose key is NOT guaranteed to be
# under ExportPrefix (docs reference a /tmp path), and its documented
# policy scopes PutObject to the whole bucket. This bucket is
# single-purpose, so bucket-wide write is still tightly bounded.
# Deliberately NO s3:GetObject / s3:DeleteObject (both optional per
# LangSmith): omitting them keeps the writer write-only (no exfil,
# no delete). Trade-off: LangSmith skips post-write size verification
# and leaves its small test object behind (harmless).
- Sid: PutExportObjects
Effect: Allow
Action:
- "s3:PutObject"
- "s3:AbortMultipartUpload"
Resource: !Sub "${TracesBucket.Arn}/*"
- Sid: EncryptWithBucketKey
Effect: Allow
Action:
- "kms:GenerateDataKey"
- "kms:Encrypt"
# Required by S3 at CompleteMultipartUpload for SSE-KMS. Safe: the
# writer has no s3:GetObject, so there is no object to decrypt/exfil.
- "kms:Decrypt"
Resource: !GetAtt TracesKey.Arn
# Usable only through S3 — blocks a leaked key from calling kms:Decrypt
# directly against arbitrary ciphertext under this CMK.
Condition:
StringEquals:
"kms:ViaService": !Sub "s3.${AWS::Region}.amazonaws.com"
# Holder only — the real access key is minted post-deploy and written in
# with `aws secretsmanager put-secret-value` (see README). Never in the template.
ExportKeySecret:
Type: AWS::SecretsManager::Secret
Properties:
Name: sh-openswe/langsmith-export-s3
Description: >
Access key for the sh-openswe-langsmith-export IAM user, consumed by
LangSmith Bulk Export. Populated out-of-band post-deploy; rotate quarterly.
# Encrypted with the aws/secretsmanager managed key — deliberately NOT the
# trace CMK, so the credential and the data it protects never share a key the
# writer principal holds any KMS grant on.
Outputs:
BucketName:
Value: !Ref TracesBucket
BucketArn:
Value: !GetAtt TracesBucket.Arn
KmsKeyArn:
Value: !GetAtt TracesKey.Arn
ExportUserName:
Value: !Ref LangSmithExportUser
ExportKeySecretName:
Value: sh-openswe/langsmith-export-s3