afterhours-shift-manager/SETUP.md

After-Hours Shift Manager — Setup Guide

1. Create a Slack App

1. Go to api.slack.com/apps → Create New App → From scratch
2. Name: After-Hours Shift Manager, pick your workspace
3. Under OAuth & Permissions, add these Bot Token Scopes:
   • chat:write — post messages to channels
   • commands — register slash commands
   • chat:write.public — post to channels the bot isn't in
4. Under Slash Commands, create a new command:
   • Command: /oncall
   • Request URL: (fill in after deploy — see step 4)
   • Short Description: Manage after-hours on-call shifts
   • Usage Hint: [schedule | pick <date> | drop <date> | swap <date> @person | register <ext> | roster | help]
5. Under Interactivity & Shortcuts, toggle ON:
   • Request URL: same URL as the slash command (the /slack/events endpoint)
6. Install to Workspace — approve the permissions
7. Copy the Bot User OAuth Token (xoxb-...) and Signing Secret (under Basic Information)

2. Store Secrets in AWS Secrets Manager

The stack reads these from Secrets Manager (the IAM roles grant secretsmanager:GetSecretValue on afterhours-shift-manager/*):

# Slack
aws secretsmanager create-secret \
  --name afterhours-shift-manager/slack-bot-token \
  --secret-string "xoxb-YOUR-BOT-TOKEN"

aws secretsmanager create-secret \
  --name afterhours-shift-manager/slack-signing-secret \
  --secret-string "YOUR-SIGNING-SECRET"

# 3CX Queue XAPI (used by roster-sync and ring-scheduler)
aws secretsmanager create-secret \
  --name afterhours-shift-manager/3cx-domain \
  --secret-string "yourcompany.3cx.us"

aws secretsmanager create-secret \
  --name afterhours-shift-manager/3cx-client-id \
  --secret-string "YOUR-3CX-CLIENT-ID"

aws secretsmanager create-secret \
  --name afterhours-shift-manager/3cx-client-secret \
  --secret-string "YOUR-3CX-CLIENT-SECRET"

# Roster HTTP API bearer token (plain string). Duplicate the same value into
# seahaven-prod as paychex-integrations/afterhours-roster-token. Generate the
# token into a temp file, pass --secret-string file://..., then delete the file.
# Never paste the value into chat, Terraform, or a PR.
aws secretsmanager create-secret \
  --name afterhours-shift-manager/roster-api-token \
  --secret-string file://./roster-api-token.tmp

Create afterhours-shift-manager/roster-api-token before the first deploy that includes afterhours-roster-api, or live PUT/DELETE calls return 503.

Rotation is coordinated: write the new value to both afterhours-shift-manager/roster-api-token and paychex-integrations/afterhours-roster-token, then recycle afterhours-roster-api so cached execution environments pick it up. Updating only one copy causes 401s. The identity processor AFTERHOURS_BASE_URL is the Terraform output api_origin (origin only, no /roster suffix). Flip that HCP variable on paychex-integrations-prod at cutover after DynamoDB is copied.

Daily roster-sync still removes DynamoDB rows that are not in the 3CX DEFAULT group. Hire stays safe because 3CX create lands the extension in that group before the identity processor PUTs /roster.

The Slack channel ID is not a secret. It is Terraform variable shift_channel (default C0APATP612N), not stored in Secrets Manager.

3. HCP Terraform and GitHub Environment

Workspaces tagged app:afterhours-shift-manager: afterhours-shift-manager-prod in seahaven-prod (account 011934824531) and afterhours-shift-manager-dev in seahaven-dev (account 710827005802). Create the dev workspace before any Slack URL flip. HCP variable environment is prod or dev.

First apply uses the hcptf-bootstrap window (exact StringEquals trust, never StringLike). Creating afterhours-shift-manager-ecs-task-boundary is iam:CreatePolicy and needs that window.

1. Create the HCP workspace. Auto-apply off. No project-level variable set. Working directory terraform. File trigger prefix terraform/** only. Speculative plans on. VCS on main.
2. From seahaven-org-baseline: scripts/create-hcptf-bootstrap-roles.sh --account prod --allow-workspace afterhours-shift-manager-prod
3. Point workspace TFC_AWS_APPLY_ROLE_ARN / TFC_AWS_PLAN_ROLE_ARN at hcptf-bootstrap / hcptf-bootstrap-plan. Set TFC_AWS_PROVIDER_AUTH=true.
4. One manual apply with schedules_enabled=false. This creates the scoped hcptf-* roles, the Lambda boundary, and the rest of the stack. If 3CX secrets already exist from seahaven-door-unlock-api, import those three names instead of creating them: terraform import 'aws_secretsmanager_secret.this["afterhours-shift-manager/3cx-domain"]' afterhours-shift-manager/3cx-domain (and the client-id / client-secret names). Do not overwrite 3CX values.
5. Retarget TFC_AWS_* to hcptf-afterhours-shift-manager / hcptf-afterhours-shift-manager-plan. Re-run the create script with no --allow-workspace.
6. Second manual apply as the scoped role. Then seal auto-apply on.

GitHub Environment prod: reviewers, branch policy main and v*, Environment variable DEPLOY_ROLE_ARN = Terraform output github_deploy_role_arn. GitHub Environment dev: no reviewers, DEPLOY_ROLE_ARN from the seahaven-dev apply of the same output. Set checkcomponents_queue_url and checkcomponents_queue_arn empty on the dev workspace.

Function zips: Actions → Deploy on push to main, or workflow_dispatch. Image CD: Actions → Deploy API (deploy-api.yaml). Keep schedules_enabled=false and ecs_schedules_enabled=false until the Fargate cutover below.

HCP outputs to copy: slack_request_url, api_origin, fargate_origin, holiday_scheduler_role_arn, github_deploy_role_arn, jobs_queue_arn, ecs_task_role_arn. Add ecs_task_role_arn to paychex-checkcomponents (alongside afterhours-shift-manager-weekly-post) before Fargate weekly_post runs. Dual-run keeps the Lambda principal until zip CD is retired.

4. Set the Slack Request URL

Reuse the existing Slack app. After the zip deploy, copy slack_request_url from HCP outputs:

• Slash Commands → edit /oncall → set Request URL to that URL
• Interactivity & Shortcuts → set Request URL to the same URL

Do this in the cutover window, not before DynamoDB is copied.

5. Seed the Schedule

python scripts/seed_schedule.py

This populates the DynamoDB table with the employee roster and default weekly schedule.

6. Invite the Bot & Register Users

1. Create a channel (e.g. #after-hours-shifts) and invite the bot: /invite @After-Hours Shift Manager
2. Each employee links their Slack account by running:

   /oncall register 114
   

   (using their own extension number)

7. Update the 3CX Scheduler (optional)

To have the 3CX scheduler read overrides from DynamoDB (so Slack-driven changes apply to future dates automatically), deploy the updated 3CX scheduler from the feature/dynamodb-shift-integration branch. See that branch's changes for details.

Without this step, the Slack bot still works — it invokes the 3CX scheduler Lambda directly for same-day changes. Future-date overrides would only take effect if the scheduler reads DynamoDB.

8. Prod cutover (PLAT-74)

Avoid Monday 06:00-08:00 ET and any holiday 08:00/17:00 ET window. Dry-run the scripts first (--execute is required for writes).

1. Merge this repo's PR (SAM CD is gone). First HCP apply is the bootstrap window above with schedules_enabled=false.
2. Copy DynamoDB afterhours-shifts mgmt → prod. Verify item counts: python scripts/cutover/copy_dynamodb.py --src-profile mgmt --dst-profile prod then --execute.
3. Confirm secrets in prod. copy_secrets.py writes Slack bot token, Slack signing secret, and roster-api-token into empty Terraform shells and skips dest names that already have a value. It never writes 3CX secrets: python scripts/cutover/copy_secrets.py --src-profile mgmt --dst-profile prod then --execute. Strip trailing newlines is built in.
4. GHA workflow_dispatch (or the merge deploy; re-run if it raced apply) to overwrite stubs.
5. Recreate outstanding future holiday-activate-* / holiday-deactivate-* in prod against the new router ARN and scheduler role: python scripts/cutover/recreate_holiday_schedules.py --src-profile mgmt --dst-profile prod
6. Instant cut: Slack Request URL → prod /slack/events; Paychex HCP variable afterhours_base_url on paychex-integrations-prod → prod api_origin; schedules_enabled=true via a terraform-only merge; disable mgmt EventBridge. Smoke: Slack /oncall, roster PUT/DELETE, weekly-post SendMessage (or simulate-principal-policy plus one smoke message), ring-scheduler invoke, holiday GetSchedule.
7. Seal auto-apply on. Delete the mgmt SAM stack. Remove the mgmt SQS principal from paychex-checkcomponents. Update Confluence AWS Architecture Map and check PLAT-71 item 4.

Do not dual-run 3CX writers. Do not flip afterhours_base_url before DynamoDB is copied.

9. Fargate cutover (PLAT-216)

Completed 2026-09-21. Live path is ECS Fargate behind https://afterhours.seahaven.com (dev: https://afterhours.dev.seahaven.com). Slack Request URL, Paychex AFTERHOURS_BASE_URL, and portal VITE_SHIFTS_API_BASE point at those hosts. Jobs use EventBridge Scheduler → SQS (ecs_schedules_enabled=true, Lambda EventBridge schedules_enabled=false). Public DNS is out of band in the mgmt seahaven.com zone and the external-dev dev.seahaven.com zone.

Commands Reference

Command	Description
/oncall	Show this week's schedule
/oncall next	Show next week's schedule
/oncall pick <date>	Pick up a shift
/oncall drop <date>	Drop your shift (marks it open)
/oncall swap <date> @person	Hand your shift to someone else
/oncall register <ext>	Link your Slack to your extension
/oncall roster	Show all employees and their link status
/oncall help	Show help

Dates can be: today, tomorrow, monday–sunday, 4/5, 2026-04-05