Add Front SLA monitor stack

Scheduled Lambda polls Front every 15 min on weekdays, checks open
conversations for SLA breaches (1hr ack, 1 day action), and sends
tiered Slack alerts — DM the assignee, channel broadcast if unassigned,
or escalate to Adam. DynamoDB dedup prevents repeat notifications.
This commit is contained in:
Adam Moussa 2026-05-11 16:05:38 -04:00
commit cf23b4a1ce
10 changed files with 609 additions and 0 deletions

11
.github/dependabot.yml vendored Normal file
View file

@ -0,0 +1,11 @@
version: 2
updates:
- package-ecosystem: "pip"
directory: "/src/monitor"
schedule:
interval: "weekly"
groups:
minor-and-patch:
update-types:
- "minor"
- "patch"

10
.github/workflows/ci.yaml vendored Normal file
View file

@ -0,0 +1,10 @@
name: CI
on:
pull_request:
branches: [main]
jobs:
ci:
uses: Sea-Haven-Industries/.github/.github/workflows/ci-python-sam.yaml@main
with:
source-dirs: "src"

22
.github/workflows/deploy.yaml vendored Normal file
View file

@ -0,0 +1,22 @@
name: Deploy
on:
push:
branches: [main]
permissions:
id-token: write
contents: read
concurrency:
group: deploy
cancel-in-progress: false
jobs:
deploy:
uses: Sea-Haven-Industries/.github/.github/workflows/cd-sam.yaml@main
with:
stack-name: front-sla-monitor
cfn-role-arn: arn:aws:iam::328440206208:role/github-cfn-execution-role
secrets:
deploy-role-arn: ${{ secrets.AWS_DEPLOY_ROLE_ARN }}
parameter-overrides: ${{ secrets.SAM_PARAMETER_OVERRIDES }}

5
.gitignore vendored Normal file
View file

@ -0,0 +1,5 @@
.aws-sam/
__pycache__/
*.pyc
.env
samconfig.toml

106
README.md Normal file
View file

@ -0,0 +1,106 @@
# front-sla-monitor
Scheduled Lambda that monitors Front conversations for SLA breaches and sends tiered Slack alerts. Runs every 15 minutes on weekdays.
## SLA Rules
| Tier | Threshold | Action |
|---|---|---|
| 1 | 1 business hour without reply | Slack DM the assignee, or post to #front-sla-alerts if unassigned |
| 2 | 1 business day without reply | Slack DM Adam |
Business time counts weekday hours only (Mon-Fri, Eastern time). The SLA clock pauses on Saturday and Sunday.
## Architecture
```
EventBridge (every 15 min, Mon-Fri)
│
▼
Lambda (Python 3.12, arm64)
│
├── Secrets Manager → front-sla-monitor/front-api-token
├── Secrets Manager → front-sla-monitor/slack-bot-token
│
├── GET Front API /inboxes → list shared inboxes
├── GET Front API /inboxes/{id}/conversations → open conversations
│
├── DynamoDB (front-sla-alerts) → dedup: skip already-alerted conversations
│
├── Tier 1 → Slack DM assignee or #front-sla-alerts
└── Tier 2 → Slack DM Adam
```
## AWS Resources
- **Stack:** `front-sla-monitor` (SAM, us-east-1)
- **Lambda:** `front-sla-monitor` — Python 3.12, arm64, 128 MB, 120s timeout, 60-day log retention
- **DynamoDB:** `front-sla-alerts` — tracks alert history per conversation, 7-day TTL
- **EventBridge:** `cron(0/15 * ? * MON-FRI *)` — every 15 min on weekdays
## Setup
### 1. Create the Slack App
1. Go to https://api.slack.com/apps and create **Front SLA Monitor**
2. Add Bot Token Scopes: `chat:write`, `users:read.email`
3. Install the app to your workspace and copy the Bot User OAuth Token
4. Create the `#front-sla-alerts` channel and invite the bot (`/invite @Front SLA Monitor`)
5. Copy the channel ID (right-click channel name > View channel details)
### 2. Create a Front API Token
1. Front > Settings > Developers > API tokens
2. Create a token with conversation read scope
3. Copy the token
### 3. Store Secrets in AWS
```bash
aws secretsmanager create-secret \
--name "front-sla-monitor/front-api-token" \
--secret-string "YOUR_FRONT_API_TOKEN" \
--region us-east-1
aws secretsmanager create-secret \
--name "front-sla-monitor/slack-bot-token" \
--secret-string "xoxb-YOUR-SLACK-BOT-TOKEN" \
--region us-east-1
```
### 4. Deploy
```bash
sam build
sam deploy --guided
```
Or push to `main` to trigger the GitHub Actions deploy workflow.
### 5. GitHub Actions Secrets
| Secret | Value |
|---|---|
| `AWS_DEPLOY_ROLE_ARN` | Org-wide OIDC deploy role (already configured) |
| `SAM_PARAMETER_OVERRIDES` | `FrontApiTokenSecretArn=arn:... SlackBotTokenSecretArn=arn:... SlackAlertChannel=CXXXXXXXXXX` |
## Manual Testing
```bash
aws lambda invoke --function-name front-sla-monitor --payload '{}' /dev/stdout --region us-east-1
```
Check logs:
```bash
aws logs tail /aws/lambda/front-sla-monitor --follow --region us-east-1
```
## Configuration
| Environment Variable | Default | Description |
|---|---|---|
| `ACK_SLA_MINUTES` | 60 | Business minutes before Tier 1 alert |
| `ACTION_SLA_MINUTES` | 1440 | Business minutes before Tier 2 alert |
| `ADAM_EMAIL` | adam@seahavenind.com | Tier 2 escalation recipient |
| `SLACK_ALERT_CHANNEL` | — | Channel ID for broadcast alerts |

17
samconfig.toml.example Normal file
View file

@ -0,0 +1,17 @@
version = 0.1
[default.deploy.parameters]
stack_name = "front-sla-monitor"
resolve_s3 = true
s3_prefix = "front-sla-monitor"
region = "us-east-1"
capabilities = "CAPABILITY_IAM"
confirm_changeset = true
parameter_overrides = [
"FrontApiTokenSecretArn=arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:front-sla-monitor/front-api-token-XXXXXX",
"SlackBotTokenSecretArn=arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:front-sla-monitor/slack-bot-token-XXXXXX",
"SlackAlertChannel=CXXXXXXXXXX",
]
[default.build.parameters]
use_container = false

16
slack-app-manifest.yaml Normal file
View file

@ -0,0 +1,16 @@
display_information:
name: Front SLA Monitor
description: Monitors Front conversations for SLA breaches and sends tiered Slack alerts
background_color: "#d32f2f"
features:
bot_user:
display_name: Front SLA Monitor
always_online: true
oauth_config:
scopes:
bot:
- chat:write
- users:read
- users:read.email

323
src/monitor/app.py Normal file
View file

@ -0,0 +1,323 @@
import json
import logging
import os
import time
from datetime import datetime, timedelta, timezone
from urllib import error, parse, request
from zoneinfo import ZoneInfo
import boto3
logger = logging.getLogger()
logger.setLevel(logging.INFO)
EASTERN = ZoneInfo("America/New_York")
FRONT_BASE = "https://api2.frontapp.com"
SLACK_BASE = "https://slack.com/api"
RATE_LIMIT_DELAY = 0.6
_sm = boto3.client("secretsmanager")
_ddb = boto3.resource("dynamodb")
_table = None
_front_token = None
_slack_token = None
_slack_user_cache = {}
def _get_table():
global _table
if _table is None:
_table = _ddb.Table(os.environ["TABLE_NAME"])
return _table
def _get_front_token():
global _front_token
if _front_token is None:
resp = _sm.get_secret_value(SecretId=os.environ["FRONT_SECRET_NAME"])
_front_token = resp["SecretString"]
return _front_token
def _get_slack_token():
global _slack_token
if _slack_token is None:
resp = _sm.get_secret_value(SecretId=os.environ["SLACK_SECRET_NAME"])
_slack_token = resp["SecretString"]
return _slack_token
# ---------------------------------------------------------------------------
# Front API
# ---------------------------------------------------------------------------
def _front_get(url_or_path, params=None):
if url_or_path.startswith("http"):
url = url_or_path
else:
url = f"{FRONT_BASE}{url_or_path}"
if params:
url += "?" + parse.urlencode(params, doseq=True)
req = request.Request(url, headers={
"Authorization": f"Bearer {_get_front_token()}",
"Accept": "application/json",
})
time.sleep(RATE_LIMIT_DELAY)
try:
with request.urlopen(req) as resp:
return json.loads(resp.read().decode())
except error.HTTPError as e:
body = e.read().decode() if e.fp else ""
raise RuntimeError(f"Front GET {url_or_path} failed ({e.code}): {body}")
def _front_paginate(path, params=None):
results = []
data = _front_get(path, params)
results.extend(data.get("_results", []))
while data.get("_pagination", {}).get("next"):
data = _front_get(data["_pagination"]["next"])
results.extend(data.get("_results", []))
return results
# ---------------------------------------------------------------------------
# Slack API
# ---------------------------------------------------------------------------
def _slack_post(method, payload):
data = json.dumps(payload).encode()
req = request.Request(f"{SLACK_BASE}/{method}", data=data, headers={
"Content-Type": "application/json; charset=utf-8",
"Authorization": f"Bearer {_get_slack_token()}",
})
try:
with request.urlopen(req) as resp:
result = json.loads(resp.read().decode())
if not result.get("ok"):
raise RuntimeError(f"Slack {method}: {result.get('error')}")
return result
except error.HTTPError as e:
body = e.read().decode() if e.fp else ""
raise RuntimeError(f"Slack {method} HTTP {e.code}: {body}")
def _slack_get(method, params):
url = f"{SLACK_BASE}/{method}?" + parse.urlencode(params)
req = request.Request(url, headers={
"Authorization": f"Bearer {_get_slack_token()}",
})
try:
with request.urlopen(req) as resp:
result = json.loads(resp.read().decode())
if not result.get("ok"):
raise RuntimeError(f"Slack {method}: {result.get('error')}")
return result
except error.HTTPError as e:
body = e.read().decode() if e.fp else ""
raise RuntimeError(f"Slack {method} HTTP {e.code}: {body}")
def _resolve_slack_user(email):
if email in _slack_user_cache:
return _slack_user_cache[email]
try:
result = _slack_get("users.lookupByEmail", {"email": email})
uid = result["user"]["id"]
_slack_user_cache[email] = uid
return uid
except RuntimeError:
logger.warning("Could not resolve Slack user for %s", email)
return None
def _send_slack(channel, blocks, text):
_slack_post("chat.postMessage", {
"channel": channel,
"blocks": blocks,
"text": text,
})
# ---------------------------------------------------------------------------
# Business time calculation (weekdays only, Eastern time)
# ---------------------------------------------------------------------------
def _business_minutes_elapsed(since_utc, now_utc):
since = since_utc.astimezone(EASTERN)
now = now_utc.astimezone(EASTERN)
if since >= now:
return 0
total = 0.0
current = since
while current < now:
if current.weekday() < 5:
end_of_day = (current + timedelta(days=1)).replace(
hour=0, minute=0, second=0, microsecond=0
)
day_end = min(end_of_day, now)
total += (day_end - current).total_seconds() / 60
current = (current + timedelta(days=1)).replace(
hour=0, minute=0, second=0, microsecond=0
)
return total
# ---------------------------------------------------------------------------
# DynamoDB dedup
# ---------------------------------------------------------------------------
def _already_alerted(conv_id, tier):
resp = _get_table().get_item(Key={"conversationId": conv_id})
item = resp.get("Item")
if not item:
return False
return f"tier{tier}AlertedAt" in item
def _record_alert(conv_id, tier):
now_iso = datetime.now(timezone.utc).isoformat()
ttl_epoch = int(time.time()) + 7 * 86400
_get_table().update_item(
Key={"conversationId": conv_id},
UpdateExpression="SET #ttl = :ttl, #alert = :ts",
ExpressionAttributeNames={
"#ttl": "ttl",
"#alert": f"tier{tier}AlertedAt",
},
ExpressionAttributeValues={
":ttl": ttl_epoch,
":ts": now_iso,
},
)
# ---------------------------------------------------------------------------
# Slack message blocks
# ---------------------------------------------------------------------------
def _tier1_blocks(conv, assignee_email=None):
subject = conv.get("subject", "No subject")
conv_id = conv.get("id", "")
link = f"https://app.frontapp.com/open/{conv_id}"
msg_at = conv.get("last_message", {}).get("created_at", 0)
ts = datetime.fromtimestamp(msg_at, tz=EASTERN).strftime("%b %d, %I:%M %p ET")
status = f"Assigned to {assignee_email}" if assignee_email else "Unassigned"
return [
{"type": "header", "text": {"type": "plain_text", "text": ":warning: SLA Breach: 1-Hour Acknowledgment"}},
{"type": "section", "text": {"type": "mrkdwn", "text": (
f"*<{link}|{subject}>*\n"
f"Last inbound: {ts}\n"
f"Status: {status}\n"
f"_No reply for over 1 business hour._"
)}},
]
def _tier2_blocks(conv):
subject = conv.get("subject", "No subject")
conv_id = conv.get("id", "")
link = f"https://app.frontapp.com/open/{conv_id}"
msg_at = conv.get("last_message", {}).get("created_at", 0)
ts = datetime.fromtimestamp(msg_at, tz=EASTERN).strftime("%b %d, %I:%M %p ET")
assignee = conv.get("assignee")
status = f"Assigned to {assignee['email']}" if assignee else "Unassigned"
return [
{"type": "header", "text": {"type": "plain_text", "text": ":rotating_light: SLA Breach: 1-Day Action Required"}},
{"type": "section", "text": {"type": "mrkdwn", "text": (
f"*<{link}|{subject}>*\n"
f"Last inbound: {ts}\n"
f"Status: {status}\n"
f"_No reply for over 1 business day. Immediate attention required._"
)}},
]
# ---------------------------------------------------------------------------
# Handler
# ---------------------------------------------------------------------------
def handler(event, context):
ack_threshold = int(os.environ["ACK_SLA_MINUTES"])
action_threshold = int(os.environ["ACTION_SLA_MINUTES"])
alert_channel = os.environ["SLACK_ALERT_CHANNEL"]
adam_email = os.environ["ADAM_EMAIL"]
now = datetime.now(timezone.utc)
inboxes = _front_paginate("/inboxes")
shared = [i for i in inboxes if not i.get("is_private", False)]
logger.info("Found %d shared inboxes", len(shared))
tier1_count = 0
tier2_count = 0
for inbox in shared:
inbox_id = inbox["id"]
inbox_name = inbox.get("name", inbox_id)
try:
conversations = _front_paginate(
f"/inboxes/{inbox_id}/conversations",
params={"q[statuses][]": ["open", "unassigned"]},
)
except RuntimeError as e:
logger.error("Failed to fetch conversations for %s: %s", inbox_name, e)
continue
logger.info("Inbox '%s': %d open conversations", inbox_name, len(conversations))
for conv in conversations:
try:
last_msg = conv.get("last_message")
if not last_msg or last_msg.get("type") != "inbound":
continue
msg_dt = datetime.fromtimestamp(last_msg["created_at"], tz=timezone.utc)
elapsed = _business_minutes_elapsed(msg_dt, now)
conv_id = conv["id"]
assignee = conv.get("assignee")
if elapsed >= action_threshold and not _already_alerted(conv_id, 2):
adam_uid = _resolve_slack_user(adam_email)
target = adam_uid or alert_channel
_send_slack(target, _tier2_blocks(conv),
f"SLA Breach: {conv.get('subject', '')} - 1 day without reply")
_record_alert(conv_id, 2)
tier2_count += 1
logger.info("Tier 2 alert: %s", conv_id)
elif elapsed >= ack_threshold and not _already_alerted(conv_id, 1):
if assignee and assignee.get("email"):
uid = _resolve_slack_user(assignee["email"])
target = uid or alert_channel
_send_slack(target, _tier1_blocks(conv, assignee["email"]),
f"SLA Breach: {conv.get('subject', '')} - 1 hour without reply")
else:
_send_slack(alert_channel, _tier1_blocks(conv),
f"SLA Breach: {conv.get('subject', '')} - unassigned, 1 hour without reply")
_record_alert(conv_id, 1)
tier1_count += 1
logger.info("Tier 1 alert: %s", conv_id)
except Exception:
logger.exception("Error processing conversation %s", conv.get("id", "?"))
result = {"tier1_alerts": tier1_count, "tier2_alerts": tier2_count}
logger.info("Run complete: %s", result)
return result

View file

99
template.yaml Normal file
View file

@ -0,0 +1,99 @@
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Front SLA Monitor - Tiered Slack alerts for conversation SLA breaches
Parameters:
FrontApiTokenSecretArn:
Type: String
Description: ARN of the Secrets Manager secret containing the Front API token
SlackBotTokenSecretArn:
Type: String
Description: ARN of the Secrets Manager secret containing the Slack bot token
SlackAlertChannel:
Type: String
Description: Slack channel ID for the front-sla-alerts channel
AdamEmail:
Type: String
Default: adam@seahavenind.com
Description: Email address for Tier 2 escalation
AckSlaMinutes:
Type: Number
Default: 60
Description: Business minutes before Tier 1 alert (1 hour)
ActionSlaMinutes:
Type: Number
Default: 1440
Description: Business minutes before Tier 2 alert (1 business day)
Globals:
Function:
Runtime: python3.12
Timeout: 120
MemorySize: 128
Architectures:
- arm64
Resources:
AlertsTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: front-sla-alerts
BillingMode: PAY_PER_REQUEST
AttributeDefinitions:
- AttributeName: conversationId
AttributeType: S
KeySchema:
- AttributeName: conversationId
KeyType: HASH
TimeToLiveSpecification:
AttributeName: ttl
Enabled: true
MonitorLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: /aws/lambda/front-sla-monitor
RetentionInDays: 60
MonitorFunction:
Type: AWS::Serverless::Function
DependsOn: MonitorLogGroup
Properties:
FunctionName: front-sla-monitor
Handler: app.handler
CodeUri: src/monitor/
Environment:
Variables:
FRONT_SECRET_NAME: !Ref FrontApiTokenSecretArn
SLACK_SECRET_NAME: !Ref SlackBotTokenSecretArn
SLACK_ALERT_CHANNEL: !Ref SlackAlertChannel
ADAM_EMAIL: !Ref AdamEmail
TABLE_NAME: !Ref AlertsTable
ACK_SLA_MINUTES: !Ref AckSlaMinutes
ACTION_SLA_MINUTES: !Ref ActionSlaMinutes
Policies:
- DynamoDBCrudPolicy:
TableName: !Ref AlertsTable
- Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- secretsmanager:GetSecretValue
Resource:
- !Ref FrontApiTokenSecretArn
- !Ref SlackBotTokenSecretArn
Events:
SlaCheck:
Type: Schedule
Properties:
Schedule: cron(0/15 * ? * MON-FRI *)
Description: Check Front conversations for SLA breaches every 15 min on weekdays
Enabled: true
Outputs:
MonitorFunctionArn:
Description: Front SLA Monitor Lambda ARN
Value: !GetAtt MonitorFunction.Arn
AlertsTableName:
Description: DynamoDB alerts table name
Value: !Ref AlertsTable