feat(infra): migrate syslog-server to HCP Terraform

Replace the mgmt CDK stack with a seahaven-prod HCP workspace so the
collector is owned by Terraform before UniFi cutover.
This commit is contained in:
Adam Moussa 2026-09-16 17:13:00 -04:00
parent a7d7722c89
commit bc85943536
No known key found for this signature in database
27 changed files with 1360 additions and 1826 deletions

View file

@ -8,7 +8,57 @@ permissions:
contents: read
jobs:
terraform:
name: Terraform
runs-on: ubuntu-latest
timeout-minutes: 15
defaults:
run:
working-directory: terraform
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: hashicorp/setup-terraform@dfe3c3f87815947d99a8997f908cb6525fc44e9e # v4.0.1
with:
terraform_version: "1.16.0"
terraform_wrapper: false
- name: Terraform fmt
run: terraform fmt -check -recursive
- name: Terraform init
run: terraform init -backend=false
- name: Terraform validate
run: terraform validate
ci:
uses: Sea-Haven-Industries/.github/.github/workflows/ci-typescript-cdk.yaml@af0f002e14a08cdbfd879c1183bfe7eb2604bce9 # v1.0.8
with:
node-version: "24"
name: ci / ci
needs: [terraform]
if: ${{ always() && !cancelled() }}
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Check jobs
env:
TERRAFORM_RESULT: ${{ needs.terraform.result }}
run: |
set -euo pipefail
fail=0
check() {
local name="$1"
local result="$2"
case "${result}" in
success)
echo "${name}: ${result}"
;;
*)
echo "${name}: ${result}" >&2
fail=1
;;
esac
}
check terraform "${TERRAFORM_RESULT}"
exit "${fail}"

View file

@ -8,10 +8,3 @@ permissions:
jobs:
review:
uses: Sea-Haven-Industries/.github/.github/workflows/callable-dependency-review.yaml@af0f002e14a08cdbfd879c1183bfe7eb2604bce9 # v1.0.8
with:
# GHSA-rgw5-rvv9-x895 (brace-expansion DoS bypass of CVE-2026-14257):
# bundled transitive inside aws-cdk-lib (inBundle, not overridable via
# lockfile); adjudicated in .security-review/suppressions.json.
# aws-cdk-lib 2.263.0 bundles 5.0.8 (fixes GHSA-3jxr-9vmj-r5cp). Remove
# when aws-cdk-lib bundles >=5.0.9.
allow-ghsas: GHSA-rgw5-rvv9-x895

View file

@ -1,20 +0,0 @@
name: Deploy
on:
push:
branches: [main]
permissions:
id-token: write
contents: read
concurrency:
group: deploy
cancel-in-progress: false
jobs:
deploy:
uses: Sea-Haven-Industries/.github/.github/workflows/cd-cdk.yaml@af0f002e14a08cdbfd879c1183bfe7eb2604bce9 # v1.0.8
with:
node-version: "24"
secrets:
deploy-role-arn: ${{ secrets.AWS_DEPLOY_ROLE_ARN }}

15
.gitignore vendored
View file

@ -1,6 +1,13 @@
node_modules/
cdk.out/
*.js
*.d.ts
*.js.map
.terraform/
*.tfstate
*.tfstate.*
crash.log
crash.*.log
override.tf
override.tf.json
*_override.tf
*_override.tf.json
.terraformrc
terraform.rc
.env

View file

@ -1,8 +0,0 @@
{
"suppressions": [
{
"id": "npmaudit-brace-expansion",
"justification": "Bundled transitive of aws-cdk-lib (node_modules/aws-cdk-lib/node_modules/brace-expansion, inBundle: true). aws-cdk-lib 2.263.0 bundles brace-expansion 5.0.8, which clears GHSA-3jxr-9vmj-r5cp / CVE-2026-14257 but remains in range for GHSA-rgw5-rvv9-x895 / CVE-2026-69152 (fixed in >=5.0.9). npm cannot override bundled deps, so no fix is available until upstream rebundles >=5.0.9. Exposure is synth-time only: aws-cdk-lib runs during cdk synth in CI on trusted repo input, and this stack deploys an EC2 syslog host with no Node.js runtime artifacts. This entry also adjudicates the allow-ghsas exception in .github/workflows/dependency-review.yml. Remove this entry AND the allow-ghsas input on the first aws-cdk-lib bump that bundles >=5.0.9 / clears npm audit."
}
]
}

116
README.md
View file

@ -1,86 +1,78 @@
# syslog-server
![TypeScript](https://img.shields.io/badge/TypeScript-3178C6?logo=typescript&logoColor=white)
![AWS CDK](https://img.shields.io/badge/AWS-CDK-FF9900?logo=amazonaws&logoColor=white)
![Terraform](https://img.shields.io/badge/Terraform-844FBA?logo=terraform&logoColor=white)
![AWS](https://img.shields.io/badge/AWS-FF9900?logo=amazonaws&logoColor=white)
![CI](https://github.com/Sea-Haven-Industries/syslog-server/actions/workflows/ci.yaml/badge.svg)
CDK stack for the **syslog-server** EC2 collector: receives remote syslog
(UDP/TCP 514) from the office UniFi fleet over its Elastic IP and ships it to
the `unifi-syslog` CloudWatch Logs group via the CloudWatch agent.
EC2 collector that receives remote syslog (UDP/TCP 514) from the office UniFi
fleet over an Elastic IP and ships it to the `unifi-syslog` CloudWatch Logs
group via the CloudWatch agent.
Brought under IaC for **INFRA-12** (AWS audit L-6). Previously a console/CLI
instance with no drift detection.
Deploy path (PLAT-78): HCP Terraform in seahaven-prod (`011934824531`),
workspace `syslog-server-prod`. CDK CD in mgmt is retired.
## Architecture
```
office UniFi devices ──syslog/514──▶ EIP 184.72.154.32 ──▶ EC2 (rsyslog)
│
/var/log/remote/<host>/*.log
│
CloudWatch agent ──▶ unifi-syslog (90d)
│
Syslog-NoIncomingLogs alarm ──▶ site-alerts
office UniFi devices ──syslog/514──▶ EIP (prod) ──▶ EC2 (rsyslog)
│
/var/log/remote/<host>/*.log
│
CloudWatch agent ──▶ unifi-syslog (90d)
│
Syslog-NoIncomingLogs alarm ──▶ site-alerts
```
| Resource | Value |
|---|---|
| Account / region | seahaven-prod `011934824531` / us-east-1 |
| HCP workspace | `syslog-server-prod` (project `seahaven-prod`; VCS `main`; working dir `terraform`; trigger `terraform/**`) |
| HCP plan/apply roles | `hcptf-syslog-server-plan` / `hcptf-syslog-server` |
| Instance | `syslog-server`, t4g.nano, Amazon Linux 2023 (arm64), 30 GiB encrypted gp3 |
| Subnet | `subnet-0eea820effe1b3ae5` (public, us-east-1a, `vpc-0d3d4b67bd0cf8a68`) |
| Elastic IP | `184.72.154.32` (`eipalloc-006bdefc9802f3285`) — **unmanaged**, re-associated by ID |
| Security group | `syslog-server` — 514 tcp/udp + 22 from office IPs + VPC/VPN CIDRs; 2055/2056 udp reserved (netflow/sflow) |
| IAM role | `syslog-server-role` — `AmazonSSMManagedInstanceCore` + `CloudWatchAgentServerPolicy` |
| Log group | `unifi-syslog` (90-day retention) — created/retained by the CW agent, **not** a CFN resource (holds history; see stack comment) |
| Alarm | `Syslog-NoIncomingLogs` — `IncomingLogEvents` Sum < 1 over 2×1-day, ALARM-only → `site-alerts` |
## CDK app
Infrastructure is a single-stack AWS CDK app written in TypeScript. `cdk.json` is
the app manifest the CDK CLI reads on every command: its `app` entry
(`npx tsx bin/app.ts`) runs the TypeScript entry point directly through `tsx`, so
`synth`/`deploy` need no separate `tsc` compile step. The file also carries the
`watch` globs (for `cdk watch`) and the CDK feature-flag `context`.
| Path | Role |
|---|---|
| `cdk.json` | CDK app manifest — `app` entry command, `watch` globs, feature-flag context |
| `bin/app.ts` | App entry point; instantiates `SyslogServerStack` with explicit `stackName: "syslog-server"` and env pinned to account `328440206208` / `us-east-1` |
| `lib/syslog-server-stack.ts` | The `syslog-server` stack — every resource in the table above (EC2 instance + rsyslog/CloudWatch/NetFlow user-data, security group, IAM role, EIP association, alarms) |
| `cdk.context.json` | Cached provider lookups — the VPC (`vpc-0d3d4b67bd0cf8a68`) and the pinned AL2023 AMI (`cachedInContext`); committed so synth is deterministic |
`aws-cdk-lib` is pinned to an exact version (`2.261.0`). The npm scripts wrap the
CDK CLI — `npm run synth`, `npm run diff`, `npm run deploy` — plus
`npm run build` (`tsc` type-check).
## Documentation
The canonical map of Sea Haven's AWS infrastructure lives in Confluence. This project's `syslog-server` stack is represented there as a Mermaid subgraph.
- **[AWS Architecture Map](https://seahaven.atlassian.net/wiki/spaces/IT/pages/1540098)** (Confluence, IT space, page 1540098)
| VPC | dedicated `10.40.0.0/16`, public subnet `10.40.10.0/24` |
| Elastic IP | Terraform-managed (see HCP output `public_ip`) |
| Security group | `syslog-server` — 514 tcp/udp + 22 from office IPs + VPC/VPN CIDRs; 2055/2056 udp from office |
| Instance IAM | `/tf-managed/syslog-server-role` with `syslog-server-instance-boundary`; `AmazonSSMManagedInstanceCore` + `CloudWatchAgentServerPolicy` |
| Log group | `unifi-syslog` (90-day retention) |
| Alarms | `Syslog-NoIncomingLogs`, `EC2-StatusCheck-syslog-server`, `EC2-StatusCheckSystem-syslog-server-recover` → `site-alerts` |
## Access
SSM Session Manager (no key pair). SSH 22 is open from office/VPC for
break-glass only.
## Deploy
## HCP first apply
CI/CD via the org reusable workflows (`ci-typescript-cdk.yaml`,
`cd-cdk.yaml`); merges to `main` deploy through the `githubdeploy-syslog-server`
OIDC role. No Docker assets, so a local `cdk deploy` is also safe.
First apply uses the hcptf-bootstrap window (exact `StringEquals` trust, never
`StringLike`):
```
npm ci
npm run diff
npm run deploy
```
1. Create the HCP workspace. Auto-apply off. No project-level variable set.
Working directory `terraform`. File trigger prefix `terraform/**` only.
Speculative plans on. VCS on `main`.
2. From `seahaven-org-baseline`:
`scripts/create-hcptf-bootstrap-roles.sh --account prod --allow-workspace syslog-server-prod`
3. Point workspace `TFC_AWS_APPLY_ROLE_ARN` / `TFC_AWS_PLAN_ROLE_ARN` at
`hcptf-bootstrap` / `hcptf-bootstrap-plan`. Set `TFC_AWS_PROVIDER_AUTH=true`.
Never `TFC_AWS_RUN_ROLE_ARN`.
4. One manual apply. This creates the scoped `hcptf-*` roles, the instance
boundary, VPC, instance, EIP, log group, and alarms.
5. Retarget `TFC_AWS_*` to `hcptf-syslog-server` / `hcptf-syslog-server-plan`.
Re-run the create script with no `--allow-workspace`.
6. Second manual apply as the scoped role. After live-path proof, seal
auto-apply on.
## Notes
HCP outputs to copy: `public_ip`, `instance_id`, `hcptf_apply_role_arn`,
`hcptf_plan_role_arn`.
- **EIP is unmanaged.** CloudFormation associates it but never releases it, so
the public forwarding target survives any instance replacement.
- **AMI is pinned in `cdk.context.json`** (`cachedInContext`). An AL2023 AMI
change forces instance replacement — refresh deliberately with
`cdk context --reset <ami key> && cdk synth`.
- To widen device coverage of the forwarded syslog feed, see **INFRA-11**
(UniFi controller remote-logging config).
AMI is pinned in `var.ami_id`. An AMI change forces instance replacement.
User-data changes also replace the instance (the box is stateless; logs live
in CloudWatch; the EIP re-associates).
## Documentation
The canonical map of Sea Haven's AWS infrastructure lives in Confluence.
- **[AWS Architecture Map](https://seahaven.atlassian.net/wiki/spaces/IT/pages/1540098)** (Confluence, IT space, page 1540098)
To widen device coverage of the forwarded syslog feed, see **INFRA-11**
(UniFi controller remote-logging config).

View file

@ -1,11 +0,0 @@
#!/usr/bin/env node
import "source-map-support/register";
import * as cdk from "aws-cdk-lib";
import { SyslogServerStack } from "../lib/syslog-server-stack";
const app = new cdk.App();
new SyslogServerStack(app, "syslog-server", {
stackName: "syslog-server",
env: { account: "328440206208", region: "us-east-1" },
});

View file

@ -1,48 +0,0 @@
{
"vpc-provider:account=328440206208:filter.vpc-id=vpc-0d3d4b67bd0cf8a68:region=us-east-1:returnAsymmetricSubnets=true": {
"vpcId": "vpc-0d3d4b67bd0cf8a68",
"vpcCidrBlock": "10.20.0.0/16",
"ownerAccountId": "328440206208",
"availabilityZones": [],
"vpnGatewayId": "vgw-073737d44762dffc2",
"subnetGroups": [
{
"name": "Private",
"type": "Private",
"subnets": [
{
"subnetId": "subnet-04e38c507e96f1926",
"cidr": "10.20.30.0/24",
"availabilityZone": "us-east-1a",
"routeTableId": "rtb-06a2f56f492b9b4de"
},
{
"subnetId": "subnet-0a0b4fc6f296dfba5",
"cidr": "10.20.40.0/24",
"availabilityZone": "us-east-1b",
"routeTableId": "rtb-01e152fe5cabca7d6"
}
]
},
{
"name": "Public",
"type": "Public",
"subnets": [
{
"subnetId": "subnet-0eea820effe1b3ae5",
"cidr": "10.20.10.0/24",
"availabilityZone": "us-east-1a",
"routeTableId": "rtb-0f2232493a5c43fe8"
},
{
"subnetId": "subnet-0012f5895182c1580",
"cidr": "10.20.20.0/24",
"availabilityZone": "us-east-1b",
"routeTableId": "rtb-0f2232493a5c43fe8"
}
]
}
]
},
"ssm:account=328440206208:parameterName=/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-6.1-arm64:region=us-east-1": "ami-02c114835b4e7739f"
}

View file

@ -1,21 +0,0 @@
{
"app": "npx tsx bin/app.ts",
"watch": {
"include": ["**"],
"exclude": [
"README.md",
"cdk*.json",
"**/*.d.ts",
"**/*.js",
"tsconfig.json",
"package*.json",
"node_modules",
"cdk.out"
]
},
"context": {
"@aws-cdk/aws-lambda:recognizeLayerVersion": true,
"@aws-cdk/core:checkSecretUsage": true,
"@aws-cdk/core:target-partitions": ["aws"]
}
}

View file

@ -1,351 +0,0 @@
import * as cdk from "aws-cdk-lib";
import * as ec2 from "aws-cdk-lib/aws-ec2";
import * as iam from "aws-cdk-lib/aws-iam";
import * as cloudwatch from "aws-cdk-lib/aws-cloudwatch";
import * as cwactions from "aws-cdk-lib/aws-cloudwatch-actions";
import * as sns from "aws-cdk-lib/aws-sns";
import { Construct } from "constructs";
/**
* syslog-server — EC2 collector that receives remote syslog (UDP/TCP 514) from
* the office UniFi fleet over the EIP and ships it to the `unifi-syslog`
* CloudWatch Logs group via the CloudWatch agent.
*
* Brought under IaC for INFRA-12 (AWS audit L-6). Recreated to mirror the
* file-share/forgejo CDK pattern; the existing EIP (184.72.154.32) is imported
* by allocation ID and re-associated so the forwarding target is unchanged.
*
* The `unifi-syslog` log group is intentionally NOT a CloudFormation resource:
* it holds 90 days of history and is created/retained by the CloudWatch agent
* per the user-data config below (log_group_name + retention_in_days). Managing
* it as a CFN resource would either collide with the live group on create or
* risk deleting the history on a future replacement. The agent owns it; this
* stack owns the instance that runs the agent.
*/
export class SyslogServerStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const vpc = ec2.Vpc.fromLookup(this, "SeaHavenVpc", {
vpcId: "vpc-0d3d4b67bd0cf8a68",
});
// Public subnet (IGW route present) — the instance must be internet-facing
// so the office gateways can forward syslog to the EIP.
const publicSubnet = ec2.Subnet.fromSubnetAttributes(this, "PublicSubnet", {
subnetId: "subnet-0eea820effe1b3ae5",
availabilityZone: "us-east-1a",
});
// Office public IPs that forward syslog (see reference_office_ips).
const OFFICE_1 = "47.21.61.4/32";
const OFFICE_2 = "96.250.164.146/32";
const sg = new ec2.SecurityGroup(this, "SecurityGroup", {
vpc,
securityGroupName: "syslog-server",
description: "Syslog collector - rsyslog 514 from office + VPC",
allowAllOutbound: true,
});
// Remote syslog (UDP + TCP 514) from the office public IPs and the internal
// VPC / VPN CIDRs.
for (const proto of [ec2.Port.tcp(514), ec2.Port.udp(514)]) {
sg.addIngressRule(ec2.Peer.ipv4(OFFICE_1), proto, "syslog from office-1");
sg.addIngressRule(ec2.Peer.ipv4(OFFICE_2), proto, "syslog from office-2");
sg.addIngressRule(ec2.Peer.ipv4("10.10.0.0/16"), proto, "syslog from office VPN");
sg.addIngressRule(ec2.Peer.ipv4("10.20.0.0/16"), proto, "syslog from VPC");
sg.addIngressRule(ec2.Peer.ipv4("10.30.0.0/16"), proto, "syslog from VPN pool");
}
// SSH (SSM is the primary access path; 22 kept for break-glass from office/VPC).
sg.addIngressRule(ec2.Peer.ipv4(OFFICE_1), ec2.Port.tcp(22), "SSH from office-1");
sg.addIngressRule(ec2.Peer.ipv4(OFFICE_2), ec2.Port.tcp(22), "SSH from office-2");
sg.addIngressRule(ec2.Peer.ipv4("10.10.0.0/16"), ec2.Port.tcp(22), "SSH from office VPN");
sg.addIngressRule(ec2.Peer.ipv4("10.20.0.0/16"), ec2.Port.tcp(22), "SSH from VPC");
// NetFlow / sFlow ingress reserved from the office IPs. No collector is
// configured in user-data yet; kept to preserve the prior capability.
for (const port of [ec2.Port.udp(2055), ec2.Port.udp(2056)]) {
sg.addIngressRule(ec2.Peer.ipv4(OFFICE_1), port, "netflow/sflow from office-1");
sg.addIngressRule(ec2.Peer.ipv4(OFFICE_2), port, "netflow/sflow from office-2");
}
const role = new iam.Role(this, "InstanceRole", {
roleName: "syslog-server-role",
assumedBy: new iam.ServicePrincipal("ec2.amazonaws.com"),
managedPolicies: [
iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonSSMManagedInstanceCore"),
iam.ManagedPolicy.fromAwsManagedPolicyName("CloudWatchAgentServerPolicy"),
],
});
const userData = ec2.UserData.forLinux();
userData.addCommands(
"set -euxo pipefail",
"",
"# ── 1 GiB swap (build headroom + stability on the 512 MiB t4g.nano) ──",
"if [ ! -f /swapfile ]; then",
" fallocate -l 1G /swapfile || dd if=/dev/zero of=/swapfile bs=1M count=1024",
" chmod 600 /swapfile",
" mkswap /swapfile",
" echo '/swapfile none swap sw 0 0' >> /etc/fstab",
"fi",
"swapon -a || true",
"",
"# ── rsyslog: listen on UDP/TCP 514 ──",
"dnf install -y rsyslog",
"cat > /etc/rsyslog.d/10-listen.conf <<'EOF'",
'module(load="imudp")',
'input(type="imudp" port="514")',
'module(load="imtcp")',
'input(type="imtcp" port="514")',
"EOF",
"",
"# ── Write remote syslog to /var/log/remote/<host>/<program>.log ──",
"cat > /etc/rsyslog.d/20-remote.conf <<'EOF'",
'template(name="RemoteHost" type="string" string="/var/log/remote/%HOSTNAME%/%PROGRAMNAME%.log")',
"if $fromhost-ip != '127.0.0.1' then {",
' action(type="omfile" dynaFile="RemoteHost" createDirs="on")',
" stop",
"}",
"EOF",
"",
"mkdir -p /var/log/remote",
"systemctl enable rsyslog",
"systemctl restart rsyslog",
"",
"# ── Rotate /var/log/remote so it can't grow unbounded ──",
"# CloudWatch (90d) is the system of record; these local files are just a",
"# spool for the CW agent, so keep only a short window. copytruncate keeps",
"# rsyslog's open dynaFile handles valid (truncate in place, same inode).",
"cat > /etc/logrotate.d/remote-syslog <<'EOF'",
"/var/log/remote/*/*.log {",
" daily",
" rotate 7",
" compress",
" delaycompress",
" missingok",
" notifempty",
" copytruncate",
"}",
"EOF",
"",
"# ── CloudWatch agent: ship /var/log/remote/**/*.log to unifi-syslog ──",
"dnf install -y amazon-cloudwatch-agent",
"cat > /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json <<'EOF'",
"{",
' "logs": {',
' "logs_collected": {',
' "files": {',
' "collect_list": [',
" {",
' "file_path": "/var/log/remote/**/*.log",',
' "log_group_name": "unifi-syslog",',
' "log_stream_name": "{hostname}/{file_name}",',
' "retention_in_days": 90',
" }",
" ]",
" }",
" }",
" }",
"}",
"EOF",
"",
"/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \\",
" -a fetch-config -m ec2 \\",
" -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s",
"systemctl enable amazon-cloudwatch-agent",
"",
"# ── NetFlow/IPFIX collectors (nfcapd) ──",
"# nfdump is not packaged for AL2023; build 1.6.23 from source (needs",
"# rrdtool-devel for librrd). Reconstructed under IaC for INFRA-12 — the",
"# original instance ran these as hand-installed systemd units. Captures",
"# are local-only (no consumer/shipping today); 30-day retention enforced.",
"dnf install -y gcc gcc-c++ make automake autoconf libtool flex bison libpcap-devel zlib-devel bzip2-devel rrdtool-devel tar",
"NFVER=1.6.23",
"curl -sfL https://github.com/phaag/nfdump/archive/refs/tags/v${NFVER}.tar.gz | tar xz -C /tmp",
"( cd /tmp/nfdump-${NFVER} && ./autogen.sh && ./configure && make -j1 && make install )",
"ldconfig",
"",
"mkdir -p /var/log/netflow/ronkonkoma /var/log/netflow/locust",
"chown -R ec2-user:ec2-user /var/log/netflow",
"",
"# Ronkonkoma gateway -> UDP 2055",
"cat > /etc/systemd/system/nfcapd.service <<'EOF'",
"[Unit]",
"Description=nfcapd NetFlow collector (Ronkonkoma, udp/2055)",
"After=network.target",
"[Service]",
"Type=simple",
"User=ec2-user",
"ExecStart=/usr/local/bin/nfcapd -p 2055 -l /var/log/netflow/ronkonkoma",
"Restart=always",
"[Install]",
"WantedBy=multi-user.target",
"EOF",
"",
"# Locust Ave gateway -> UDP 2056",
"cat > /etc/systemd/system/nfcapd-locust.service <<'EOF'",
"[Unit]",
"Description=nfcapd NetFlow collector (Locust Ave, udp/2056)",
"After=network.target",
"[Service]",
"Type=simple",
"User=ec2-user",
"ExecStart=/usr/local/bin/nfcapd -p 2056 -l /var/log/netflow/locust",
"Restart=always",
"[Install]",
"WantedBy=multi-user.target",
"EOF",
"",
"# 30-day retention sweep (daily 03:30 UTC)",
"cat > /usr/local/sbin/netflow-retention.sh <<'EOF'",
"#!/bin/bash",
"find /var/log/netflow -type f -name 'nfcapd.*' -mtime +30 -delete",
"EOF",
"chmod +x /usr/local/sbin/netflow-retention.sh",
"cat > /etc/systemd/system/netflow-retention.service <<'EOF'",
"[Unit]",
"Description=Delete NetFlow captures older than 30 days",
"[Service]",
"Type=oneshot",
"ExecStart=/usr/local/sbin/netflow-retention.sh",
"EOF",
"cat > /etc/systemd/system/netflow-retention.timer <<'EOF'",
"[Unit]",
"Description=Daily NetFlow retention sweep",
"[Timer]",
"OnCalendar=*-*-* 03:30:00 UTC",
"Persistent=true",
"[Install]",
"WantedBy=timers.target",
"EOF",
"",
"systemctl daemon-reload",
"systemctl enable --now nfcapd.service nfcapd-locust.service netflow-retention.timer",
);
const instance = new ec2.Instance(this, "Instance", {
instanceName: "syslog-server",
vpc,
vpcSubnets: { subnets: [publicSubnet] },
instanceType: ec2.InstanceType.of(ec2.InstanceClass.T4G, ec2.InstanceSize.NANO),
machineImage: ec2.MachineImage.latestAmazonLinux2023({
cpuType: ec2.AmazonLinuxCpuType.ARM_64,
// Cache the resolved AMI in cdk.context.json so deploys don't implicitly
// pick up new AL2023 releases (AMI change forces instance replacement).
// Refresh deliberately: cdk context --reset <ami key> && cdk synth
cachedInContext: true,
}),
securityGroup: sg,
role,
userData,
// A user-data change must actually re-run, so force instance replacement
// (the box is stateless — logs live in CloudWatch, the EIP re-associates).
userDataCausesReplacement: true,
blockDevices: [
{
deviceName: "/dev/xvda",
volume: ec2.BlockDeviceVolume.ebs(30, {
volumeType: ec2.EbsDeviceVolumeType.GP3,
encrypted: true,
}),
},
],
});
// Re-associate the existing Elastic IP (184.72.154.32) so the UniFi fleet's
// forwarding target is unchanged. The allocation is UNMANAGED (referenced by
// ID) — CloudFormation can associate it but never release it.
new ec2.CfnEIPAssociation(this, "EipAssociation", {
allocationId: "eipalloc-006bdefc9802f3285",
instanceId: instance.instanceId,
});
// ALARM-only "no incoming logs" alarm to the shared site-alerts topic
// (alias/seahaven-alarm-topics CMK). Mirrors the prior standalone alarm:
// IncomingLogEvents (Sum) < 1 over two 1-day periods. 2-day window tolerates
// quiet weekends; treatMissingData=breaching catches a dead pipeline.
const alarmTopic = sns.Topic.fromTopicArn(
this, "SiteAlerts", "arn:aws:sns:us-east-1:328440206208:site-alerts",
);
const noLogsAlarm = new cloudwatch.Alarm(this, "NoIncomingLogsAlarm", {
alarmName: "Syslog-NoIncomingLogs",
alarmDescription:
"No log events delivered to unifi-syslog for 2 days — syslog pipeline may be down.",
metric: new cloudwatch.Metric({
namespace: "AWS/Logs",
metricName: "IncomingLogEvents",
dimensionsMap: { LogGroupName: "unifi-syslog" },
statistic: "Sum",
period: cdk.Duration.days(1),
}),
threshold: 1,
comparisonOperator: cloudwatch.ComparisonOperator.LESS_THAN_THRESHOLD,
evaluationPeriods: 2,
treatMissingData: cloudwatch.TreatMissingData.BREACHING,
});
noLogsAlarm.addAlarmAction(new cwactions.SnsAction(alarmTopic));
// Primary EC2 status-check alarm — pages on-call when the box is hung or
// unreachable. Uses the combined StatusCheckFailed metric so it covers BOTH
// instance and system failures. Dimension is instance.instanceId (Ref), not
// a literal id, so the alarm tracks the CFN-managed instance across future
// replacements (e.g. userDataCausesReplacement above) — this is the durable
// fix for the orphaned EC2-StatusCheck-syslog-server alarm that pointed at a
// since-terminated instance. Maximum>=1 over two 5-min periods;
// treatMissingData=breaching so a metric gap (instance gone/not reporting)
// also fires.
const statusCheckAlarm = new cloudwatch.Alarm(this, "StatusCheckFailedAlarm", {
alarmName: "EC2-StatusCheck-syslog-server",
alarmDescription:
"syslog-server EC2 status check failed (instance and/or system) for 10 min — host may be hung or unreachable.",
metric: new cloudwatch.Metric({
namespace: "AWS/EC2",
metricName: "StatusCheckFailed",
dimensionsMap: { InstanceId: instance.instanceId },
statistic: "Maximum",
period: cdk.Duration.seconds(300),
}),
threshold: 1,
comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD,
evaluationPeriods: 2,
treatMissingData: cloudwatch.TreatMissingData.BREACHING,
});
statusCheckAlarm.addAlarmAction(new cwactions.SnsAction(alarmTopic));
// Auto-recovery alarm — migrates the instance onto healthy hardware on an
// underlying host failure (instance id, EIP association and EBS volume are
// preserved). AWS only permits the RECOVER action on StatusCheckFailed_System
// (NOT the combined StatusCheckFailed / _Instance), so this is a separate
// alarm. Per AWS guidance for recovery alarms, missing data is treated as
// NOT breaching to avoid a spurious recover on transient INSUFFICIENT_DATA,
// and evaluation periods differ from any reboot alarm to avoid a race.
const systemRecoverAlarm = new cloudwatch.Alarm(this, "StatusCheckSystemRecoverAlarm", {
alarmName: "EC2-StatusCheckSystem-syslog-server-recover",
alarmDescription:
"syslog-server EC2 system status check failed — underlying host impaired; auto-recovering onto new hardware.",
metric: new cloudwatch.Metric({
namespace: "AWS/EC2",
metricName: "StatusCheckFailed_System",
dimensionsMap: { InstanceId: instance.instanceId },
statistic: "Maximum",
period: cdk.Duration.seconds(300),
}),
threshold: 1,
comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD,
evaluationPeriods: 2,
treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING,
});
systemRecoverAlarm.addAlarmAction(new cwactions.SnsAction(alarmTopic));
systemRecoverAlarm.addAlarmAction(new cwactions.Ec2Action(cwactions.Ec2InstanceAction.RECOVER));
new cdk.CfnOutput(this, "InstanceId", { value: instance.instanceId });
new cdk.CfnOutput(this, "PublicIp", {
value: "184.72.154.32",
description: "Elastic IP — UniFi remote-syslog forwarding target",
});
}
}

1242
package-lock.json generated

File diff suppressed because it is too large Load diff

View file

@ -1,25 +0,0 @@
{
"name": "syslog-server",
"version": "1.0.0",
"bin": {
"app": "bin/app.js"
},
"scripts": {
"build": "tsc",
"cdk": "cdk",
"synth": "cdk synth",
"deploy": "cdk deploy",
"diff": "cdk diff"
},
"devDependencies": {
"@types/node": "^26.2.0",
"aws-cdk": "^2.1138.0",
"source-map-support": "^0.5.21",
"tsx": "4.23.12",
"typescript": "~7.0.2"
},
"dependencies": {
"aws-cdk-lib": "2.266.0",
"constructs": "^10.8.1"
}
}

4
renovate.json Normal file
View file

@ -0,0 +1,4 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["local>Sea-Haven-Industries/.github"]
}

26
terraform/.terraform.lock.hcl generated Normal file
View file

@ -0,0 +1,26 @@
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
provider "registry.terraform.io/hashicorp/aws" {
version = "6.64.0"
constraints = "~> 6.64"
hashes = [
"h1:wXARLY+IeQ7ufYxCLTPCwToWGMRvOpiOTfJS97iwUzI=",
"zh:07172315d67bc9781240272759cdfc7bd32b7e72384a56862c2c1da3cca99a81",
"zh:154ce7d2659de9a59ddfe96d7cab41a9ddc2cb267a7d4bcdf4e737ff2ffdec06",
"zh:17324d4335a7a7ac01cc23eded530775606680ff53b47cb74a3cb95d1121f836",
"zh:307ab92324ec5a61b124881ab8cac1d9e316f4527dfd0e1b59794c229407eb4e",
"zh:31e25f1903661332e36a95283042dd3ec50b47c186db00663fbd976a11e6a6b2",
"zh:3311d9f3bd12a24886027dbe73859dcd1e67bd0e3046227a338cf2c7ca04d18e",
"zh:37916156a3aac3b29be3acebd15d53145ea4ab5d4aaa825eaebe75481fa00500",
"zh:4158cb8c38b3ac6aa98eb15935ec6bd7c30838d85d2b00acc9812df8382ae908",
"zh:5bfb9499c66d9db5b34dc5c60f426a1ab1baa5457ce2aefebca826a9c3f92fb0",
"zh:6eb29ead5a4aca3b1f35812e7e8c75419180e1928e479b458f206861277736db",
"zh:7a82b6dd0c0cdef8045a4adfbddd36acb86b6b23fcbed8e189c2d71f7dc4a502",
"zh:9556bd792032c3f7e73ea4dd08cec88dc1327f5a4a57d79c30ba844ae2b9a3c0",
"zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425",
"zh:c5234180464cb800c83a41f57462742b802c150ad7d4417626fcd9cb511c01d2",
"zh:cd776b83b1f7b36635957350afe7ce28ba4e4ea3a5e2deb00d13dbd3b35d9d40",
"zh:fb583a7b791c6f915b86573d04f05ddbf7f1a5e4120c5d8a7450a3086c1225c4",
]
}

56
terraform/alarms.tf Normal file
View file

@ -0,0 +1,56 @@
resource "aws_cloudwatch_metric_alarm" "no_incoming_logs" {
alarm_name = "Syslog-NoIncomingLogs"
alarm_description = "No log events delivered to unifi-syslog for 2 days — syslog pipeline may be down."
comparison_operator = "LessThanThreshold"
evaluation_periods = 2
metric_name = "IncomingLogEvents"
namespace = "AWS/Logs"
period = 86400
statistic = "Sum"
threshold = 1
treat_missing_data = "breaching"
alarm_actions = [local.site_alerts_arn]
dimensions = {
LogGroupName = local.log_group_name
}
}
resource "aws_cloudwatch_metric_alarm" "status_check" {
alarm_name = "EC2-StatusCheck-syslog-server"
alarm_description = "syslog-server EC2 status check failed (instance and/or system) for 10 min — host may be hung or unreachable."
comparison_operator = "GreaterThanOrEqualToThreshold"
evaluation_periods = 2
metric_name = "StatusCheckFailed"
namespace = "AWS/EC2"
period = 300
statistic = "Maximum"
threshold = 1
treat_missing_data = "breaching"
alarm_actions = [local.site_alerts_arn]
dimensions = {
InstanceId = aws_instance.this.id
}
}
resource "aws_cloudwatch_metric_alarm" "system_recover" {
alarm_name = "EC2-StatusCheckSystem-syslog-server-recover"
alarm_description = "syslog-server EC2 system status check failed — underlying host impaired; auto-recovering onto new hardware."
comparison_operator = "GreaterThanOrEqualToThreshold"
evaluation_periods = 2
metric_name = "StatusCheckFailed_System"
namespace = "AWS/EC2"
period = 300
statistic = "Maximum"
threshold = 1
treat_missing_data = "notBreaching"
alarm_actions = [
local.site_alerts_arn,
"arn:aws:automate:${var.aws_region}:ec2:recover",
]
dimensions = {
InstanceId = aws_instance.this.id
}
}

39
terraform/ec2.tf Normal file
View file

@ -0,0 +1,39 @@
resource "aws_instance" "this" {
ami = var.ami_id
instance_type = "t4g.nano"
subnet_id = aws_subnet.public.id
vpc_security_group_ids = [aws_security_group.this.id]
iam_instance_profile = aws_iam_instance_profile.this.name
user_data = file("${path.module}/user_data.sh")
user_data_replace_on_change = true
root_block_device {
volume_size = 30
volume_type = "gp3"
encrypted = true
}
metadata_options {
http_endpoint = "enabled"
http_tokens = "required"
}
tags = {
Name = "syslog-server"
}
}
resource "aws_eip" "this" {
domain = "vpc"
tags = {
Name = "syslog-server"
}
depends_on = [aws_internet_gateway.this]
}
resource "aws_eip_association" "this" {
instance_id = aws_instance.this.id
allocation_id = aws_eip.this.id
}

570
terraform/hcp_iam.tf Normal file
View file

@ -0,0 +1,570 @@
# HCP plan/apply roles for syslog-server-prod (PLAT-78 / PLAT-144).
# Copy of seahaven-org-baseline/examples/hcptf-workspace-iam/hcp_iam.tf.example
# with the syslog-server EC2 service set. Create, do not import.
#
# Live seahaven-hcptf-iam-management DenySelfMutation blocks DetachRolePolicy
# and PutRolePolicy on hcptf-* (including this role). First-apply sequence:
# 1. seahaven-org-baseline scripts/create-hcptf-bootstrap-roles.sh
# --account prod --allow-workspace syslog-server-prod
# 2. Point this workspace's TFC_AWS_* at hcptf-bootstrap /
# hcptf-bootstrap-plan (workspace vars, never a project set).
# 3. One Manual apply (create roles + scoped inline + boundary + stack).
# 4. Point TFC_AWS_* back at hcptf-syslog-server / hcptf-syslog-server-plan.
# 5. Re-run the script without --allow-workspace to pin trust back to
# iam-bootstrap-prod only.
# Later apply-role IAM edits use the same window. Do not add StringLike
# on bootstrap trust. CreatePolicy stays on hcptf-bootstrap only; boundary
# document changes after seal also need that window.
data "aws_iam_policy_document" "hcptf_apply_trust" {
statement {
sid = "HcpApply"
effect = "Allow"
actions = ["sts:AssumeRoleWithWebIdentity"]
principals {
type = "Federated"
identifiers = ["arn:aws:iam::${local.account_id}:oidc-provider/app.terraform.io"]
}
condition {
test = "StringEquals"
variable = "app.terraform.io:aud"
values = ["aws.workload.identity"]
}
condition {
test = "StringEquals"
variable = "app.terraform.io:sub"
values = [
"organization:seahaven:project:${local.hcp_project}:workspace:${local.hcp_workspace}:run_phase:apply",
]
}
}
}
data "aws_iam_policy_document" "hcptf_plan_trust" {
statement {
sid = "HcpPlan"
effect = "Allow"
actions = ["sts:AssumeRoleWithWebIdentity"]
principals {
type = "Federated"
identifiers = ["arn:aws:iam::${local.account_id}:oidc-provider/app.terraform.io"]
}
condition {
test = "StringEquals"
variable = "app.terraform.io:aud"
values = ["aws.workload.identity"]
}
condition {
test = "StringEquals"
variable = "app.terraform.io:sub"
values = [
"organization:seahaven:project:${local.hcp_project}:workspace:${local.hcp_workspace}:run_phase:plan",
]
}
}
}
data "aws_iam_policy_document" "hcptf_scoped_iam" {
statement {
sid = "DenyCreatePolicy"
effect = "Deny"
actions = [
"iam:CreatePolicy",
"iam:CreatePolicyVersion",
"iam:DeletePolicy",
"iam:DeletePolicyVersion",
"iam:SetDefaultPolicyVersion",
]
resources = ["*"]
}
statement {
sid = "CreateExecRoleWithBoundary"
effect = "Allow"
actions = ["iam:CreateRole"]
resources = [
"arn:aws:iam::${local.account_id}:role/tf-managed/${local.stack_prefix}*",
]
condition {
test = "StringLike"
variable = "iam:PermissionsBoundary"
values = [
"arn:aws:iam::${local.account_id}:policy/tf-managed/${local.stack_prefix}*",
"arn:aws:iam::${local.account_id}:policy/seahaven-lambda-execution-boundary",
"arn:aws:iam::${local.account_id}:policy/seahaven-lambda-execution-boundary-${local.stack_name}",
]
}
}
statement {
sid = "MutateExecRoleWithBoundary"
effect = "Allow"
actions = [
"iam:AttachRolePolicy",
"iam:PutRolePolicy",
"iam:PutRolePermissionsBoundary",
]
resources = [
"arn:aws:iam::${local.account_id}:role/tf-managed/${local.stack_prefix}*",
]
condition {
test = "StringLike"
variable = "iam:PermissionsBoundary"
values = [
"arn:aws:iam::${local.account_id}:policy/tf-managed/${local.stack_prefix}*",
"arn:aws:iam::${local.account_id}:policy/seahaven-lambda-execution-boundary",
"arn:aws:iam::${local.account_id}:policy/seahaven-lambda-execution-boundary-${local.stack_name}",
]
}
}
statement {
sid = "WriteExecRoles"
effect = "Allow"
actions = [
"iam:DeleteRole",
"iam:DeleteRolePolicy",
"iam:DetachRolePolicy",
"iam:TagRole",
"iam:UntagRole",
"iam:UpdateAssumeRolePolicy",
"iam:UpdateRole",
"iam:UpdateRoleDescription",
]
resources = [
"arn:aws:iam::${local.account_id}:role/tf-managed/${local.stack_prefix}*",
]
}
statement {
sid = "PassExecRolesToEc2"
effect = "Allow"
actions = ["iam:PassRole"]
resources = ["arn:aws:iam::${local.account_id}:role/tf-managed/${local.instance_role_name}"]
condition {
test = "StringEquals"
variable = "iam:PassedToService"
values = ["ec2.amazonaws.com"]
}
}
statement {
sid = "InstanceProfiles"
effect = "Allow"
actions = [
"iam:AddRoleToInstanceProfile",
"iam:CreateInstanceProfile",
"iam:DeleteInstanceProfile",
"iam:GetInstanceProfile",
"iam:ListInstanceProfileTags",
"iam:RemoveRoleFromInstanceProfile",
"iam:TagInstanceProfile",
"iam:UntagInstanceProfile",
]
resources = [
"arn:aws:iam::${local.account_id}:instance-profile/tf-managed/${local.instance_profile_name}",
]
}
statement {
sid = "IamReadOnly"
effect = "Allow"
actions = [
"iam:GetPolicy",
"iam:GetPolicyVersion",
"iam:GetRole",
"iam:GetRolePolicy",
"iam:GetInstanceProfile",
"iam:ListAttachedRolePolicies",
"iam:ListInstanceProfiles",
"iam:ListInstanceProfilesForRole",
"iam:ListPolicies",
"iam:ListPolicyVersions",
"iam:ListRolePolicies",
"iam:ListRoleTags",
"iam:ListRoles",
]
resources = ["*"]
}
statement {
sid = "DenySelfMutation"
effect = "Deny"
actions = [
"iam:AttachRolePolicy",
"iam:DeleteRole",
"iam:DeleteRolePolicy",
"iam:DeleteRolePermissionsBoundary",
"iam:DetachRolePolicy",
"iam:PutRolePolicy",
"iam:PutRolePermissionsBoundary",
"iam:UpdateAssumeRolePolicy",
"iam:UpdateRole",
"iam:UpdateRoleDescription",
]
resources = [
"arn:aws:iam::${local.account_id}:role/hcptf-*",
"arn:aws:iam::${local.account_id}:role/github-cfn-execution-role",
"arn:aws:iam::${local.account_id}:role/githubdeploy-*",
"arn:aws:iam::${local.account_id}:role/cdk-hnb659fds-*",
"arn:aws:iam::${local.account_id}:role/OrganizationAccountAccessRole",
"arn:aws:iam::${local.account_id}:role/seahaven-*",
]
}
statement {
sid = "DenyBoundaryTampering"
effect = "Deny"
actions = [
"iam:DeleteRolePermissionsBoundary",
"iam:DeleteUserPermissionsBoundary",
]
resources = [
"arn:aws:iam::${local.account_id}:role/*",
"arn:aws:iam::${local.account_id}:user/*",
]
}
statement {
sid = "DenyBoundaryPolicyEdit"
effect = "Deny"
actions = [
"iam:CreatePolicyVersion",
"iam:DeletePolicy",
"iam:DeletePolicyVersion",
"iam:SetDefaultPolicyVersion",
]
resources = ["arn:aws:iam::${local.account_id}:policy/seahaven-*"]
}
}
data "aws_iam_policy_document" "hcptf_apply_services" {
# checkov:skip=CKV_AWS_111: EC2 VPC/instance lifecycle and describe APIs require Resource=*. Log group, alarm, and SNS writes are ARN-prefixed.
statement {
sid = "CloudWatchLogs"
effect = "Allow"
actions = [
"logs:CreateLogGroup",
"logs:DeleteLogGroup",
"logs:PutRetentionPolicy",
"logs:DeleteRetentionPolicy",
"logs:TagResource",
"logs:UntagResource",
"logs:ListTagsForResource",
"logs:AssociateKmsKey",
"logs:DisassociateKmsKey",
]
resources = [
"arn:aws:logs:${var.aws_region}:${local.account_id}:log-group:${local.log_group_name}",
"arn:aws:logs:${var.aws_region}:${local.account_id}:log-group:${local.log_group_name}:*",
]
}
statement {
sid = "CloudWatchLogsDescribe"
effect = "Allow"
actions = ["logs:DescribeLogGroups"]
resources = ["*"]
}
statement {
sid = "CloudWatchAlarms"
effect = "Allow"
actions = [
"cloudwatch:PutMetricAlarm",
"cloudwatch:DeleteAlarms",
"cloudwatch:DescribeAlarms",
"cloudwatch:TagResource",
"cloudwatch:UntagResource",
"cloudwatch:ListTagsForResource",
]
resources = [
"arn:aws:cloudwatch:${var.aws_region}:${local.account_id}:alarm:Syslog-*",
"arn:aws:cloudwatch:${var.aws_region}:${local.account_id}:alarm:EC2-StatusCheck*syslog*",
]
}
statement {
sid = "CloudWatchDescribeAlarms"
effect = "Allow"
actions = ["cloudwatch:DescribeAlarms"]
resources = ["*"]
}
statement {
sid = "SnsPublishSiteAlerts"
effect = "Allow"
actions = [
"sns:Publish",
"sns:GetTopicAttributes",
"sns:ListTagsForResource",
]
resources = [local.site_alerts_arn]
}
statement {
sid = "ManageTfManagedBoundary"
effect = "Allow"
actions = [
"iam:GetPolicy",
"iam:GetPolicyVersion",
"iam:ListPolicyVersions",
"iam:ListPolicyTags",
"iam:TagPolicy",
"iam:UntagPolicy",
]
resources = [
"arn:aws:iam::${local.account_id}:policy/tf-managed/${local.stack_prefix}*",
]
}
statement {
sid = "Ec2VpcManagement"
effect = "Allow"
actions = [
"ec2:AllocateAddress",
"ec2:AssociateAddress",
"ec2:AssociateRouteTable",
"ec2:AttachInternetGateway",
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CreateInternetGateway",
"ec2:CreateRoute",
"ec2:CreateRouteTable",
"ec2:CreateSecurityGroup",
"ec2:CreateSubnet",
"ec2:CreateTags",
"ec2:CreateVpc",
"ec2:DeleteInternetGateway",
"ec2:DeleteRoute",
"ec2:DeleteRouteTable",
"ec2:DeleteSecurityGroup",
"ec2:DeleteSubnet",
"ec2:DeleteTags",
"ec2:DeleteVpc",
"ec2:DescribeAccountAttributes",
"ec2:DescribeAddresses",
"ec2:DescribeAddressesAttribute",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeInternetGateways",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeRouteTables",
"ec2:DescribeSecurityGroupRules",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeTags",
"ec2:DescribeVpcAttribute",
"ec2:DescribeVpcs",
"ec2:DescribePrefixLists",
"ec2:DetachInternetGateway",
"ec2:DisassociateAddress",
"ec2:DisassociateRouteTable",
"ec2:ModifySubnetAttribute",
"ec2:ModifyVpcAttribute",
"ec2:ReleaseAddress",
"ec2:RevokeSecurityGroupEgress",
"ec2:RevokeSecurityGroupIngress",
"ec2:UpdateSecurityGroupRuleDescriptionsEgress",
"ec2:UpdateSecurityGroupRuleDescriptionsIngress",
]
resources = ["*"]
}
statement {
sid = "Ec2InstanceManagement"
effect = "Allow"
actions = [
"ec2:AssociateIamInstanceProfile",
"ec2:AttachVolume",
"ec2:CreateVolume",
"ec2:DeleteVolume",
"ec2:DescribeIamInstanceProfileAssociations",
"ec2:DescribeImages",
"ec2:DescribeInstanceAttribute",
"ec2:DescribeInstanceCreditSpecifications",
"ec2:DescribeInstanceStatus",
"ec2:DescribeInstanceTypes",
"ec2:DescribeInstances",
"ec2:DescribeVolumes",
"ec2:DescribeVolumeAttribute",
"ec2:DescribeVolumeStatus",
"ec2:DetachVolume",
"ec2:DisassociateIamInstanceProfile",
"ec2:GetConsoleOutput",
"ec2:ModifyInstanceAttribute",
"ec2:ModifyVolume",
"ec2:MonitorInstances",
"ec2:RebootInstances",
"ec2:ReplaceIamInstanceProfileAssociation",
"ec2:RunInstances",
"ec2:StartInstances",
"ec2:StopInstances",
"ec2:TerminateInstances",
"ec2:UnmonitorInstances",
]
resources = ["*"]
}
}
data "aws_iam_policy_document" "hcptf_plan_refresh" {
statement {
sid = "RefreshIamRoles"
effect = "Allow"
actions = [
"iam:GetRole",
"iam:GetRolePolicy",
"iam:GetInstanceProfile",
"iam:ListRolePolicies",
"iam:ListAttachedRolePolicies",
"iam:ListInstanceProfilesForRole",
"iam:ListRoleTags",
]
resources = [
"arn:aws:iam::${local.account_id}:role/tf-managed/${local.stack_prefix}*",
"arn:aws:iam::${local.account_id}:instance-profile/tf-managed/${local.stack_prefix}*",
"arn:aws:iam::${local.account_id}:role/${local.apply_role}",
"arn:aws:iam::${local.account_id}:role/${local.plan_role}",
]
}
statement {
sid = "RefreshManagedPolicies"
effect = "Allow"
actions = [
"iam:GetPolicy",
"iam:GetPolicyVersion",
]
resources = ["*"]
}
statement {
sid = "RefreshLogs"
effect = "Allow"
actions = [
"logs:DescribeLogGroups",
"logs:ListTagsForResource",
]
resources = ["*"]
}
statement {
sid = "RefreshAlarms"
effect = "Allow"
actions = [
"cloudwatch:DescribeAlarms",
"cloudwatch:ListTagsForResource",
]
resources = ["*"]
}
statement {
sid = "RefreshSns"
effect = "Allow"
actions = [
"sns:GetTopicAttributes",
"sns:ListTagsForResource",
]
resources = [local.site_alerts_arn]
}
statement {
sid = "RefreshEc2"
effect = "Allow"
actions = [
"ec2:DescribeAccountAttributes",
"ec2:DescribeAddresses",
"ec2:DescribeAddressesAttribute",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeIamInstanceProfileAssociations",
"ec2:DescribeImages",
"ec2:DescribeInstanceAttribute",
"ec2:DescribeInstanceCreditSpecifications",
"ec2:DescribeInstanceStatus",
"ec2:DescribeInstanceTypes",
"ec2:DescribeInstances",
"ec2:DescribeInternetGateways",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribePrefixLists",
"ec2:DescribeRouteTables",
"ec2:DescribeSecurityGroupRules",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeTags",
"ec2:DescribeVolumeAttribute",
"ec2:DescribeVolumeStatus",
"ec2:DescribeVolumes",
"ec2:DescribeVpcAttribute",
"ec2:DescribeVpcs",
"ec2:GetConsoleOutput",
]
resources = ["*"]
}
}
resource "aws_iam_role" "hcptf_apply" {
name = local.apply_role
assume_role_policy = data.aws_iam_policy_document.hcptf_apply_trust.json
max_session_duration = 3600
tags = {
Owner = "adam@seahavenind.com"
ManagedBy = "terraform"
}
}
resource "aws_iam_role" "hcptf_plan" {
name = local.plan_role
assume_role_policy = data.aws_iam_policy_document.hcptf_plan_trust.json
max_session_duration = 3600
tags = {
Owner = "adam@seahavenind.com"
ManagedBy = "terraform"
}
}
resource "aws_iam_role_policy" "hcptf_scoped_iam" {
name = "scoped-iam-management"
role = aws_iam_role.hcptf_apply.id
policy = data.aws_iam_policy_document.hcptf_scoped_iam.json
}
resource "aws_iam_role_policy" "hcptf_apply_services" {
# checkov:skip=CKV_AWS_111: EC2 VPC/instance lifecycle and describe APIs require Resource=*. Log group, alarm, and SNS writes are ARN-prefixed.
name = "syslog-server-services"
role = aws_iam_role.hcptf_apply.id
policy = data.aws_iam_policy_document.hcptf_apply_services.json
}
resource "aws_iam_role_policy" "hcptf_plan_refresh" {
# checkov:skip=CKV_AWS_107: ViewOnlyAccess plus this sidecar is the org HCP plan-role pattern (PLAT-144). Sidecar Get* is named (GetRole, GetPolicy, GetInstanceProfile, GetConsoleOutput, GetTopicAttributes) and scoped to this stack. It does not add iam:CreateAccessKey, secretsmanager:GetSecretValue, or ssm:GetParameter on *.
name = "syslog-server-plan-refresh"
role = aws_iam_role.hcptf_plan.id
policy = data.aws_iam_policy_document.hcptf_plan_refresh.json
}
resource "aws_iam_role_policy_attachment" "hcptf_plan_view_only" {
role = aws_iam_role.hcptf_plan.name
policy_arn = "arn:aws:iam::aws:policy/job-function/ViewOnlyAccess"
}
resource "aws_iam_role_policy_attachments_exclusive" "hcptf_apply" {
role_name = aws_iam_role.hcptf_apply.name
policy_arns = []
}
resource "aws_iam_role_policy_attachments_exclusive" "hcptf_plan" {
role_name = aws_iam_role.hcptf_plan.name
policy_arns = [
"arn:aws:iam::aws:policy/job-function/ViewOnlyAccess",
]
}

174
terraform/iam.tf Normal file
View file

@ -0,0 +1,174 @@
# Instance permissions boundary. Created on the first (bootstrap) apply.
# The scoped apply role denies iam:CreatePolicy / CreatePolicyVersion,
# so later edits to this document need the hcptf-bootstrap window.
data "aws_iam_policy_document" "instance_boundary" {
# checkov:skip=CKV_AWS_111: SSM and CloudWatch agent managed policies require Resource=* for ssmmessages, ec2messages, and describe APIs. Log writes are ARN-prefixed.
statement {
sid = "CloudWatchLogsWrite"
effect = "Allow"
actions = [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:DescribeLogGroups",
"logs:DescribeLogStreams",
"logs:PutLogEvents",
"logs:PutRetentionPolicy",
]
resources = [
"arn:aws:logs:${var.aws_region}:${local.account_id}:log-group:${local.log_group_name}",
"arn:aws:logs:${var.aws_region}:${local.account_id}:log-group:${local.log_group_name}:*",
]
}
statement {
sid = "CloudWatchLogsDescribe"
effect = "Allow"
actions = ["logs:DescribeLogGroups"]
resources = ["*"]
}
statement {
sid = "CloudWatchMetrics"
effect = "Allow"
actions = [
"cloudwatch:PutMetricData",
]
resources = ["*"]
}
statement {
sid = "Ec2DescribeForAgent"
effect = "Allow"
actions = [
"ec2:DescribeTags",
"ec2:DescribeVolumes",
"ec2:DescribeInstances",
]
resources = ["*"]
}
statement {
sid = "SsmAgentBuckets"
effect = "Allow"
actions = [
"s3:GetObject",
]
resources = [
"arn:aws:s3:::aws-ssm-*/*",
"arn:aws:s3:::aws-windows-downloads-*/*",
"arn:aws:s3:::amazon-ssm-*/*",
"arn:aws:s3:::amazon-ssm-packages-*/*",
"arn:aws:s3:::patch-baseline-snapshot-*/*",
]
}
statement {
sid = "SsmManagedInstance"
effect = "Allow"
actions = [
"ssm:DescribeAssociation",
"ssm:GetDeployablePatchSnapshotForInstance",
"ssm:GetDocument",
"ssm:DescribeDocument",
"ssm:GetManifest",
"ssm:ListAssociations",
"ssm:ListInstanceAssociations",
"ssm:PutInventory",
"ssm:PutComplianceItems",
"ssm:PutConfigurePackageResult",
"ssm:UpdateAssociationStatus",
"ssm:UpdateInstanceAssociationStatus",
"ssm:UpdateInstanceInformation",
]
resources = ["*"]
}
statement {
sid = "SsmAgentParameters"
effect = "Allow"
actions = [
"ssm:GetParameter",
"ssm:GetParameters",
]
resources = [
"arn:aws:ssm:${var.aws_region}::parameter/aws/service/*",
"arn:aws:ssm:${var.aws_region}:${local.account_id}:parameter/aws/service/*",
"arn:aws:ssm:${var.aws_region}:${local.account_id}:parameter/AmazonCloudWatch-*",
]
}
statement {
sid = "SsmMessages"
effect = "Allow"
actions = [
"ssmmessages:CreateControlChannel",
"ssmmessages:CreateDataChannel",
"ssmmessages:OpenControlChannel",
"ssmmessages:OpenDataChannel",
]
resources = ["*"]
}
statement {
sid = "Ec2Messages"
effect = "Allow"
actions = [
"ec2messages:AcknowledgeMessage",
"ec2messages:DeleteMessage",
"ec2messages:FailMessage",
"ec2messages:GetEndpoint",
"ec2messages:GetMessages",
"ec2messages:SendReply",
]
resources = ["*"]
}
}
resource "aws_iam_policy" "instance_boundary" {
# checkov:skip=CKV_AWS_111: SSM and CloudWatch agent managed policies require Resource=* for ssmmessages, ec2messages, and describe APIs. Log writes are ARN-prefixed.
name = local.boundary_name
path = "/tf-managed/"
description = "Per-workload EC2 permissions boundary for syslog-server (PLAT-78)."
policy = data.aws_iam_policy_document.instance_boundary.json
}
data "aws_iam_policy_document" "instance_assume" {
statement {
sid = "Ec2Assume"
effect = "Allow"
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = ["ec2.amazonaws.com"]
}
}
}
resource "aws_iam_role" "instance" {
name = local.instance_role_name
path = "/tf-managed/"
assume_role_policy = data.aws_iam_policy_document.instance_assume.json
permissions_boundary = aws_iam_policy.instance_boundary.arn
tags = {
Name = local.instance_role_name
}
}
resource "aws_iam_role_policy_attachment" "ssm" {
role = aws_iam_role.instance.name
policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}
resource "aws_iam_role_policy_attachment" "cloudwatch_agent" {
role = aws_iam_role.instance.name
policy_arn = "arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy"
}
resource "aws_iam_instance_profile" "this" {
name = local.instance_profile_name
path = "/tf-managed/"
role = aws_iam_role.instance.name
}

33
terraform/locals.tf Normal file
View file

@ -0,0 +1,33 @@
locals {
project = "syslog-server"
account_id = "011934824531"
environment = "prod"
hcp_project = "seahaven-prod"
hcp_workspace = "syslog-server-prod"
apply_role = "hcptf-syslog-server"
plan_role = "hcptf-syslog-server-plan"
stack_name = local.project
stack_prefix = "syslog-server-"
instance_role_name = "syslog-server-role"
instance_profile_name = "syslog-server-profile"
boundary_name = "syslog-server-instance-boundary"
log_group_name = "unifi-syslog"
site_alerts_arn = "arn:aws:sns:${var.aws_region}:${local.account_id}:site-alerts"
vpc_cidr = "10.40.0.0/16"
public_subnet_cidr = "10.40.10.0/24"
office_cidrs = ["47.21.61.4/32", "96.250.164.146/32"]
office_vpn_cidr = "10.10.0.0/16"
vpn_pool_cidr = "10.30.0.0/16"
syslog_vpc_cidr = local.vpc_cidr
syslog_ingress_cidrs = concat(
local.office_cidrs,
[local.office_vpn_cidr, local.vpn_pool_cidr, local.syslog_vpc_cidr],
)
ssh_ingress_cidrs = concat(
local.office_cidrs,
[local.office_vpn_cidr, local.syslog_vpc_cidr],
)
}

4
terraform/logs.tf Normal file
View file

@ -0,0 +1,4 @@
resource "aws_cloudwatch_log_group" "unifi_syslog" {
name = local.log_group_name
retention_in_days = 90
}

29
terraform/outputs.tf Normal file
View file

@ -0,0 +1,29 @@
output "instance_id" {
description = "syslog-server EC2 instance id."
value = aws_instance.this.id
}
output "public_ip" {
description = "Elastic IP — UniFi remote-syslog forwarding target."
value = aws_eip.this.public_ip
}
output "allocation_id" {
description = "Elastic IP allocation id."
value = aws_eip.this.allocation_id
}
output "log_group_name" {
description = "CloudWatch Logs group the agent ships remote syslog into."
value = aws_cloudwatch_log_group.unifi_syslog.name
}
output "hcptf_apply_role_arn" {
description = "HCP apply role ARN. Set TFC_AWS_APPLY_ROLE_ARN after the bootstrap window."
value = aws_iam_role.hcptf_apply.arn
}
output "hcptf_plan_role_arn" {
description = "HCP plan role ARN. Set TFC_AWS_PLAN_ROLE_ARN after the bootstrap window."
value = aws_iam_role.hcptf_plan.arn
}

12
terraform/providers.tf Normal file
View file

@ -0,0 +1,12 @@
provider "aws" {
region = var.aws_region
default_tags {
tags = {
Project = local.project
Environment = "prod"
ManagedBy = "terraform"
Workspace = local.hcp_workspace
}
}
}

143
terraform/user_data.sh Normal file
View file

@ -0,0 +1,143 @@
#!/bin/bash
set -euxo pipefail
# ── 1 GiB swap (build headroom + stability on the 512 MiB t4g.nano) ──
if [ ! -f /swapfile ]; then
fallocate -l 1G /swapfile || dd if=/dev/zero of=/swapfile bs=1M count=1024
chmod 600 /swapfile
mkswap /swapfile
echo '/swapfile none swap sw 0 0' >> /etc/fstab
fi
swapon -a || true
# ── rsyslog: listen on UDP/TCP 514 ──
dnf install -y rsyslog
cat > /etc/rsyslog.d/10-listen.conf <<'EOF'
module(load="imudp")
input(type="imudp" port="514")
module(load="imtcp")
input(type="imtcp" port="514")
EOF
# ── Write remote syslog to /var/log/remote/<host>/<program>.log ──
cat > /etc/rsyslog.d/20-remote.conf <<'EOF'
template(name="RemoteHost" type="string" string="/var/log/remote/%HOSTNAME%/%PROGRAMNAME%.log")
if $fromhost-ip != '127.0.0.1' then {
action(type="omfile" dynaFile="RemoteHost" createDirs="on")
stop
}
EOF
mkdir -p /var/log/remote
systemctl enable rsyslog
systemctl restart rsyslog
# ── Rotate /var/log/remote so it can't grow unbounded ──
# CloudWatch (90d) is the system of record; these local files are just a
# spool for the CW agent, so keep only a short window. copytruncate keeps
# rsyslog's open dynaFile handles valid (truncate in place, same inode).
cat > /etc/logrotate.d/remote-syslog <<'EOF'
/var/log/remote/*/*.log {
daily
rotate 7
compress
delaycompress
missingok
notifempty
copytruncate
}
EOF
# ── CloudWatch agent: ship /var/log/remote/**/*.log to unifi-syslog ──
dnf install -y amazon-cloudwatch-agent
cat > /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json <<'EOF'
{
"logs": {
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/var/log/remote/**/*.log",
"log_group_name": "unifi-syslog",
"log_stream_name": "{hostname}/{file_name}",
"retention_in_days": 90
}
]
}
}
}
}
EOF
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
-a fetch-config -m ec2 \
-c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s
systemctl enable amazon-cloudwatch-agent
# ── NetFlow/IPFIX collectors (nfcapd) ──
# nfdump is not packaged for AL2023; build 1.6.23 from source (needs
# rrdtool-devel for librrd). Reconstructed under IaC for INFRA-12 — the
# original instance ran these as hand-installed systemd units. Captures
# are local-only (no consumer/shipping today); 30-day retention enforced.
dnf install -y gcc gcc-c++ make automake autoconf libtool flex bison libpcap-devel zlib-devel bzip2-devel rrdtool-devel tar
NFVER=1.6.23
curl -sfL https://github.com/phaag/nfdump/archive/refs/tags/v${NFVER}.tar.gz | tar xz -C /tmp
( cd /tmp/nfdump-${NFVER} && ./autogen.sh && ./configure && make -j1 && make install )
ldconfig
mkdir -p /var/log/netflow/ronkonkoma /var/log/netflow/locust
chown -R ec2-user:ec2-user /var/log/netflow
# Ronkonkoma gateway -> UDP 2055
cat > /etc/systemd/system/nfcapd.service <<'EOF'
[Unit]
Description=nfcapd NetFlow collector (Ronkonkoma, udp/2055)
After=network.target
[Service]
Type=simple
User=ec2-user
ExecStart=/usr/local/bin/nfcapd -p 2055 -l /var/log/netflow/ronkonkoma
Restart=always
[Install]
WantedBy=multi-user.target
EOF
# Locust Ave gateway -> UDP 2056
cat > /etc/systemd/system/nfcapd-locust.service <<'EOF'
[Unit]
Description=nfcapd NetFlow collector (Locust Ave, udp/2056)
After=network.target
[Service]
Type=simple
User=ec2-user
ExecStart=/usr/local/bin/nfcapd -p 2056 -l /var/log/netflow/locust
Restart=always
[Install]
WantedBy=multi-user.target
EOF
# 30-day retention sweep (daily 03:30 UTC)
cat > /usr/local/sbin/netflow-retention.sh <<'EOF'
#!/bin/bash
find /var/log/netflow -type f -name 'nfcapd.*' -mtime +30 -delete
EOF
chmod +x /usr/local/sbin/netflow-retention.sh
cat > /etc/systemd/system/netflow-retention.service <<'EOF'
[Unit]
Description=Delete NetFlow captures older than 30 days
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/netflow-retention.sh
EOF
cat > /etc/systemd/system/netflow-retention.timer <<'EOF'
[Unit]
Description=Daily NetFlow retention sweep
[Timer]
OnCalendar=*-*-* 03:30:00 UTC
Persistent=true
[Install]
WantedBy=timers.target
EOF
systemctl daemon-reload
systemctl enable --now nfcapd.service nfcapd-locust.service netflow-retention.timer

11
terraform/variables.tf Normal file
View file

@ -0,0 +1,11 @@
variable "aws_region" {
description = "Region every resource in this configuration is created in."
type = string
default = "us-east-1"
}
variable "ami_id" {
description = "Pinned Amazon Linux 2023 arm64 AMI. Changing this replaces the instance."
type = string
default = "ami-02c114835b4e7739f"
}

18
terraform/versions.tf Normal file
View file

@ -0,0 +1,18 @@
terraform {
required_version = ">= 1.14.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 6.64"
}
}
cloud {
organization = "seahaven"
workspaces {
name = "syslog-server-prod"
}
}
}

123
terraform/vpc.tf Normal file
View file

@ -0,0 +1,123 @@
data "aws_availability_zones" "available" {
state = "available"
}
resource "aws_vpc" "this" {
cidr_block = local.vpc_cidr
enable_dns_support = true
enable_dns_hostnames = true
tags = {
Name = "syslog-server-vpc"
}
}
resource "aws_internet_gateway" "this" {
vpc_id = aws_vpc.this.id
tags = {
Name = "syslog-server-igw"
}
}
resource "aws_subnet" "public" {
vpc_id = aws_vpc.this.id
cidr_block = local.public_subnet_cidr
availability_zone = data.aws_availability_zones.available.names[0]
map_public_ip_on_launch = true
tags = {
Name = "syslog-server-public"
}
}
resource "aws_route_table" "public" {
vpc_id = aws_vpc.this.id
tags = {
Name = "syslog-server-public"
}
}
resource "aws_route" "public_default" {
route_table_id = aws_route_table.public.id
destination_cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.this.id
}
resource "aws_route_table_association" "public" {
subnet_id = aws_subnet.public.id
route_table_id = aws_route_table.public.id
}
resource "aws_security_group" "this" {
name = "syslog-server"
description = "Syslog collector - rsyslog 514 from office + VPC"
vpc_id = aws_vpc.this.id
tags = {
Name = "syslog-server"
}
}
resource "aws_vpc_security_group_egress_rule" "all" {
security_group_id = aws_security_group.this.id
ip_protocol = "-1"
cidr_ipv4 = "0.0.0.0/0"
description = "All outbound for package installs and CloudWatch"
}
resource "aws_vpc_security_group_ingress_rule" "syslog_tcp" {
for_each = toset(local.syslog_ingress_cidrs)
security_group_id = aws_security_group.this.id
ip_protocol = "tcp"
from_port = 514
to_port = 514
cidr_ipv4 = each.value
description = "syslog TCP 514"
}
resource "aws_vpc_security_group_ingress_rule" "syslog_udp" {
for_each = toset(local.syslog_ingress_cidrs)
security_group_id = aws_security_group.this.id
ip_protocol = "udp"
from_port = 514
to_port = 514
cidr_ipv4 = each.value
description = "syslog UDP 514"
}
resource "aws_vpc_security_group_ingress_rule" "ssh" {
for_each = toset(local.ssh_ingress_cidrs)
security_group_id = aws_security_group.this.id
ip_protocol = "tcp"
from_port = 22
to_port = 22
cidr_ipv4 = each.value
description = "SSH break-glass"
}
resource "aws_vpc_security_group_ingress_rule" "netflow_2055" {
for_each = toset(local.office_cidrs)
security_group_id = aws_security_group.this.id
ip_protocol = "udp"
from_port = 2055
to_port = 2055
cidr_ipv4 = each.value
description = "netflow/sflow UDP 2055"
}
resource "aws_vpc_security_group_ingress_rule" "netflow_2056" {
for_each = toset(local.office_cidrs)
security_group_id = aws_security_group.this.id
ip_protocol = "udp"
from_port = 2056
to_port = 2056
cidr_ipv4 = each.value
description = "netflow/sflow UDP 2056"
}

View file

@ -1,24 +0,0 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "commonjs",
"lib": ["ES2022"],
"types": ["node"],
"declaration": true,
"strict": true,
"noImplicitAny": true,
"strictNullChecks": true,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true,
"inlineSourceMap": true,
"inlineSources": true,
"strictPropertyInitialization": false,
"outDir": "./cdk.out",
"rootDir": ".",
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"esModuleInterop": true
},
"exclude": ["node_modules", "cdk.out"]
}