Two durable fixes for the root causes that took the UniFi syslog feed dark
when the collector was recreated under IaC on 2026-06-09:
1. SecurityGroup: allow udp/tcp 514 from 192.168.0.0/24. The UDM gateways
forward via syslog-ng / unifi-core sourced from the IPsec VTI tunnel
inside-address (192.168.0.x), not their LAN IP, so the recreated SG
silently denied all gateway-originated syslog at the ENI. (A LAN client
keeps its 10.10.x source and was always allowed — which masked the issue.)
2. Pin the instance PrivateIpAddress to 10.20.10.221. The fleet forwards to
this address over the site-to-site VPN; the recreation moved it .111->.221
and orphaned every console's syslog target. Pinning keeps it stable across
future replacements.
DEPLOY NOTES:
- (1) was applied live out-of-band to restore service; delete that live SG
rule immediately before `cdk deploy` or CFN errors on InvalidPermission.Duplicate.
- (2) forces an instance replacement (cdk diff: PrivateIpAddress requires
replacement; EIP re-associates). Deploy in a maintenance window — ~2-3 min
log-collection gap while user-data re-runs; ends at the same .221 so no
console reconfig needed.
- Build nfdump 1.6.23 from source in user-data (rrdtool-devel for librrd;
not packaged on AL2023) and run nfcapd collectors as systemd units:
nfcapd.service (Ronkonkoma udp/2055), nfcapd-locust.service (Locust udp/2056),
+ netflow-retention.timer (30d sweep). 1 GiB swapfile for build headroom.
- userDataCausesReplacement: true — a user-data change must actually re-run,
so force instance replacement (stateless box, EIP re-associates).
Validated: build + all services active, listeners on 514/2055/2056.
CDK stack for the EC2 syslog collector (rsyslog 514 -> CloudWatch agent ->
unifi-syslog), mirroring the file-share/forgejo pattern. Recreated from the
captured console config; EIP 184.72.154.32 imported + re-associated so the
UniFi forwarding target is unchanged. Deployed + verified 2026-06-09.
Note: deploy role can assume cdk-hnb659fds-* (account-admin via CDK
bootstrap) — same exposure as every org CDK deploy role; per-app qualifier
is a known org-wide follow-up.
