mirror of
https://github.com/Sea-Haven-Industries/apm-wo-analysis.git
synced 2026-09-30 21:43:14 +00:00
6ee218eea3
CDK-define the apm_wo_snapshots Glue table with partition projection over analytics/ (dt as projected date partition, 2026-01-01..NOW). Projection means no crawler, no MSCK REPAIR, and — critically — the classifier needs no Glue catalog access at all. pipeline_stack.py: glue.CfnTable (Parquet SerDe, 17-column schema mirroring the classifier's snapshot incl. contractor_description and the two boolean flags) + athena.CfnWorkGroup `apm-wo-analysis` (enforced result location, SSE-S3) + a 30-day lifecycle rule on athena-results/ (disposable query output in a RETAIN bucket). Trim the classifier role: drop the entire Glue policy statement. handler.py: stop registering the table at runtime — drop database=/table= from to_parquet so the classifier writes pure Parquet; partition projection handles the rest. Keeps overwrite_partitions for idempotent same-day re-uploads. test_pipeline_synth.py: offline synth assertions (bundling skipped) — projection properties, column schema/types, workgroup result enforcement, and that no IAM policy grants glue:* to the classifier. cdk synth green; 11/11 tests pass. |
||
|---|---|---|
| .. | ||
| stacks | ||
| app.py | ||
| cdk.json | ||
| requirements.txt | ||