adam/procurement-ingest
1
0
Fork 0
mirror of https://github.com/Sea-Haven-Industries/procurement-ingest.git synced 2026-09-30 20:03:14 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
59 commits 6 branches 0 tags 6.7 MiB
Commit graph

27 commits

Author SHA1 Message Date
dependabot[bot]
dedbbda7a1
Update boto3 requirement in /lambdas/wo/web_ui (#46)
Some checks failed
Deploy / deploy (push) Has been cancelled
Details 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.43.22...1.43.23)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.23
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Adam Moussa <166072409+amoussa1229@users.noreply.github.com>
 2026-06-08 22:37:44 +00:00
dependabot[bot]
d26714c6e5
Update boto3 requirement in /lambdas/po/email_processor (#45) 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.43.22...1.43.23)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.23
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Adam Moussa <166072409+amoussa1229@users.noreply.github.com>
 2026-06-08 22:35:51 +00:00
dependabot[bot]
7b3b01ec05
Update boto3 requirement in /lambdas/wo/email_processor (#44) 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.43.10...1.43.23)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.23
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Adam Moussa <166072409+amoussa1229@users.noreply.github.com>
 2026-06-08 22:33:01 +00:00
dependabot[bot]
e47cd7685e
Update boto3 requirement in /lambdas/po/email_processor (#39)
Some checks failed
Deploy / deploy (push) Has been cancelled
Details 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.43.10...1.43.22)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.22
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 22:52:38 +00:00
dependabot[bot]
d145178c8d
Update boto3 requirement in /lambdas/wo/web_ui (#41) 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.43.10...1.43.22)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.22
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 22:51:28 +00:00
dependabot[bot]
53e906bf56
Update anthropic requirement in /lambdas/wo/email_processor (#38) 
Updates the requirements on [anthropic](https://github.com/anthropics/anthropic-sdk-python) to permit the latest version.
- [Release notes](https://github.com/anthropics/anthropic-sdk-python/releases)
- [Changelog](https://github.com/anthropics/anthropic-sdk-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anthropics/anthropic-sdk-python/compare/v0.103.1...v0.105.2)

---
updated-dependencies:
- dependency-name: anthropic
  dependency-version: 0.105.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 22:49:37 +00:00
dependabot[bot]
3481a7a8d5
Update anthropic requirement in /lambdas/po/email_processor (#37) 
Updates the requirements on [anthropic](https://github.com/anthropics/anthropic-sdk-python) to permit the latest version.
- [Release notes](https://github.com/anthropics/anthropic-sdk-python/releases)
- [Changelog](https://github.com/anthropics/anthropic-sdk-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anthropics/anthropic-sdk-python/compare/v0.103.1...v0.105.2)

---
updated-dependencies:
- dependency-name: anthropic
  dependency-version: 0.105.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 22:49:20 +00:00
dependabot[bot]
dfd2a77126
Update boto3 requirement in /lambdas/wo/email_processor (#33)
Some checks failed
Deploy / deploy (push) Has been cancelled
Details 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.43.6...1.43.10)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.10
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-19 19:24:01 +00:00
dependabot[bot]
49ecf29631
Update anthropic requirement in /lambdas/po/email_processor (#29) 
Updates the requirements on [anthropic](https://github.com/anthropics/anthropic-sdk-python) to permit the latest version.
- [Release notes](https://github.com/anthropics/anthropic-sdk-python/releases)
- [Changelog](https://github.com/anthropics/anthropic-sdk-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anthropics/anthropic-sdk-python/compare/v0.101.0...v0.103.1)

---
updated-dependencies:
- dependency-name: anthropic
  dependency-version: 0.103.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-19 19:23:37 +00:00
dependabot[bot]
73838c7da9
Update boto3 requirement in /lambdas/po/email_processor (#32) 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.43.6...1.43.10)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.10
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-19 19:21:23 +00:00
dependabot[bot]
dc3355b7a8
Update boto3 requirement in /lambdas/wo/web_ui (#31) 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.43.6...1.43.10)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.10
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-19 19:21:02 +00:00
dependabot[bot]
51f4272b0e
Update anthropic requirement in /lambdas/wo/email_processor (#30) 
Updates the requirements on [anthropic](https://github.com/anthropics/anthropic-sdk-python) to permit the latest version.
- [Release notes](https://github.com/anthropics/anthropic-sdk-python/releases)
- [Changelog](https://github.com/anthropics/anthropic-sdk-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anthropics/anthropic-sdk-python/compare/v0.101.0...v0.103.1)

---
updated-dependencies:
- dependency-name: anthropic
  dependency-version: 0.103.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-19 19:20:53 +00:00
dependabot[bot]
cb2af81615
Update anthropic requirement in /lambdas/wo/email_processor (#27)
Some checks failed
Deploy / deploy (push) Has been cancelled
Details 
Updates the requirements on [anthropic](https://github.com/anthropics/anthropic-sdk-python) to permit the latest version.
- [Release notes](https://github.com/anthropics/anthropic-sdk-python/releases)
- [Changelog](https://github.com/anthropics/anthropic-sdk-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anthropics/anthropic-sdk-python/compare/v0.42.0...v0.101.0)

---
updated-dependencies:
- dependency-name: anthropic
  dependency-version: 0.101.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-12 19:26:04 +00:00
dependabot[bot]
b7dd722de6
Update boto3 requirement from >=1.35.0 to >=1.43.6 in /lambdas/wo/web_ui (#25) 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.35.0...1.43.6)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.6
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-12 19:23:39 +00:00
dependabot[bot]
3c9f845a32
Update boto3 requirement in /lambdas/wo/email_processor (#26) 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.35.0...1.43.6)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.6
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-12 19:23:36 +00:00
Adam Moussa
5112c1345b
Merge workorder-ingest into unified procurement repo (#22) 
* Merge workorder-ingest pipeline into unified repo

Move PO lambdas under lambdas/po/, add WO pipeline under lambdas/wo/.
Two independent CloudFormation stacks in one CDK app. Fix WO stack
compliance: ARM64 architecture, 60-day log retention, aarch64 bundling,
RETAIN on Anthropic secret. Remove stale CodePipeline buildspec.

* Fix test_local.py import path and remove dead shared/models.py

test_local.py referenced the old lambdas/email_processor path. Updated
to lambdas/wo/email_processor. Removed shared/ directory entirely as
nothing imports from it.

* Escape HTML in both web UI dashboards to prevent XSS

Both Function URLs are public (auth_type=NONE) and render
email-derived content via f-strings. Attacker-crafted emails
could inject scripts. Added html.escape() on all interpolated
values in both PO and WO dashboards.

* Add pagination to WO web UI scan

get_work_orders() only fetched the first 1MB page from DynamoDB.
Loop on LastEvaluatedKey to match the PO web UI pattern.

* Fix esc(None) TypeError and javascript: scheme in PO web UI

Coerce supplier name through `or ""` before escaping to handle
nested None from DynamoDB. Add scheme allowlist on view_order_url
to block javascript:/data: hrefs from LLM-extracted URLs.

* Fix WO render_badge None guard, updated_at slice, and backfill path

Add null guard to WO render_badge matching the PO version. Use
`or ""` before slicing updated_at to handle explicit None values.
Fix backfill_sites.py sys.path to use new lambdas/po/site_extractor.

* Harden WO web UI and fix JS-context XSS in both dashboards

- Use json.dumps for onclick URLs to prevent JS string breakout
- Add .lower() to WO render_badge color lookup matching PO pattern
- Add pagination to get_comments query
- Cap get_work_orders to 500 results matching PO pattern

* Apply ruff formatting to web UI handlers
 2026-05-12 15:21:06 -04:00
dependabot[bot]
c0b276f2bc
Update anthropic requirement in /lambdas/email_processor (#21) 
Updates the requirements on [anthropic](https://github.com/anthropics/anthropic-sdk-python) to permit the latest version.
- [Release notes](https://github.com/anthropics/anthropic-sdk-python/releases)
- [Changelog](https://github.com/anthropics/anthropic-sdk-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anthropics/anthropic-sdk-python/compare/v0.100.0...v0.101.0)

---
updated-dependencies:
- dependency-name: anthropic
  dependency-version: 0.101.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-12 11:46:32 +00:00
dependabot[bot]
e7022d0772
Update boto3 requirement in /lambdas/email_processor (#17) 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.43.3...1.43.6)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.6
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-08 22:27:31 +00:00
Adam Moussa
abdf2aa035
Add CI workflow (#18) 
* Add CI workflow and apply ruff formatting

* Disable cdk synth — email_processor uses pre-built package dir

The email_processor Lambda bundles deps into a gitignored package/
directory. cdk synth fails in CI without a build step to recreate it.
Disabling until packaging is standardized.

* Use CDK BundlingOptions for email_processor Lambda packaging

Replaces the pre-built gitignored package/ directory with CDK's
built-in bundling. Deps are now installed inside a Docker container
during cdk synth, so the build works identically locally and in CI.
Re-enables run-cdk-synth in the CI workflow.
 2026-05-08 16:01:21 -04:00
dependabot[bot]
9e2ac362dc
Update anthropic requirement in /lambdas/email_processor (#12) 
Updates the requirements on [anthropic](https://github.com/anthropics/anthropic-sdk-python) to permit the latest version.
- [Release notes](https://github.com/anthropics/anthropic-sdk-python/releases)
- [Changelog](https://github.com/anthropics/anthropic-sdk-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anthropics/anthropic-sdk-python/compare/v0.42.0...v0.100.0)

---
updated-dependencies:
- dependency-name: anthropic
  dependency-version: 0.98.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-06 18:23:55 -04:00
dependabot[bot]
7225f5c768
Update boto3 requirement in /lambdas/email_processor (#11) 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.43.2...1.43.3)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-06 18:21:29 -04:00
dependabot[bot]
636177ecfd
Update boto3 requirement in /lambdas/email_processor 
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version.
- [Release notes](https://github.com/boto/boto3/releases)
- [Commits](https://github.com/boto/boto3/compare/1.35.0...1.43.2)

---
updated-dependencies:
- dependency-name: boto3
  dependency-version: 1.43.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-05-02 21:17:26 +00:00
Adam Moussa
e66062c934 Align PO schema with enriched records and improve extraction prompt 
Replaces extraction prompt with domain-specific rules: trade
classification taxonomy (23 categories), site_code skip list,
zip padding, revision email type, and structured extraction for
fiscal_year, trade, and coupa_category.

Handler changes:
- New "revision" email type overwrites existing PO via put_item
- enrich_parsed() adds top-level state, ship_to_raw, data_source
- pad_zip() zero-pads short zip codes (e.g., "7001" → "07001")
- Removed invoice_total/invoice_count (Payee Central only)

Web UI: added revision badge, new detail fields (site code, state,
trade, fiscal year, coupa category, data source), line item table
now shows Qty/Unit/Price columns, list view shows Site and Trade.

CDK: fixed StreamViewType to match deployed table (NEW_IMAGE).
README: documented PO record schema and revision flow.
 2026-05-01 19:53:26 -04:00
Adam Moussa
f5d1eaeb5b Add address reverse-lookup fallback and pending-site-review table 
When no site code is found via Claude extraction or regex cascade,
the Lambda now checks the PO address against a cold-start cache of
verified-sites (normalized street + zip match). If still no match,
the PO is written to a new pending-site-review table for manual
verification against Payee Central.
 2026-04-30 15:01:09 -04:00
Adam Moussa
5a86b3c96b Add site_code and structured address fields to extraction prompt 
The Claude extraction prompt now explicitly asks for site_code (the
Amazon facility code) and structured ship_to address fields (street,
city, state, zip). The site-extractor Lambda prefers these direct
fields when available, falling back to regex for older PO records.
 2026-04-30 14:42:09 -04:00
Adam Moussa
ec416079f5 Add verified-sites pipeline via DynamoDB Streams 
Enable DynamoDB Streams on purchase-orders table and add a site-extractor
Lambda that extracts Amazon facility codes and addresses from PO ship-to
data, upserting them into a new verified-sites table. Includes a backfill
script for existing POs and upgrades existing Lambdas to arm64 + 60-day
log retention.
 2026-04-30 14:26:53 -04:00
Adam Moussa
fc690dd958 Initial commit: PO email ingestion pipeline 
CDK stack with SES receipt rule, S3 bucket, email processor Lambda
(Claude-powered extraction), web UI Lambda with Function URL, and
DynamoDB for storage. Includes reprocessing script for missed emails.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-07 12:12:30 -04:00