a6bb1adfbc
- Add scrape-fill-chunk.mjs: reusable chunk-based Payee Central scraper with resume support and concurrent instance capability - Add prompts/po-email-parser.md: LLM prompt for Coupa PO email parsing with site code extraction patterns, skip list, and trade classification - Add README.md with architecture docs, DynamoDB schema, and script index - Track output/site-state-extra-mapping.json (392 facility codes) - Remove scrape-payee.mjs and scrape-payee-notfound.mjs (superseded by chunk scraper)
57 lines
2.3 KiB
Markdown
57 lines
2.3 KiB
Markdown
# amazon-po-parser
|
|
|
|
Tools for ingesting, scraping, and enriching Amazon purchase order data from Coupa emails and Payee Central.
|
|
|
|
## Architecture
|
|
|
|
- **Data source**: Coupa PO notification emails → parsed into structured JSON
|
|
- **Enrichment**: Playwright scrapers hit Amazon Payee Central to resolve site codes, ship-to addresses, and line items
|
|
- **Storage**: DynamoDB `purchase-orders` table (us-east-1) is the single source of truth
|
|
- **Mapping**: `output/site-state-extra-mapping.json` — curated Amazon facility code → US state mapping (392 codes)
|
|
|
|
## Scripts
|
|
|
|
| Script | Purpose |
|
|
|---|---|
|
|
| `parse-mbox.py` | Parse Coupa PO emails from MBOX export into structured JSON |
|
|
| `scrape.mjs` | Single-browser Payee Central scraper |
|
|
| `scrape-payee-chunk.mjs` | Chunk-based concurrent scraper (run N instances in parallel) |
|
|
| `scrape-fill-chunk.mjs` | Lightweight scraper for backfilling site codes and ship-to addresses |
|
|
| `gen-po-list.mjs` | Generate PO lists from invoice spreadsheets |
|
|
| `monitor.mjs` | Monitor scraper progress across chunks |
|
|
|
|
## DynamoDB Schema
|
|
|
|
Table: `purchase-orders` | Key: `po_number` (string)
|
|
|
|
| Field | Type | Source |
|
|
|---|---|---|
|
|
| `po_number` | string | Email/Coupa |
|
|
| `site_code` | string | Payee Central scrape |
|
|
| `state` | string | Derived from site_code mapping |
|
|
| `ship_to_raw` | string | Payee Central scrape |
|
|
| `line_items` | list | Email parse + Payee Central |
|
|
| `trade` | string | Classified from descriptions |
|
|
| `trades` | list | All trades for multi-line POs |
|
|
| `fiscal_year` | string | Order date year |
|
|
| `coupa_category` | string | Coupa commodity category |
|
|
| `source_system` | string | "coupa" |
|
|
| `data_source` | string | "email+payee_scrape" |
|
|
| `invoice_total` | string | Downstream invoice matching |
|
|
| `invoice_count` | number | Downstream invoice matching |
|
|
| `order_date` | string | Email parse |
|
|
| `po_status` | string | Email parse |
|
|
| `payment_terms` | string | Email parse |
|
|
| `requisition_number` | string | Email parse |
|
|
|
|
## Prompt
|
|
|
|
`prompts/po-email-parser.md` contains the LLM prompt for parsing incoming Coupa PO emails. It includes site code extraction patterns, a skip list for non-site codes, trade classification rules, and ship-to address parsing guidance.
|
|
|
|
## Setup
|
|
|
|
```bash
|
|
npm install # playwright
|
|
```
|
|
|
|
Requires a Chrome profile logged into Amazon Payee Central at `./chrome-profile/`.
|