mirror of
https://github.com/Sea-Haven-Industries/security-review.git
synced 2026-09-30 06:53:15 +00:00
Adam Moussa
094a253c37
Adds canary/ (the planted-vuln corpus from the local-only security-review-testbed, answer-revealing comments stripped so it measures real detection) and canary-meta/ (KEY.md ground truth + CANARY_FLOOR=8, kept OUT of canary/ so detectors never read it). One provider-pattern secret (sk_live_) was sanitized to a non-provider hardcoded key so it stays a CWE-798 finding without tripping push protection. Adds a root .security-review-skip so the org-wide sweep and the local pre-push gate skip this repo's intentional vuln/fixture content; the nightly sweep scans canary/ directly as its recall floor. 20 planted vulns (19 crit/high), 2 decoys, 3 traps.
52 lines
4.2 KiB
Markdown
52 lines
4.2 KiB
Markdown
# Canary answer key — anti-complacency recall floor
|
|
|
|
Ground truth for the `canary/` corpus. **Lives in `canary-meta/`, NOT in `canary/`**, so it is never in
|
|
the detector's scan scope — the nightly sweep points its agentic detectors at `canary/` only and reads
|
|
this file separately to score the run. Do not move this file under `canary/`.
|
|
|
|
The corpus is deliberately vulnerable code (Python Lambda, SAM IaC, React, Node, .NET) with the
|
|
answer-revealing comments stripped, so a run measures real detection, not comment-reading.
|
|
|
|
**Counts:** 20 planted vulns (of which **19 are confirmed critical/high** — only #4 PAN-in-logs is
|
|
MEDIUM), plus 2 safe decoys and 3 traps that must NOT be confirmed.
|
|
|
|
**`CANARY_FLOOR = 8`** (see `canary-meta/CANARY_FLOOR`) — PROVISIONAL. Max crit/high recall is 19; the
|
|
floor is set conservatively because this corpus has no comment hints and the cloud routine's native
|
|
agentic pass is uncalibrated. Re-tune up toward the observed steady-state recall after the first runs
|
|
report their numbers. A run that confirms FEWER than the floor in crit/high → COMPLACENCY ALARM.
|
|
|
|
| # | File | Location | CWE | Severity | What |
|
|
|---|---|---|---|---|---|
|
|
| 1 | src/payment_handler.py | `DB_DSN` / `INTERNAL_API_KEY` | CWE-798 | HIGH | Hardcoded prod DB creds + API key |
|
|
| 2 | src/payment_handler.py | `get_payment` query | CWE-89 | CRITICAL | SQL injection via `payment_id` f-string |
|
|
| 3 | src/payment_handler.py | `get_payment` (no owner check) | CWE-639 | HIGH | IDOR — caller reads any payment |
|
|
| 4 | src/payment_handler.py | `logger.info(... card ...)` | CWE-532 | MEDIUM | Full PAN to CloudWatch logs |
|
|
| 5 | src/webhook_handler.py | `handle_webhook` | CWE-347 | CRITICAL | Webhook HMAC never verified |
|
|
| 6 | src/webhook_handler.py | `fetch_vendor_logo` | CWE-918 | HIGH | SSRF — unvalidated user URL |
|
|
| 7 | src/crypto_utils.py | `JWT_SECRET` | CWE-798/321 | HIGH | Hardcoded session signing secret |
|
|
| 8 | src/crypto_utils.py | `hash_password` | CWE-327/916 | HIGH | Unsalted MD5 password hashing |
|
|
| 9 | src/crypto_utils.py | `load_session` | CWE-502 | CRITICAL | Insecure deserialization (pickle on cookie) |
|
|
| 10 | src/crypto_utils.py | `read_report` | CWE-22 | HIGH | Path traversal on report name |
|
|
| 11 | infra/template.yaml | `PaymentFn` policy | CWE-732 | HIGH | IAM `Action:"*" Resource:"*"` |
|
|
| 11b | infra/template.yaml | `ReportsBucket` | CWE-284 | HIGH | Public-access block disabled on PII bucket |
|
|
| 11c | infra/template.yaml | `AdminSG` | CWE-284 | HIGH | SSH 0.0.0.0/0 |
|
|
| 12 | web/PaymentForm.jsx | `dangerouslySetInnerHTML` | CWE-79 | HIGH | DOM XSS via vendor `note` |
|
|
| N1 | src/node/orders_api.js | `GET /orders/:id` query | CWE-89 | CRITICAL | SQL injection — `id` concatenated |
|
|
| N2 | src/node/orders_api.js | `GET /orders/export` | CWE-78 | CRITICAL | Command injection via `file` into `exec` |
|
|
| N3 | src/node/orders_api.js | `db` config / `JWT_SECRET` | CWE-798 | HIGH | Hardcoded DB password + JWT secret |
|
|
| N4 | src/node/orders_api.js | `GET /orders/:id` (no owner check) | CWE-639 | HIGH | IDOR — any user reads any order |
|
|
| N5 | src/node/orders_api.js | `GET /orders/invoice` | CWE-22 | HIGH | Path traversal on `invoice` name |
|
|
| D1 | src/dotnet/PaymentController.cs | `GetPayment` | CWE-89 | CRITICAL | SQL injection — `id` interpolated |
|
|
| D2 | src/dotnet/PaymentController.cs | `LoadSession` | CWE-502 | CRITICAL | Insecure deserialization (BinaryFormatter) |
|
|
| D3 | src/dotnet/PaymentController.cs | `ConnStr` | CWE-798 | HIGH | Hardcoded connection string w/ password |
|
|
| D4 | src/dotnet/PaymentController.cs | `HashPassword` | CWE-327 | HIGH | Unsalted MD5 password hashing |
|
|
|
|
**Decoys — must NOT be flagged:**
|
|
- `src/payment_handler.py::list_my_payments` — parameterized query, scoped to `caller`.
|
|
- `src/node/orders_api.js` `GET /my-orders` — parameterized query, scoped to `user_id`.
|
|
|
|
**Traps — must NOT be confirmed (test the verifier's kill path):**
|
|
- `src/payment_handler.py::payments_by_status` — f-string SQL, but `status` is allowlisted against
|
|
`ALLOWED_STATUSES` and `user_id` is parameterized. SQLi claim must be killed → unverified.
|
|
- `infra/template.yaml` `WebSG` — `0.0.0.0/0` on port 443 is normal public HTTPS; info at most.
|
|
- `src/dotnet/PaymentController.cs::ByStatus` — `status` passed as a `SqlCommand` parameter; safe.
|