Adam Moussa
094a253c37
Adds canary/ (the planted-vuln corpus from the local-only security-review-testbed, answer-revealing comments stripped so it measures real detection) and canary-meta/ (KEY.md ground truth + CANARY_FLOOR=8, kept OUT of canary/ so detectors never read it). One provider-pattern secret (sk_live_) was sanitized to a non-provider hardcoded key so it stays a CWE-798 finding without tripping push protection. Adds a root .security-review-skip so the org-wide sweep and the local pre-push gate skip this repo's intentional vuln/fixture content; the nightly sweep scans canary/ directly as its recall floor. 20 planted vulns (19 crit/high), 2 decoys, 3 traps.
4.2 KiB
Canary answer key — anti-complacency recall floor
Ground truth for the canary/ corpus. Lives in canary-meta/, NOT in canary/, so it is never in
the detector's scan scope — the nightly sweep points its agentic detectors at canary/ only and reads
this file separately to score the run. Do not move this file under canary/.
The corpus is deliberately vulnerable code (Python Lambda, SAM IaC, React, Node, .NET) with the answer-revealing comments stripped, so a run measures real detection, not comment-reading.
Counts: 20 planted vulns (of which 19 are confirmed critical/high — only #4 PAN-in-logs is MEDIUM), plus 2 safe decoys and 3 traps that must NOT be confirmed.
CANARY_FLOOR = 8 (see canary-meta/CANARY_FLOOR) — PROVISIONAL. Max crit/high recall is 19; the
floor is set conservatively because this corpus has no comment hints and the cloud routine's native
agentic pass is uncalibrated. Re-tune up toward the observed steady-state recall after the first runs
report their numbers. A run that confirms FEWER than the floor in crit/high → COMPLACENCY ALARM.
| # | File | Location | CWE | Severity | What |
|---|---|---|---|---|---|
| 1 | src/payment_handler.py | DB_DSN / INTERNAL_API_KEY |
CWE-798 | HIGH | Hardcoded prod DB creds + API key |
| 2 | src/payment_handler.py | get_payment query |
CWE-89 | CRITICAL | SQL injection via payment_id f-string |
| 3 | src/payment_handler.py | get_payment (no owner check) |
CWE-639 | HIGH | IDOR — caller reads any payment |
| 4 | src/payment_handler.py | logger.info(... card ...) |
CWE-532 | MEDIUM | Full PAN to CloudWatch logs |
| 5 | src/webhook_handler.py | handle_webhook |
CWE-347 | CRITICAL | Webhook HMAC never verified |
| 6 | src/webhook_handler.py | fetch_vendor_logo |
CWE-918 | HIGH | SSRF — unvalidated user URL |
| 7 | src/crypto_utils.py | JWT_SECRET |
CWE-798/321 | HIGH | Hardcoded session signing secret |
| 8 | src/crypto_utils.py | hash_password |
CWE-327/916 | HIGH | Unsalted MD5 password hashing |
| 9 | src/crypto_utils.py | load_session |
CWE-502 | CRITICAL | Insecure deserialization (pickle on cookie) |
| 10 | src/crypto_utils.py | read_report |
CWE-22 | HIGH | Path traversal on report name |
| 11 | infra/template.yaml | PaymentFn policy |
CWE-732 | HIGH | IAM Action:"*" Resource:"*" |
| 11b | infra/template.yaml | ReportsBucket |
CWE-284 | HIGH | Public-access block disabled on PII bucket |
| 11c | infra/template.yaml | AdminSG |
CWE-284 | HIGH | SSH 0.0.0.0/0 |
| 12 | web/PaymentForm.jsx | dangerouslySetInnerHTML |
CWE-79 | HIGH | DOM XSS via vendor note |
| N1 | src/node/orders_api.js | GET /orders/:id query |
CWE-89 | CRITICAL | SQL injection — id concatenated |
| N2 | src/node/orders_api.js | GET /orders/export |
CWE-78 | CRITICAL | Command injection via file into exec |
| N3 | src/node/orders_api.js | db config / JWT_SECRET |
CWE-798 | HIGH | Hardcoded DB password + JWT secret |
| N4 | src/node/orders_api.js | GET /orders/:id (no owner check) |
CWE-639 | HIGH | IDOR — any user reads any order |
| N5 | src/node/orders_api.js | GET /orders/invoice |
CWE-22 | HIGH | Path traversal on invoice name |
| D1 | src/dotnet/PaymentController.cs | GetPayment |
CWE-89 | CRITICAL | SQL injection — id interpolated |
| D2 | src/dotnet/PaymentController.cs | LoadSession |
CWE-502 | CRITICAL | Insecure deserialization (BinaryFormatter) |
| D3 | src/dotnet/PaymentController.cs | ConnStr |
CWE-798 | HIGH | Hardcoded connection string w/ password |
| D4 | src/dotnet/PaymentController.cs | HashPassword |
CWE-327 | HIGH | Unsalted MD5 password hashing |
Decoys — must NOT be flagged:
src/payment_handler.py::list_my_payments— parameterized query, scoped tocaller.src/node/orders_api.jsGET /my-orders— parameterized query, scoped touser_id.
Traps — must NOT be confirmed (test the verifier's kill path):
src/payment_handler.py::payments_by_status— f-string SQL, butstatusis allowlisted againstALLOWED_STATUSESanduser_idis parameterized. SQLi claim must be killed → unverified.infra/template.yamlWebSG—0.0.0.0/0on port 443 is normal public HTTPS; info at most.src/dotnet/PaymentController.cs::ByStatus—statuspassed as aSqlCommandparameter; safe.