44 lines
2.9 KiB
Markdown
44 lines
2.9 KiB
Markdown
|
# doc-drift canary fixtures
|
||
|
|
|
||
|
|
Planted-drift corpus for `checkers/doc-drift.sh --canary` (offline, no network/token).
|
||
|
|
The checker asserts the total drift count equals `EXPECTED_DRIFT_COUNT` (anti-complacency
|
||
|
|
floor, design §6.4). If a check regresses (stops firing), the count drops and the canary
|
||
|
|
FAILS (exit 3).
|
||
|
|
|
||
|
|
doc-drift flags repos whose **architecture moved but the README did not** (design §4,
|
||
|
|
doc-drift row). It is deliberately deterministic and grounded — no fuzzy LLM judgment. The
|
||
|
|
LLM judge layer (Gemini large-context) is a later enhancement and is inert offline (see the
|
||
|
|
`maybe_judge` stub in the checker).
|
||
|
|
|
||
|
|
Each fixture is a real git checkout (its `.git` is shipped as `dotgit/` so it commits into
|
||
|
|
THIS repo without becoming a nested submodule; the checker renames it back to `.git/` at run
|
||
|
|
time, the same trick `compliance-drift.sh` / `dependency-cve.sh` use). Real commit history is
|
||
|
|
required because the staleness check reads `git log` dates.
|
||
|
|
|
||
|
|
## Detected drift (the deterministic checklist)
|
||
|
|
|
||
|
|
| Check | Rule cited | What fires |
|
||
|
|
|---|---|---|
|
||
|
|
| `readme-omits-component` | global CLAUDE.md: "README must accurately describe architecture, services, data flow" | A README exists but omits mention of a major existing component present in the tree: a top-level service dir, a SAM/CDK stack (`template.yaml` / `app.py` / `cdk.json`), a Lambda handler dir, or an `openapi`/`docs` API spec. |
|
||
|
|
| `readme-stale-vs-code` | global CLAUDE.md: "update the README in the same commit" as functionality changes | The README's last-touched commit is far older than the newest code commit (≥ `DOC_DRIFT_STALE_DAYS` days) AND ≥ `DOC_DRIFT_STALE_COMMITS` substantial code commits landed after the README was last touched. |
|
||
|
|
|
||
|
|
A repo with **no README at all** is SKIPPED by doc-drift, not flagged — `readme-present` is
|
||
|
|
`compliance-drift.sh`'s job, and double-flagging would be a false alarm
|
||
|
|
(memory `feedback_cloudwatch_alarms`).
|
||
|
|
|
||
|
|
## Fixtures
|
||
|
|
|
||
|
|
| Fixture | Planted drift | Count |
|
||
|
|
|---|---|---|
|
||
|
|
| `clean-repo` | none — README names every component (`api/`, the SAM stack, `handlers/`, `openapi/`) and the README was committed alongside the code | 0 |
|
||
|
|
| `drift-omits-repo` | README mentions only `notifier-service`; omits `payments-service/`, the SAM `template.yaml` stack, and the `handlers/charge` Lambda dir | 3 |
|
||
|
|
| `drift-stale-repo` | README names its one component (no omission) but was last touched 2026-01-05 while 5 substantial code commits landed in 2026-06 — stale | 1 |
|
||
|
|
| `no-readme-repo` | no README — doc-drift SKIPS it (must NOT fire; compliance-drift owns this) | 0 |
|
||
|
|
|
||
|
|
Total = **4** (`EXPECTED_DRIFT_COUNT`). The canary pins the staleness thresholds it was
|
||
|
|
authored against (`DOC_DRIFT_STALE_DAYS`, `DOC_DRIFT_STALE_COMMITS`) internally so it is
|
||
|
|
deterministic regardless of the operator's env.
|
||
|
|
|
||
|
|
When you add/remove a check or fixture, update both the fixture and `EXPECTED_DRIFT_COUNT`
|
||
|
|
in the same commit (the canary edit is itself caught on the next run — design §6.4).
|