feat(agent-team): durable GitHub-issue intake de-dup + intake hardening #32

Merged

amoussa1229 merged 2 commits from feature/agent-team-github-intake-durable-dedup into main

2026-06-22 21:20:44 +00:00

amoussa1229 commented

2026-06-22 21:07:42 +00:00

(Migrated from github.com)

What & why

Phase-1 of wiring all three intake channels into the live agent-team pipeline (Claude Code / GitHub issue / Slack). This PR makes GitHub-issue intake safe to run on a scheduled timer.

The intake poller de-duped ingested issues in an in-memory set that does not survive a process restart. A scheduled/cron intake (each run a fresh process) would therefore re-ingest every still-open labeled issue on every run and spawn duplicate pipeline tasks. The box is read-only (no write token to remove the intake label), so durable de-dup is the only correct guard.

Changes

schema v2 — new ingested_issues(source, issue_id, ingested_at) table + issue_already_ingested / record_issue_ingested / delete_issue_ingested helpers. migrate() adds the table to a legacy v1 DB and restamps; init_db creates it. connect()/WAL unchanged.
github_intake — pluggable IngestStore seam. In-memory default preserved for tests/one-off runs; durable build_ledger_ingest_store for production. Claim-then-do discipline (claim reserves the id before the non-idempotent start_task; release on failure keeps transient failures retryable).
run-team.py intake-github — wires the ledger store keyed by github:owner/repo, making a scheduled timer idempotent across runs.

Security review (`/sh-security-review`) — fixes folded in

Detector fan-out (injection / logic / authz) + proof-or-kill verifier. Findings fixed:

INTAKE-LOGIC-01 (idempotency): claim-then-do + release-on-failure closes the duplicate-on-crash window.
INTAKE-SSRF-001 / PATHSPLICE-002 (CWE-918) in build_default_issue_client: dropped the caller-overridable api_root (hardcoded GITHUB_API_ROOT) and validate owner/repo before URL construction — mirrors the sibling ci_fetcher BLOCK-3/FIX-3.
SQL injection: clean (parameterized). authz allowlist gap: pre-existing, not introduced here; bounded by the Slack answer gate. Tracked as follow-up.

Tests

982 pass, ruff clean. New tests: cross-process duplicate prevention, release-on-failure retry, schema v1→v2 migration, owner/repo + api_root rejection.

Deploy

Inert until the box-side intake timer is installed. Deploy-before-merge: rsync to secrev + coordinator restart (schema v2 table created on next setup) before merge.

## What & why Phase-1 of wiring all three intake channels into the live agent-team pipeline (Claude Code / GitHub issue / Slack). This PR makes **GitHub-issue intake safe to run on a scheduled timer**. The intake poller de-duped ingested issues in an **in-memory set that does not survive a process restart**. A scheduled/cron intake (each run a fresh process) would therefore re-ingest every still-open labeled issue on every run and spawn **duplicate pipeline tasks**. The box is read-only (no write token to remove the intake label), so durable de-dup is the only correct guard. ## Changes - **schema v2** — new `ingested_issues(source, issue_id, ingested_at)` table + `issue_already_ingested` / `record_issue_ingested` / `delete_issue_ingested` helpers. `migrate()` adds the table to a legacy v1 DB and restamps; `init_db` creates it. `connect()`/WAL unchanged. - **`github_intake`** — pluggable `IngestStore` seam. In-memory default preserved for tests/one-off runs; durable `build_ledger_ingest_store` for production. **Claim-then-do** discipline (claim reserves the id *before* the non-idempotent `start_task`; release on failure keeps transient failures retryable). - **`run-team.py intake-github`** — wires the ledger store keyed by `github:owner/repo`, making a scheduled timer idempotent across runs. ## Security review (`/sh-security-review`) — fixes folded in Detector fan-out (injection / logic / authz) + proof-or-kill verifier. Findings fixed: - **INTAKE-LOGIC-01** (idempotency): claim-then-do + release-on-failure closes the duplicate-on-crash window. - **INTAKE-SSRF-001 / PATHSPLICE-002** (CWE-918) in `build_default_issue_client`: dropped the caller-overridable `api_root` (hardcoded `GITHUB_API_ROOT`) and validate `owner`/`repo` before URL construction — mirrors the sibling `ci_fetcher` BLOCK-3/FIX-3. - SQL injection: clean (parameterized). authz allowlist gap: **pre-existing**, not introduced here; bounded by the Slack answer gate. Tracked as follow-up. ## Tests 982 pass, ruff clean. New tests: cross-process duplicate prevention, release-on-failure retry, schema v1→v2 migration, owner/repo + api_root rejection. ## Deploy Inert until the box-side intake timer is installed. Deploy-before-merge: rsync to `secrev` + coordinator restart (schema v2 table created on next `setup`) before merge.

This repo is archived. You cannot comment on pull requests.

No reviewers