open-swe/docs/upstream-sync/cherry-pick-hook-plan.md
Adam Moussa dfdd41879c
feat(infra): upstream-sync triage ledger, cherry-pick hooks, and git cp (#118)
* docs(upstream-sync): add triage ledger + cherry-pick hook plan

Seeds the upstream triage ledger (52 diverged commits from langchain-ai/open-swe
categorized: landed/won't-merge/deferred/untriaged) and the design plan for a
git-hook mechanism to keep it in sync during cherry-picks.

* feat(upstream-sync): jsonl-backed triage ledger + generator CLI

triage.jsonl is now the source of truth (52 rows migrated from triage.md);
triage.md is generated with a do-not-edit banner. scripts/triage.py provides
migrate/generate/reconcile/lookup/check-reject/set; make triage-render/check/reconcile
added (triage-check is CI-safe staleness gate). Stdlib-only so git hooks can call it.

* feat(upstream-sync): cherry-pick triage git hooks + git cp wrapper

post-commit journals each -x pick to an untracked .git-local journal; prepare-commit-msg
hard-blocks known-reject picks (commit-msg is a secondary backstop — clean picks skip it on
git 2.50.1), overridable via git cp --force / SH_CHERRYPICK_ALLOW_REJECT=1 /
sh.cherrypick.blockRejects=false. git-cp is the pre-apply guard + auto-reconcile. pre-push is
a SHIM that re-execs the global Sea Haven security pre-push so core.hooksPath=.githooks does
not shadow it; install-hooks.sh verifies that shim FIRST and refuses if it is missing.

* docs(upstream-sync): correct hook plan + git cp runbook

Record the verified git 2.50.1 finding that clean cherry-picks skip commit-msg, so the block
lives in prepare-commit-msg; note the locked HARD-BLOCK-by-default reject policy. CHERRYPICK.md
now leads with make install-hooks + git cp and explains the generated-md ledger.

* chore(upstream-sync): mark #1651 landed (Bedrock family fix on gateway-routing)

* docs(upstream-sync): rewrite CHERRYPICK.md as a repo-specific runbook

* feat(upstream-sync): add triage.py sync + make triage-sync

Discovers commits on dev..upstream/main not yet in the ledger and appends them
as untriaged (PR # and subject parsed from each commit), then bumps _meta
'last synced' to the upstream tip and regenerates triage.md. Closes the
discovery side of the workflow: triage-sync to pull in new work, git cp to land it.

* docs(upstream-sync): move cherry-pick runbook to PR1 branch as cherry-pick-runbook.md
2026-07-03 11:46:00 -04:00

24 KiB

Cherry-pick triage-ledger sync — design plan

Status: design only (nothing here is installed or wired yet). This document is the build spec for a git-hook mechanism that keeps the upstream-sync triage ledger in sync during git cherry-pick -x, identically for a human at the terminal and for Claude Code driving git. Companion runbook: ../../CHERRYPICK.md.


1. Problem statement and the "no cherry-pick hook exists" reality

This is a long-lived fork of langchain-ai/open-swe (remote upstream). We pull upstream commits one at a time via git cherry-pick -x <sha>. A human keeps a triage ledger at docs/upstream-sync/triage.md recording, per upstream SHA, a disposition:

  • Landed — cherry-picked into the fork.
  • Won't-merge — already-in-dev / regression / tooling-rejected (a decided no).
  • Deferred→<branch> — parked for later on a named branch.
  • Untriaged — seen but not yet decided.

The ledger keys every row on the upstream SHA because it is stable; cherry-pick rewrites the SHA locally, so the local SHA is not a durable key.

We want two behaviors during a cherry-pick:

  1. Auto-land: when a pick succeeds, move that upstream SHA into the Landed section from wherever it currently sits.
  2. Reject-warning: when someone cherry-picks a SHA the ledger marks Won't-merge, warn them as early as possible (ideally before the change is applied).

The hard reality

Git has no pre-cherry-pick or post-cherry-pick hook. The only hooks that fire during a cherry-pick are, per successfully-applied commit:

prepare-commit-msg  →  commit-msg  →  post-commit

There is no native hook at the start of a cherry-pick and no hook that sees the list of SHAs about to be picked. Everything below is designed around that fact — we do not invent a hook that does not exist.

Build correction (verified on git 2.50.1). A clean cherry-pick auto-commit runs only prepare-commit-msg and post-commit — it skips pre-commit and commit-msg. commit-msg fires only on the git commit-backed path (a normal commit, or git cherry-pick --continue after a conflict). So the reject hard-block must live in prepare-commit-msg (fires on every pick, clean or resolved), with commit-msg kept only as a secondary backstop. The original plan named commit-msg as the primary gate; that would silently miss every clean pick. prepare-commit-msg returning non-zero aborts the commit cleanly and leaves CHERRY_PICK_HEAD in place, so --abort/--skip/--continue still recover.

Two signals are load-bearing:

  • .git/CHERRY_PICK_HEAD exists while a pick is in progress and contains the full upstream SHA being applied. It is present at prepare-commit-msg and commit-msg time (before the commit object is finalized) and is gone by post-commit.
  • The -x trailer (cherry picked from commit <full-sha>) is written into the commit message by git cherry-pick -x. It is present in the message file at commit-msg time and is recoverable from git log -1 --format=%B at post-commit time.

So commit-msg is the earliest gate that can act with knowledge of the upstream SHA, and post-commit is the only reliable "the pick actually landed" signal.


2. Chosen architecture

Two hooks plus a thin wrapper and a small Python CLI. Division of labour:

Component Fires / runs Job Touches tracked files?
.githooks/prepare-commit-msg per pick (incl. clean auto-commit), before commit object is finalized Behavior #2 primary hard-block: recover upstream SHA, block if ledger says Won't-merge (override to allow) No
.githooks/commit-msg git commit-backed path only (--continue, normal commit) Behavior #2 secondary backstop (same guard); clean picks skip this hook No
.githooks/_reject_guard.sh sourced by both hooks above Shared reject-guard function (not a hook — git ignores non-hook-named files) No
.githooks/post-commit per pick, after commit object exists Behavior #1: recover upstream SHA + new local SHA, append to an untracked .git/ journal No
scripts/triage.py reconcile end of a pick run (auto under wrapper, one command otherwise) Drain journal → set those SHAs to Landed in triage.jsonl → regenerate triage.md → stage both Yes (once, as a follow-up commit)
scripts/git-cp (wrapper / git cp alias) user-invoked instead of raw cherry-pick True pre-warning: check ledger before any tree change; then cherry-pick -x; then auto-reconcile via reconcile
.githooks/pre-push on push Shim that re-invokes the global Sea Haven security pre-push (see §6) No

Why hooks never edit the tracked ledger directly: editing triage.md/triage.jsonl inside a hook during a multi-pick sequence leaves the tracked file dirty between picks (see §4). We avoid that entirely — hooks only ever append to an untracked .git/-local journal; the tracked ledger is mutated exactly once, by an explicit reconcile, as its own commit.

End-to-end: single pick

$ git cp -x A            # wrapper (recommended). Raw `git cherry-pick -x A` also works.
      │
      │ (wrapper) pre-check A against triage.jsonl
      │    └─ A is Won't-merge → print reason, require --force to proceed  ◄─ true pre-warning
      │
      ├─ git cherry-pick -x A
      │      │  applies A's diff to index/worktree
      │      ├─ prepare-commit-msg   (unused)
      │      ├─ commit-msg           CHERRY_PICK_HEAD=A present → look up A
      │      │                        └─ Won't-merge? loud stderr warning (exit 0 by default)
      │      │                        finalize commit object A' with -x trailer
      │      └─ post-commit          CHERRY_PICK_HEAD gone; parse -x trailer → A
      │                               append "A<TAB>A'<TAB>landed" to .git/sh-cherrypick-journal
      │
      └─ (wrapper) scripts/triage.py reconcile
             drain journal → triage.jsonl: A→landed (local_sha=A')
             regenerate triage.md → git add both → report "1 landed; commit the ledger"

Raw git cherry-pick -x A runs everything except the wrapper's pre-check and the auto-reconcile; post-commit still journals, and it prints N pick(s) journaled — run: make triage-reconcile.

End-to-end: multi-pick sequence git cp -x A B C

wrapper pre-check A,B,C  ── any Won't-merge?  → list them, require --force
      │
git cherry-pick -x A B C          (git sequencer)
   A → commit-msg(warn?) → post-commit → journal: A A'
   B → commit-msg(warn?) → post-commit → journal: B B'
   C → commit-msg(warn?) → post-commit → journal: C C'
      │   (tracked ledger NEVER touched mid-sequence → no dirty-tree hazard, §4)
      │
wrapper → scripts/triage.py reconcile
   drain {A,B,C} → triage.jsonl all→landed → regenerate triage.md → stage → one report

If the sequence stops on a conflict at B: A is already journaled. The user resolves and git cherry-pick --continue (B and C journal as they land). Because reconcile is journal-driven and idempotent, running it after the sequence finally completes lands exactly A, B, C once. Under raw cherry-pick the user runs make triage-reconcile at the end; the journal survived the conflict pause because it lives in .git/, untouched by the sequencer.


Decision: triage.jsonl is the source of truth; triage.md is generated from it. Do not have hooks parse/edit the human markdown table.

  • docs/upstream-sync/triage.jsonl — one JSON object per line, the canonical record.
  • docs/upstream-sync/triage.md — generated view with a <!-- GENERATED … do not edit --> banner, rendered by scripts/triage.py render. Grouped into the same sections/columns the human ledger uses today (| sha | #pr | subject | reason |).

Record schema (one line):

{"sha":"<full-upstream-sha>","pr":123,"subject":"...","disposition":"landed",
 "reason":"...","deferred_branch":null,"local_sha":"<rewritten-sha-or-null>",
 "updated":"2026-07-02T00:00:00Z"}

disposition ∈ {landed, wont-merge, deferred, untriaged}; deferred_branch set only when deferred. Render maps deferred rows into a Deferred→<branch> subsection.

Why not edit the markdown directly

  • Editing a human-formatted markdown table from a shell hook is the fragile path the brief warns about: alignment, escaped pipes in subjects, multi-line reasons, section boundaries, and hand-edits all break naive sed/awk. One malformed edit corrupts the ledger.
  • JSONL is append/patch-friendly, trivially greppable (grep '"sha":"<sha>"' for the reject-check), has clean line-oriented diffs, and is mutated safely by a tiny Python with real JSON parsing. The repo already has a Python scripts/ dir and uv, so a scripts/triage.py CLI is idiomatic here and far more robust than shell string-surgery.
  • JSONL over TSV: dispositions carry structure (deferred_branch, local_sha, pr) and reason/subject are free text that can contain tabs — TSV would need escaping rules JSONL gives for free.
  • Humans still get a readable, reviewable triage.md in PRs; they just edit it through triage.jsonl (directly, or via scripts/triage.py set <sha> …). A make triage-check in CI fails if triage.md is stale vs triage.jsonl, so the generated view can never drift.

Migration from today's markdown ledger: a one-shot scripts/triage.py import triage.md parses the current hand-written table into triage.jsonl, after which triage.md becomes a generated artifact. This is a build task, not a runtime dependency.


4. Behavior #1 — auto-move to Landed, step by step

SHA recovery. post-commit runs after the commit object exists and CHERRY_PICK_HEAD is already gone, so recover the upstream SHA from the -x trailer:

msg="$(git log -1 --format=%B)"
up_sha="$(printf '%s\n' "$msg" | sed -n 's/.*cherry picked from commit \([0-9a-f]\{40\}\).*/\1/p' | tail -1)"
[ -z "$up_sha" ] && exit 0          # not a cherry-pick (or no -x) → no-op, normal commits are untouched
local_sha="$(git rev-parse HEAD)"

Journal, don't edit. Append to an untracked journal and dedupe:

.git/sh-cherrypick-journal          # <upstream_sha>\t<local_sha>\t<iso8601>, one line per pick

The journal lives under .git/ — outside version control and outside the working tree — so it is invisible to git status, never conflicts with an incoming pick, and survives conflict pauses in a sequence. post-commit does nothing else.

Multi-pick dirty-tree handling (the crux). If the hook instead edited the tracked ledger in place, every intermediate pick would leave docs/upstream-sync/triage.* modified and unstaged. Analysis:

  • It would not hard-break the sequence: cherry-pick applies the next commit's diff to the index, and upstream commits never touch our fork-only docs/upstream-sync/ files, so a dirty ledger is a file the incoming pick doesn't care about — git allows that.
  • But it is still the wrong design: git status is polluted mid-run, the ledger edits get interleaved with pick state, and — worst case — folding a ledger edit into a cherry-picked commit would pollute the pristine -x provenance we depend on. There is also no reliable in-hook signal for "this is the last pick" (.git/sequencer/ is torn down racily, and a single git cherry-pick A may never create a sequencer dir at all), so a hook cannot know when to do the "final" reconcile.

So the tracked ledger is mutated once, outside any hook, by reconcile:

scripts/triage.py reconcile
  read .git/sh-cherrypick-journal
  for each (upstream_sha, local_sha): triage.jsonl[sha].disposition = landed
                                      triage.jsonl[sha].local_sha   = local_sha
  render triage.md from triage.jsonl
  git add docs/upstream-sync/triage.jsonl docs/upstream-sync/triage.md
  truncate the journal
  print summary  (does NOT commit — the human/agent commits the ledger separately)

reconcile runs automatically as the last step of the git cp wrapper (fully hands-off for wrapper users and for Claude Code when it calls the wrapper). For raw git cherry-pick, post-commit prints N pick(s) journaled — run: make triage-reconcile, and the user runs it once at the end. Reconcile is idempotent: a drained journal reconciles to a no-op, and re-landing an already-landed SHA is a no-op, so double-running is safe.

The ledger update lands as its own commit, keeping cherry-picked commits pristine.


5. Behavior #2 — block on known-reject: honest verdict

Locked decision (overrides the recommendation below): HARD-BLOCK by default. Picking a Won't-merge SHA is blocked by both the git cp pre-apply check and the prepare-commit-msg hook (the plan text below still discusses warn-only as an option; the shipped default is block). Documented overrides: git cp --force, env SH_CHERRYPICK_ALLOW_REJECT=1, or repo-wide git config sh.cherrypick.blockRejects false.

Constraint: by commit-msg the pick's diff is already staged in the index/worktree; by post-commit the commit exists. No hook can warn before the change is applied to the tree — the earliest a hook sees the SHA is commit-msg, and by then the diff is staged (the commit just isn't finalized). A true pre-application warning is impossible with hooks alone.

Three honest options:

  • (a) commit-msg hard-block (exit 1). Aborts the commit cleanly: the commit object is not created, CHERRY_PICK_HEAD stays, the index holds the applied diff, and the user recovers with git cherry-pick --abort / --skip / --continue. In a sequence it stops at that commit. Downsides: it's post-apply (tree already changed), and hard-blocking a decided SHA that the maintainer legitimately wants to re-pick is annoying and, if the hook ever misfires, wedges a pick.
  • (b) commit-msg warn-only (exit 0). Loud stderr warning with the recorded reason, but the commit proceeds. Never wedges anything. Downside: also post-apply, and easy to miss in a multi-pick scroll.
  • (c) Wrapper pre-check. git cp reads triage.jsonl before calling cherry-pick and refuses (or prompts) on a Won't-merge SHA — a genuine pre-apply warning, before any tree change. Downside: only fires when people use the wrapper; raw git cherry-pick bypasses it.

Recommendation: hook + wrapper — do both, with these defaults.

  1. scripts/git-cp wrapper (primary, true pre-warning). Before invoking cherry-pick, look up every requested SHA in triage.jsonl. If any is wont-merge, print the SHA, subject, and reason and abort unless --force is passed. This is the real "stop before you apply a known-no" guard, and it is the path we point both humans (CHERRYPICK.md) and Claude Code at. Expose it as git cp via git config alias.cp '!bash scripts/git-cp'.
  2. .githooks/commit-msg backstop (catches raw git cherry-pick). Recover the upstream SHA from CHERRY_PICK_HEAD (fallback: the -x trailer in message file $1); if triage.jsonl marks it wont-merge, print a loud stderr warning with the reason. Default: warn and exit 0 (non-blocking). Opt-in hard-block via git config sh.cherrypick.blockRejects true for anyone who wants option (a). Blocking is off by default so the hook can never wedge a legitimate pick or a normal commit.

Rationale: the wrapper gives the real pre-warning we actually want; the hook guarantees that even a raw git cherry-pick (or Claude Code shelling straight to git) still gets a visible signal, without ever risking a wedged pick by default.


6. Installation / bootstrapping via core.hooksPath

Hooks live in an in-repo, version-controlled .githooks/ so they're shared. Activation is per-clone:

git config core.hooksPath .githooks

Git does not auto-adopt a repo's core.hooksPath (that would let a clone run arbitrary code on checkout), so this one line is unavoidable per clone. Automate/ship it via:

  • scripts/install-hooks.sh — sets core.hooksPath .githooks, chmod +x .githooks/*, and prints the global-hook note below.
  • make hooks target calling that script; optionally invoke it from make install so a standard setup wires hooks too. Document the one line in CHERRYPICK.md / README.

CRITICAL: collision with the global Sea Haven security pre-push

This machine has a global hook path already configured:

core.hooksPath = ~/.config/git/hooks     # holds the mandatory Sea Haven security pre-push gate

core.hooksPath is a single directory, not a search path — a repo-level core.hooksPath=.githooks overrides the global one for this repo and would silently disable the security pre-push gate here. That is a compliance violation, not a cosmetic issue.

Mitigation (required): ship a pre-push shim in .githooks/ that re-invokes the global hook.

# .githooks/pre-push
#!/usr/bin/env bash
GLOBAL="${SH_GLOBAL_HOOKS:-$HOME/.config/git/hooks}/pre-push"
[ -x "$GLOBAL" ] && exec "$GLOBAL" "$@"
exit 0    # no global hook present → succeed, don't block the push

install-hooks.sh must detect a pre-existing global core.hooksPath, confirm the shim is in place, and warn loudly if the global security hook exists but the shim is missing. If Sea Haven ever adds more global hooks, add a matching shim for each (or a generic dispatcher that execs every same-named hook under the global dir). This keeps the security gate intact while the cherry-pick hooks run.


7. Failure modes and safety guarantees

  • Never abort/corrupt a normal commit. Every hook first checks the cherry-pick signal (CHERRY_PICK_HEAD for commit-msg, the -x trailer for post-commit) and no-ops instantly otherwise. A plain git commit never reaches ledger logic.
  • Fail safe. All hooks run set -uo pipefail and wrap their body so any internal error (missing/mangled triage.jsonl, no Python, unreadable journal) results in exit 0 — a hook failure must never abort the user's git operation. The only path that can exit non-zero is the opt-in blockRejects block, and that is a clean, recoverable cherry-pick abort.
  • No half-written tracked ledger. Hooks only append to the untracked .git/ journal; the tracked ledger changes solely inside reconcile, which writes triage.jsonl + regenerated triage.md atomically (write temp → os.replace) and stages them. A crash mid-reconcile leaves the journal intact (source of truth for a re-run) and the tracked files either fully old or fully new.
  • No mid-sequence dirty-tree hazard (see §4): the tracked ledger is never touched during a multi-pick run.
  • Idempotent + crash-tolerant. Journal lines are deduped by upstream SHA; reconcile on a drained journal is a no-op; re-landing an already-landed SHA is a no-op. Safe to run twice, and safe across a conflict pause (journal survives in .git/).
  • Malformed source of truth. If triage.jsonl fails to parse, reconcile aborts without writing and leaves the journal intact; commit-msg/post-commit degrade to a stderr note and exit 0.
  • Unknown SHA. A picked SHA absent from the ledger is journaled and, on reconcile, inserted as a new landed row (subject/PR backfilled from git show), so the ledger self-heals instead of silently dropping the pick.
  • Missing Python / hooks not installed. If core.hooksPath isn't set, nothing runs and cherry-pick behaves normally (ledger just goes stale until someone reconciles) — no breakage.

8. File/directory layout and implementation checklist

Files to create

.githooks/
  prepare-commit-msg       # behavior #2 PRIMARY hard-block (fires on clean picks too)
  commit-msg               # behavior #2 secondary backstop (git commit / --continue path)
  _reject_guard.sh         # shared guard sourced by the two hooks above (not a hook itself)
  post-commit              # behavior #1: recover SHA from -x trailer, append to .git journal
  pre-push                 # SHIM → global Sea Haven security pre-push (§6)
docs/upstream-sync/
  cherry-pick-hook-plan.md # this document
  triage.jsonl             # SOURCE OF TRUTH (created by the import step)
  triage.md                # GENERATED view (do-not-edit banner)
scripts/
  triage.py                # CLI: import | set | check-reject | reconcile | render | lint
  git-cp                   # wrapper: pre-check ledger → cherry-pick -x → reconcile
  install-hooks.sh         # sets core.hooksPath=.githooks, verifies pre-push shim vs global
# runtime, untracked (add to .gitignore is unnecessary — it lives under .git/):
.git/sh-cherrypick-journal

scripts/triage.py subcommands

  • import <triage.md> — one-shot migration of the current hand-written ledger → triage.jsonl.
  • set <sha> --disposition … [--pr … --subject … --reason … --branch …] — human/agent edit path.
  • check-reject <sha> — exit non-zero + print reason iff wont-merge (used by hook + wrapper).
  • reconcile — drain .git/sh-cherrypick-journal → mark landed → render → stage → summarize.
  • render [--check] — regenerate triage.md; --check fails if stale (for CI).
  • lint — validate triage.jsonl schema/dispositions.

Makefile targets

  • hooks → bash scripts/install-hooks.sh
  • triage-reconcile → uv run scripts/triage.py reconcile
  • triage-render → uv run scripts/triage.py render
  • triage-check → uv run scripts/triage.py render --check (wire into CI)

Build checklist (ordered, no re-deciding required)

  1. Ledger model. Write scripts/triage.py with the schema in §3; implement render (grouped sections + | sha | #pr | subject | reason |, do-not-edit banner) and lint.
  2. Migrate. triage.py import docs/upstream-sync/triage.md → commit triage.jsonl + regenerated triage.md; from here triage.md is generated.
  3. post-commit hook. -x trailer recovery (§4), append <up>\t<local>\t<ts> to .git/sh-cherrypick-journal (deduped), no-op on non-cherry-pick, exit 0 on any error, print the "run reconcile" reminder.
  4. reconcile. Implement triage.py reconcile (drain → land → atomic render → stage → truncate journal → summary; idempotent; self-heal unknown SHAs).
  5. commit-msg hook. Recover SHA from CHERRY_PICK_HEAD (fallback -x trailer in $1); check-reject; default warn + exit 0; opt-in sh.cherrypick.blockRejects → exit 1.
  6. git-cp wrapper + alias. Pre-check all SHAs (abort on wont-merge unless --force) → git cherry-pick -x "$@" → triage.py reconcile. Ship alias.cp setup in install script.
  7. pre-push shim (§6). .githooks/pre-push execs the global security hook; make it the first thing install-hooks.sh verifies.
  8. install-hooks.sh + make hooks. Set core.hooksPath=.githooks, chmod +x, set git cp alias, detect/reconcile the global-hooksPath collision, warn if the security shim is missing.
  9. Docs. Update CHERRYPICK.md to lead with git cp, note the one-time make hooks, and explain the warn/block behavior. Add make triage-check to CI so triage.md never drifts.
  10. Tests. Cover: single pick lands; multi-pick sequence lands all once; conflict-pause then --continue still lands correctly; reject warning fires (warn and block modes); normal non-cherry-pick commit is untouched; hook errors never abort git; reconcile is idempotent; pre-push shim delegates to the global security hook.

Open items for the human to confirm before build

  • Default reject policy: warn-only (recommended) vs block-by-default. Plan assumes warn-only with opt-in block.
  • Branch target for the reconcile commit: picks land on chore/cherry-pick-* off dev (per CHERRYPICK.md); confirm the ledger commit rides the same branch/PR.