feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
"""Tool: ``publish_review``. Post the findings list to GitHub as a PR Review."""
|
|
|
|
|
|
|
|
|
|
from __future__ import annotations
|
|
|
|
|
|
2026-07-20 15:28:00 -04:00
|
|
|
import logging
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
from typing import Any
|
|
|
|
|
|
|
|
|
|
from langgraph.config import get_config
|
|
|
|
|
|
2026-05-28 13:48:14 -07:00
|
|
|
from ..dashboard.team_settings import get_team_review_trace_links_enabled
|
refactor: consolidate reviewer modules into agent/review/
Part of the domain-reorg adoption (build plan step C2): fork content,
upstream layout. Nine 1:1 module moves (reviewer_diff/eval_store/
findings/groups/publish/reconcile/trace_context + review_style_
collector/guidance) into agent/review/, with internal relative
imports re-wired to the new package depth. agent/review/__init__.py
mirrors upstream's thin re-export shim (one of the 21 verified "A"
structural adds).
Rewrote the 38 grep hits across importer files (agent/{analyzer,
ci_autofix,reviewer,webapp}.py, agent/dashboard/*, agent/middleware/
settle_review_check.py, agent/tools/*, agent/utils/github_feedback.py,
agent/webhooks/github.py, evals/reviewer/*, and the reviewer test
suite) to point at agent.review.*; 4 of the 38 hits were name
collisions (list_reviewer_findings, reviewer_outcomes,
_reviewer_thread_id, reviewer_thread_id — not the moved modules) and
were left untouched. tests/test_github_checks.py's module-alias
import (`from agent import reviewer_publish`) follows upstream's own
`from agent.review import publish as reviewer_publish` pattern so
downstream `reviewer_publish.*` call sites needed no changes.
agent/reviewer.py and agent/webapp.py stay in place per the hard
rule (fork content, import-only rewire) and are not part of this
package.
Gates: ruff check + ruff format --check, pytest --co -q (1637
collected), full unit suite (1637 passed), and the reviewer/findings
suite in isolation (pytest -k "review or finding", 421 passed).
2026-07-17 13:52:03 -04:00
|
|
|
from ..review.diff import compute_diff_line_set, fetch_pr_diff, is_range_in_diff
|
|
|
|
|
from ..review.findings import (
|
2026-07-17 12:10:54 -04:00
|
|
|
REVIEW_FINDING_CAP,
|
|
|
|
|
REVIEWER_EVAL_PUBLICATION_KEY,
|
2026-06-18 11:15:42 -07:00
|
|
|
SEVERITY_ORDER,
|
2026-05-27 10:52:30 -07:00
|
|
|
Finding,
|
2026-06-10 10:44:13 -07:00
|
|
|
ReviewerThreadMissingError,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
Severity,
|
2026-07-31 13:02:07 -04:00
|
|
|
_coerce_findings_list,
|
2026-05-27 17:26:08 -07:00
|
|
|
_coerce_surface,
|
2026-07-31 13:02:07 -04:00
|
|
|
_finding_mutation_lock,
|
|
|
|
|
_get_thread_metadata_strict,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
filter_findings_for_publish,
|
|
|
|
|
get_thread_id_from_runtime,
|
2026-06-10 10:17:31 -07:00
|
|
|
get_thread_last_reviewed_sha,
|
2026-05-07 16:11:36 -07:00
|
|
|
get_thread_metadata,
|
2026-07-20 15:28:00 -04:00
|
|
|
get_thread_pr_meta,
|
2026-05-07 16:11:36 -07:00
|
|
|
get_thread_slack_ref,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
replace_findings,
|
fix: reviewer publishes against stale head_sha on mid-run re-review (#1393)
* fix: resolve reviewer head_sha from thread metadata, not frozen run config
A push that lands while a reviewer run is in flight is delivered as a
queued message into that run. The run's configurable is frozen at
creation, so its head_sha still names the commit the run was created for
— not the commit just pushed. publish_review then anchored the GitHub
review to the stale commit and regressed last_reviewed_sha to it, and
add_finding/update_finding stamped findings with the stale SHA.
Persist the current head in thread metadata at every reviewer dispatch
(both the ready-for-review and push paths, before they branch to create
a run or queue a message), and add resolve_review_head_sha() which
prefers the metadata head over the run config. Wire it into
publish_review (review commit_id + last_reviewed_sha), add_finding
(first_seen_sha) and update_finding (last_confirmed_sha). Falls back to
the run config when metadata carries no head (first review, eval, tests).
* fix: persist head_sha in manual review dispatch (trigger_pr_review_from_ref)
resolve_review_head_sha prefers metadata[head_sha] over the run config,
and the push/ready dispatchers write it — but trigger_pr_review_from_ref
(Slack/GitHub @open-swe review, request_pr_review tool) created a run
with a freshly-fetched config head while leaving metadata's head stale
from a prior dispatch. A manual re-review at a newer commit would then
resolve to the old head and publish/advance findings against it.
Persist head_sha in that dispatch's metadata write too, so every
run-creating reviewer dispatch keeps metadata in sync with the head its
run targets. Caught by the Open SWE reviewer on this PR.
2026-06-03 11:38:56 -07:00
|
|
|
resolve_review_head_sha,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
set_reviewer_thread_metadata,
|
2026-06-10 10:44:13 -07:00
|
|
|
thread_missing_tool_result,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
)
|
refactor: consolidate reviewer modules into agent/review/
Part of the domain-reorg adoption (build plan step C2): fork content,
upstream layout. Nine 1:1 module moves (reviewer_diff/eval_store/
findings/groups/publish/reconcile/trace_context + review_style_
collector/guidance) into agent/review/, with internal relative
imports re-wired to the new package depth. agent/review/__init__.py
mirrors upstream's thin re-export shim (one of the 21 verified "A"
structural adds).
Rewrote the 38 grep hits across importer files (agent/{analyzer,
ci_autofix,reviewer,webapp}.py, agent/dashboard/*, agent/middleware/
settle_review_check.py, agent/tools/*, agent/utils/github_feedback.py,
agent/webhooks/github.py, evals/reviewer/*, and the reviewer test
suite) to point at agent.review.*; 4 of the 38 hits were name
collisions (list_reviewer_findings, reviewer_outcomes,
_reviewer_thread_id, reviewer_thread_id — not the moved modules) and
were left untouched. tests/test_github_checks.py's module-alias
import (`from agent import reviewer_publish`) follows upstream's own
`from agent.review import publish as reviewer_publish` pattern so
downstream `reviewer_publish.*` call sites needed no changes.
agent/reviewer.py and agent/webapp.py stay in place per the hard
rule (fork content, import-only rewire) and are not part of this
package.
Gates: ruff check + ruff format --check, pytest --co -q (1637
collected), full unit suite (1637 passed), and the reviewer/findings
suite in isolation (pytest -k "review or finding", 421 passed).
2026-07-17 13:52:03 -04:00
|
|
|
from ..review.findings import (
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
list_findings as list_findings_async,
|
|
|
|
|
)
|
refactor: consolidate reviewer modules into agent/review/
Part of the domain-reorg adoption (build plan step C2): fork content,
upstream layout. Nine 1:1 module moves (reviewer_diff/eval_store/
findings/groups/publish/reconcile/trace_context + review_style_
collector/guidance) into agent/review/, with internal relative
imports re-wired to the new package depth. agent/review/__init__.py
mirrors upstream's thin re-export shim (one of the 21 verified "A"
structural adds).
Rewrote the 38 grep hits across importer files (agent/{analyzer,
ci_autofix,reviewer,webapp}.py, agent/dashboard/*, agent/middleware/
settle_review_check.py, agent/tools/*, agent/utils/github_feedback.py,
agent/webhooks/github.py, evals/reviewer/*, and the reviewer test
suite) to point at agent.review.*; 4 of the 38 hits were name
collisions (list_reviewer_findings, reviewer_outcomes,
_reviewer_thread_id, reviewer_thread_id — not the moved modules) and
were left untouched. tests/test_github_checks.py's module-alias
import (`from agent import reviewer_publish`) follows upstream's own
`from agent.review import publish as reviewer_publish` pattern so
downstream `reviewer_publish.*` call sites needed no changes.
agent/reviewer.py and agent/webapp.py stay in place per the hard
rule (fork content, import-only rewire) and are not part of this
package.
Gates: ruff check + ruff format --check, pytest --co -q (1637
collected), full unit suite (1637 passed), and the reviewer/findings
suite in isolation (pytest -k "review or finding", 421 passed).
2026-07-17 13:52:03 -04:00
|
|
|
from ..review.publish import (
|
2026-06-07 01:17:09 -04:00
|
|
|
clear_review_started_comment,
|
2026-07-20 15:28:00 -04:00
|
|
|
dismiss_pull_request_review,
|
2026-05-26 16:24:34 -07:00
|
|
|
fetch_pr_review_threads,
|
2026-07-31 13:02:07 -04:00
|
|
|
fetch_pull_request_head_sha,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
fetch_review_comments,
|
|
|
|
|
fetch_review_thread_id_for_comment,
|
2026-06-03 10:30:13 -07:00
|
|
|
open_swe_review_exists,
|
2026-05-27 10:52:30 -07:00
|
|
|
parse_review_comment_marker,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
post_pull_request_review,
|
|
|
|
|
render_inline_comment_payload,
|
2026-05-28 12:44:00 -07:00
|
|
|
render_resolution_comment,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
render_review_body,
|
2026-05-28 12:44:00 -07:00
|
|
|
reply_to_review_comment,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
resolve_review_thread,
|
2026-06-10 11:53:48 -07:00
|
|
|
settle_review_check_run,
|
2026-07-31 13:02:07 -04:00
|
|
|
update_pull_request_review_body,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
)
|
refactor: consolidate reviewer modules into agent/review/
Part of the domain-reorg adoption (build plan step C2): fork content,
upstream layout. Nine 1:1 module moves (reviewer_diff/eval_store/
findings/groups/publish/reconcile/trace_context + review_style_
collector/guidance) into agent/review/, with internal relative
imports re-wired to the new package depth. agent/review/__init__.py
mirrors upstream's thin re-export shim (one of the 21 verified "A"
structural adds).
Rewrote the 38 grep hits across importer files (agent/{analyzer,
ci_autofix,reviewer,webapp}.py, agent/dashboard/*, agent/middleware/
settle_review_check.py, agent/tools/*, agent/utils/github_feedback.py,
agent/webhooks/github.py, evals/reviewer/*, and the reviewer test
suite) to point at agent.review.*; 4 of the 38 hits were name
collisions (list_reviewer_findings, reviewer_outcomes,
_reviewer_thread_id, reviewer_thread_id — not the moved modules) and
were left untouched. tests/test_github_checks.py's module-alias
import (`from agent import reviewer_publish`) follows upstream's own
`from agent.review import publish as reviewer_publish` pattern so
downstream `reviewer_publish.*` call sites needed no changes.
agent/reviewer.py and agent/webapp.py stay in place per the hard
rule (fork content, import-only rewire) and are not part of this
package.
Gates: ruff check + ruff format --check, pytest --co -q (1637
collected), full unit suite (1637 passed), and the reviewer/findings
suite in isolation (pytest -k "review or finding", 421 passed).
2026-07-17 13:52:03 -04:00
|
|
|
from ..review.reconcile import reconcile_findings_with_review_threads
|
2026-06-12 14:15:58 -07:00
|
|
|
from ..utils.dashboard_links import dashboard_review_url
|
2026-06-10 11:53:48 -07:00
|
|
|
from ..utils.github_checks import review_check_conclusion
|
2026-07-20 15:28:00 -04:00
|
|
|
from ..utils.github_org_membership import INTERNAL_BOT_LOGINS
|
2026-05-08 22:57:01 +00:00
|
|
|
from ..utils.github_token import (
|
|
|
|
|
GitHubAuthError,
|
|
|
|
|
get_github_token,
|
|
|
|
|
invalidate_cached_github_token,
|
|
|
|
|
)
|
2026-05-28 13:48:14 -07:00
|
|
|
from ..utils.langsmith import get_langsmith_trace_url
|
2026-05-07 16:11:36 -07:00
|
|
|
from ..utils.slack import post_slack_thread_reply
|
2026-06-13 09:37:15 -07:00
|
|
|
from ..utils.tracing import REVIEW_TRACING_PROJECT
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
|
2026-07-20 15:28:00 -04:00
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
|
|
|
|
_VERDICT_EVENTS = {"approve": "APPROVE", "request_changes": "REQUEST_CHANGES"}
|
|
|
|
|
# Map the GitHub review event we POST to the review "state" GitHub reports back
|
|
|
|
|
# on the created review object, so we can confirm the verdict actually landed.
|
|
|
|
|
_EVENT_TO_STATE = {"APPROVE": "APPROVED", "REQUEST_CHANGES": "CHANGES_REQUESTED"}
|
|
|
|
|
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
|
chore: sync upstream/main, defer #1621 modular webhooks (#81)
* chore: bake sfw binary into sandbox image (#1611)
sfw only ships a launcher that fetches its real binary at first run and does a
daily update check against api.github.com/repos/SocketDev/sfw-free. Both fail in
the sandbox (restricted egress; the proxy injects the GitHub App installation
token, which lacks access to that repo), so `sfw yarn install` errors with
"could not fetch its binary". Pin sfw 2.0.6, warm + verify the binary cache at
build, and set SFW_SKIP_UPDATE_CHECK=1 so runs use the baked binary offline.
* feat: editable plan mode + fix review-plan banner overlap (#1610)
* feat: editable plan mode + fix review-plan banner overlap
Lets the thread owner edit the plan markdown by hand from the plan-review
page (Edit -> textarea -> Save) via a new PUT /dashboard/api/plan/{id}
endpoint that re-publishes the plan and mirrors it into the sandbox
plan.md, so approve hands the edited plan to the agent as the source of
truth. Also fixes the collapsed git-panel's floating expand button
covering the "Review plan ->" banner by reserving space for it.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: abort plan approval when the published plan read fails
get_plan_content() swallowed store errors and returned None, so a
transient failure during approve would still mark the plan approved and
dispatch the generic fallback text — silently dropping an owner's edited
plan. Read the plan strictly (raise_on_error=True) so approval aborts
instead, matching the comment read.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: show message timestamps (#1609)
* feat: show message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: suppress fallback message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: stable message + tool-call hover timestamps
Stamp a stable client-side arrival time per message and tool call (keyed
by id, persisted to localStorage). Messages render the timestamp inline;
tool rows reveal a dim timestamp chip on hover. Real backend created_at
still takes precedence when present.
* fix: hide client-stamped message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add PR trace resolution (#1612)
* feat: add PR trace resolution
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: inject reviewer trace context as JSON
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: address review on PR trace resolution
Use the documented LangSmith metadata filter syntax
(and(eq(metadata_key,...), eq(metadata_value,...))) instead of
has(metadata, '{...}'), which does not match runs — _list_thread_runs
was silently returning nothing. Bound full-text searches to a 90-day
window so they don't hit LangSmith's large-window rate limit.
Also folds in the best-effort branch->head-sha resolver (dropping the
weighted scoring/threshold + repo/file evidence + GitHub hydration),
sandbox JSON injection, and the admin "Resolve trace" dry-run endpoint.
The IDOR findings are moot: resolve_pr_to_threads/summarize_agent_session
were removed; resolution now runs deterministically from the trusted run
config with no model-controlled pr_url or thread_id.
* fix: scope branch trace search to the repo
Branch names like fix-tests aren't unique across repos (or older PRs) in
a shared tracing project, so an unscoped branch hit could resolve to an
unrelated thread and write its runs into the reviewer sandbox. Require
the repo slug to co-occur with the branch in matched runs; the full head
SHA stays unscoped since it is globally unique. Addresses open-swe review
on PR #1612.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: include plan links in PR descriptions (#1613)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: gate workflow pushes with approval (#1614)
* feat: gate workflow pushes with approval
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve proxy refresh test compatibility
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: bind workflow approvals to pushed ref
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: recover thread work as patch (#1615)
* feat: recover thread work as patch
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: search sandbox cwd for recovery patches
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: omit plan link in PR description when no plan exists (#1618)
Plan links in PR descriptions were always built from the thread id, so
runs that never produced a plan linked to an empty plan-review page.
Now the plan content store is consulted first; the link is only added
when a plan with non-empty markdown actually exists. A transient store
failure degrades gracefully (no link) rather than blocking PR creation.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add filter & grouping menu to agents threads sidebar (#1617)
Add a Cursor-style control to the agents sidebar that groups (None/Date/
Status/Project), filters (ownership, status, source, pull request, model,
repo, include-resolved), and compacts the threads list. All client-side over
already-fetched sidebar threads; preferences persist in localStorage.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: update langsmith sdk to 0.9.3 (#1616)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: clickable shared PR header in git panel and reviews (#1620)
* feat: clickable shared PR header in git panel and reviews
Replace the standalone "View PR" button in the agent git panel with a
clickable PR title, matching the reviews view. Extract a shared PrHeader
component reused by both the git panel and the review main body.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* refactor: drop PrHeader wrapper, use shared component directly
The review-side PrHeader was just a thin adapter mapping detail -> the
shared component's props. Inline it at the call site and use the shared
PrHeader directly so there's a single component.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* refactor: durable interrupt dispatch + completion webhook (#1621)
* wip(rebuild): core reliability spine
- remove PR-babysitting (ci_autofix + ci_monitor graph + webhook wiring)
- dispatch core: agent/dispatch.py with multitask_strategy=interrupt +
durability=sync + completion webhook; reroute all webhook + plan triggers;
drop the racy in-process lock + is_thread_active busy-check
- completion webhook: agent/completion.py + /webhooks/run-complete loopback
route for failure/timeout replies (idempotent)
Co-authored-by: open-swe[bot]
* feat(rebuild): async tools, reconcile, shared http timeouts, assembly tuning
Parallel batch on top of the reliability spine:
- async-ify all 24 tools (drop asyncio.run; requests->httpx); re-implement the
http_request/fetch_url SSRF + DNS-rebinding defense httpx-natively and harden
the IP check to 'not is_global' (+ IPv4-mapped unwrap)
- reconcile.py: stale pending-run sweep (threads.search -> per-thread runs.list
-> cancel_many), wired into the scheduler graph via task='reconcile'
- shared DEFAULT_HTTP_TIMEOUT (agent/utils/http.py) on every bare
httpx.AsyncClient() across utils/dashboard/webapp/middleware
- run budget: MODEL_CALL_RECURSION_LIMIT 5000->250
- fix stale OpenAI->Anthropic fallback id (claude-opus-4-5 -> 4-8)
- drop redundant custom repair middleware (deepagents auto-adds PatchToolCalls)
- confirm tool-result eviction + summarization auto-wired via backend
- slim system prompt ~8% (full harness-profile rewrite deferred)
Co-authored-by: open-swe[bot]
* feat(rebuild): harness-profile prompt + split webhooks out of webapp
- prompt.py: own the system prompt via a registered harness profile
(OPEN_SWE_SHARED_BASE, kept neutral so the read-only reviewer/analyzer that
share it stay safe), registered across all 4 providers; per-thread values
stay in construct_system_prompt. Assembled main-agent prompt ~6.8k -> ~3.1k
tokens (~55% smaller); de-duped PR/commit/suite/force-push guidance; dropped
ALL-CAPS markers.
- webapp.py 3325 -> 1890 LOC: moved 14 per-source handlers into
agent/webhooks/{linear,slack,github}.py; webapp re-exports them for the
routes + tests; moved handlers reach shared helpers via the webapp namespace
to preserve the test suite's monkeypatch targets.
Full suite: 1168 passing, lint clean.
Co-authored-by: open-swe[bot]
* Restore MODEL_CALL_RECURSION_LIMIT to 5000 for long-running tasks
Reverts the 250 cap from the run-budget change — long-running tasks legitimately
need many model calls. The notify_step_limit_reached safety net still fires if a
run does hit the cap, so runs end with a signal either way.
Co-authored-by: open-swe[bot]
* fix: address PR review (auth, SSRF, interrupted status, redirect headers)
- completion.py: drop `interrupted` from failure statuses — with
multitask_strategy=interrupt a follow-up ends the prior run as interrupted,
which is healthy, not a failure to report. [open-swe]
- /webhooks/run-complete: shared-secret auth — dispatch appends ?token= when
RUN_COMPLETE_WEBHOOK_SECRET is set; route verifies via hmac.compare_digest.
[corridor-security]
- SSRF: extract the URL validator to agent/utils/url_safety.py and apply it
before server-side image fetches in multimodal.fetch_image_block.
[corridor-security]
- http_request: preserve caller headers/extensions across redirect hops instead
of dropping them on the first hop. [open-swe]
Co-authored-by: open-swe[bot]
* chore: remove REBUILD_PLAN.md (planning doc, not needed in the repo)
Co-authored-by: open-swe[bot]
* fix: fail closed on run-complete webhook auth when secret unset
Corridor follow-up: verify_run_complete_token returns False (not True) when
RUN_COMPLETE_WEBHOOK_SECRET is unset, so the public route is never
unauthenticated. Logs a startup warning when the secret is absent, and dispatch
skips registering the webhook when there's no secret (no rejected callbacks).
Co-authored-by: open-swe[bot]
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: restore forced tool call to prevent premature run stops (#1622)
Restore the ensure_no_empty_msg middleware and the always-call-a-tool system-prompt instruction that #1535 removed. When the model emits a message with no tool call (and hasn't already messaged the user or confirmed completion), the middleware re-injects a no_op / confirming_completion tool call so the run continues instead of ending mid-task.
Shipping to test whether it fixes runs that stop halfway through.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore(deps): bump langgraph-checkpoint from 4.1.0 to 4.1.1 (#1619)
Bumps [langgraph-checkpoint](https://github.com/langchain-ai/langgraph) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/langchain-ai/langgraph/releases)
- [Commits](https://github.com/langchain-ai/langgraph/compare/checkpoint==4.1.0...checkpoint==4.1.1)
---
updated-dependencies:
- dependency-name: langgraph-checkpoint
dependency-version: 4.1.1
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix: post reviewer resolution notes verbatim (#1624)
* fix: post reviewer resolution notes verbatim
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: stabilize dashboard follow-up e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve dashboard attribution in e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: make e2e attribution marker durable
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: only echo found e2e attribution
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: check live dashboard attribution in e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* hotfix: stop prompting agent/reviewer to wrap installs in sfw (#1625)
Installs hung when prefixed with sfw inside the sandbox (trace 019f0608
stalled on a pending `sfw npm install` execute, never returned). Strip the
Socket Firewall guidance from the agent and reviewer prompts so installs run
through the project's package manager directly. sfw stays in the Docker image;
nothing invokes it now.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: make plan view mobile friendly (#1636)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: fall back to vision model for image threads (#1626)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: surface Slack thread errors (#1627)
* fix: surface Slack thread errors
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: don't set failure_reply_posted on Slack preprocessing errors
The preprocessing error handler was setting failure_reply_posted=True,
the same idempotency flag handle_run_completion checks to suppress
duplicate run-failure replies. Since preprocessing failures happen
before any run exists but the flag persists on the thread, a subsequent
run failure on the same thread would be silently ignored.
The preprocessing handler already posts its own Slack reply, so the
run-completion idempotency flag should not be set here.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: avoid recapping Slack replies (#1629)
* chore: avoid recapping Slack replies
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: simplify Slack reply prompt wording
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: update Slack trace reply on web handoff (#1630)
* fix: update Slack trace reply on web handoff
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: trigger web handoff on dashboard starts
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: format web handoff as contextual fragment
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve trace_message_ts when overwriting Slack run mapping
When store_slack_run_mapping is called without trace_message_ts (e.g. on
follow-up Slack mentions), it was unconditionally overwriting the
thread-level mapping and clobbering the timestamp captured from the
initial trace reply. After that, _notify_slack_web_handoff could not find
the original message, so a subsequent move to Web silently skipped the
Slack trace update.
Now, when trace_message_ts is not passed, the existing thread mapping is
read first and its trace_message_ts is preserved.
* style: ruff format
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* fix: pre-bundle shiki/@pierre deps to stop dev dynamic-import failures (#1643)
* fix(ui): pre-bundle shiki/@pierre deps to stop dev dynamic-import failures
shiki lazy-imports a grammar per language and these libs only live inside
lazy route components, so Vite's startup scanner never sees them. They get
discovered on first thread navigation, triggering a dep re-optimize +
force-reload that aborts the in-flight route-chunk import, surfacing as
"Failed to fetch dynamically imported module: .../$threadId.tsx".
Pre-bundle them (and the github themes + common code-block languages) via
optimizeDeps.include so the optimize happens once at startup. Dev-only;
production bundles are unaffected.
* fix: pre-bundle canonical shiki docker/make langs instead of aliases
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: show queued dashboard follow-ups (#1631)
* feat: show queued dashboard follow-ups
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: de-dupe queued follow-ups while streaming
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat: notify Slack on plan approval (#1632)
* feat: notify Slack on plan approval
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: post Slack approval notice after successful dispatch
Move the _maybe_post_plan_approved_to_slack call until after
_dispatch_followup succeeds so the Slack thread is not told
implementation is beginning before the LangGraph run is created.
Addresses PR review comment.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat: include Slack channel context in prompts (#1633)
Add cached Slack channel metadata enrichment for Slack-triggered runs so prompts can include channel names and descriptions without duplicate conversations.info calls.\n\nCo-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
* chore: keep plan guidance high-level (#1634)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: publish plans from sandbox files (#1635)
* feat: publish plans from sandbox files
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: avoid fixed plan filenames
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: virtualize local sandbox file paths
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve plan_file_path across set_plan_status
set_plan_status was rewriting the content record with only markdown
and status, dropping plan_file_path. After a reject, the owner's
dashboard edit would mirror to a different file than the agent's
original, and the next save_plan could republish the stale file.
Preserve plan_file_path when updating status.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: return to thread after plan approval (#1637)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add Slack breakout thread tool (#1638)
* feat: add Slack breakout thread tool
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: make fake LLM scripts declarative
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: exclude slack_start_new_thread from plan mode
The breakout tool can dispatch a fresh agent run that starts outside the
current plan-mode state, bypassing the approval flow. Add it to
PLAN_MODE_EXCLUDED_TOOLS so it's hidden alongside the other mutating
tools while planning.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: require bun for ui agent work (#1639)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: request actions read for sandbox logs (#1642)
* fix: request actions read for sandbox logs
Request optional Actions read permission for sandbox proxy tokens, with fallback for installations that have not approved it yet. Update setup docs and prompt guidance for safe GitHub Actions log usage.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: restore actions:read scope after workflow push
After an approved workflow push, the guard was restoring the proxy with
BASE_RUNTIME_PROXY_TOKEN_PERMISSIONS, which excludes the actions: read
scope this PR adds. Restore with RUNTIME_PROXY_TOKEN_PERMISSIONS (which
includes actions: read) and fall back to BASE if the install hasn't
granted Actions read — mirroring the pattern in _create_sandbox_with_proxy.
Addresses review comment on PR #1642.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: widen split review diffs (#1647)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: install missing deps before verification (#1646)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: switch ui to pnpm (#1645)
* chore: require pnpm for ui agent work
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: switch ui to pnpm
Replace Bun and Yarn lockfiles with pnpm lockfile and update UI/Vercel commands to use pnpm.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* ci: use corepack for ui pnpm e2e build
Run pnpm through Corepack in the E2E global setup so CI can use the pinned package manager without a separate pnpm install step.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add Sonnet 5 to model picker (#1651)
* chore: update Sonnet examples to Sonnet 5
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: add Sonnet 5 to model picker
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* Remove dead breakout-thread e2e scenario after dropping the tool
The merge resolution deferred upstream's Slack breakout-thread tool
(slack_start_new_thread, #1638) since it depends on the #1621 dispatch
module, but the e2e harness still scripted it. Removing the tool name
from fake_llm.py's _tool_step call left a malformed scenario, crashing
the langgraph-dev web server at import (TypeError: _tool_step() missing
'call_id') and failing Playwright E2E.
Drop the "breakout" script scenario, its _is_breakout_request helper +
ScriptRule, and the corresponding full_flow.spec.ts test.
* Revert upstream pnpm switch; keep bun for the UI build
The merge auto-adopted upstream's pnpm switch (#1645) in tests/e2e/
global-setup.ts and ui/package.json, but our fork builds the UI with
bun (vercel.json + the E2E workflow's setup-bun). That left the
Playwright globalSetup running `corepack pnpm install --frozen-lockfile`
with no pnpm-lock.yaml, failing E2E at UI build time.
Revert global-setup.ts and ui/package.json to the dev (bun) baseline,
drop the merge-added ui/pnpm-lock.yaml, and remove the re-added
ui/AGENTS.md (our fork had deleted it).
* Align plan-review e2e + UI with the HEAD (pre-#1635) backend
The merge left a split plan vertical: the backend save_plan/plan_api are
HEAD (we deferred the editable-plan/sandbox-publish features #1610/#1635/
#1637 per #80), but the plan UI and e2e harness were upstream's. The
fake_llm scenario called save_plan(plan_file_path=...) — upstream's
file-based #1635 contract — while HEAD save_plan takes plan_markdown,
so the plan never saved and PlanReview never rendered (E2E failure on
the plan-review locator).
Pass plan_markdown to save_plan, and revert PlanReview.tsx / plan.ts /
$threadId_.plan.tsx / plan_review.spec.ts to the dev baseline so the
whole plan flow (save -> render -> approve -> implement) is consistent
with the HEAD backend.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Ramon Nogueira <ramon.nogueira@langchain.dev>
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: Caroline di Vittorio <43390382+carolinedivittorio@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
Co-authored-by: Mukil Loganathan <mukil@langchain.dev>
2026-06-30 16:45:19 -04:00
|
|
|
async def publish_review(
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
severity_threshold: str = "medium",
|
2026-07-20 15:28:00 -04:00
|
|
|
verdict: str | None = None,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
) -> dict[str, Any]:
|
|
|
|
|
"""Post all current findings to the PR as a GitHub Review.
|
|
|
|
|
|
|
|
|
|
Call this once at the end of a review run, after you have finished adding
|
|
|
|
|
findings (and, on a re-review, after marking resolved findings via
|
2026-05-26 16:24:34 -07:00
|
|
|
``update_finding``). The tool posts one GitHub PR Review for eligible
|
|
|
|
|
inline findings, records the GitHub comment/thread IDs for future
|
|
|
|
|
re-reviews, resolves GitHub threads for findings now marked resolved, and
|
|
|
|
|
advances the reviewer thread's ``last_reviewed_sha``.
|
|
|
|
|
|
|
|
|
|
On a re-review with no new findings to surface, it skips posting a new
|
|
|
|
|
GitHub Review but still resolves fixed threads and updates reviewer state.
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
|
|
|
|
|
Args:
|
2026-06-18 11:15:42 -07:00
|
|
|
severity_threshold: Lowest severity to surface as inline GitHub comments
|
|
|
|
|
(default ``medium``). Lower-severity findings stay in state and are
|
|
|
|
|
mentioned in the review summary with a link to the web app, but are
|
|
|
|
|
not posted as inline PR comments.
|
2026-07-20 15:28:00 -04:00
|
|
|
verdict: Optional review verdict — ``"approve"`` or
|
2026-07-31 13:02:07 -04:00
|
|
|
``"request_changes"``. Explicitly requested verdicts are honored
|
|
|
|
|
subject to the safety checks below. Automatically authorized
|
|
|
|
|
verdicts must agree with authoritative finding state: approve
|
|
|
|
|
requires no open findings and request_changes requires at least
|
|
|
|
|
one. A downgraded verdict is posted as a comment review and carries
|
|
|
|
|
``verdict_ignored: true``.
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
Returns:
|
|
|
|
|
Dictionary with ``success``, ``review_id``, ``surfaced_count``,
|
2026-06-10 10:44:13 -07:00
|
|
|
``hidden_count``, ``resolved_thread_count``, and sometimes
|
2026-06-12 10:19:30 -07:00
|
|
|
``unresolvable_findings``, plus the flags below.
|
2026-06-10 10:44:13 -07:00
|
|
|
|
|
|
|
|
``success: true`` alone does NOT mean a GitHub Review was posted —
|
|
|
|
|
check the flags:
|
|
|
|
|
|
|
|
|
|
- ``skipped_empty_re_review: true`` (with ``review_id: null``): an
|
|
|
|
|
empty re-review was deliberately skipped. No GitHub Review was
|
|
|
|
|
created; the call was a valid no-op. Do not describe the review as
|
|
|
|
|
published/posted/submitted.
|
|
|
|
|
- ``dry_run: true`` (with ``review_id: null``): eval/benchmark mode —
|
|
|
|
|
the publish was simulated and nothing was posted to GitHub. Do not
|
|
|
|
|
claim publication.
|
|
|
|
|
|
|
|
|
|
Only a numeric ``review_id`` (with neither flag set) confirms a real
|
|
|
|
|
GitHub Review was created.
|
2026-07-20 15:28:00 -04:00
|
|
|
|
|
|
|
|
When ``verdict`` was passed, the result also carries
|
|
|
|
|
``verdict_submitted`` (GitHub confirmed the requested APPROVE/
|
|
|
|
|
REQUEST_CHANGES state) or ``verdict_ignored`` +
|
|
|
|
|
``verdict_ignored_reason`` (``"verdict_not_requested"``,
|
2026-07-31 13:02:07 -04:00
|
|
|
``"approve_with_open_findings"``,
|
|
|
|
|
``"request_changes_without_open_findings"``, ``"self_review"``,
|
|
|
|
|
``"head_moved"`` — the reviewed commit is no longer the PR head —
|
2026-07-31 13:12:25 -04:00
|
|
|
``"head_check_failed"``, ``"author_unknown"``, or
|
|
|
|
|
``"github_state_mismatch"``).
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
"""
|
feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312)
* feat: tune reviewer for precision — web/wiki tools + recalibrated prompt
Reviewer agent now has web_search, fetch_url, and http_request alongside the
finding tools, so it can verify library semantics and consult the DeepWiki
auto-generated wiki for public repos (https://deepwiki.com/<owner>/<repo>)
before flagging cross-file or architectural concerns.
Prompt rewritten to push precision over recall:
- explicit severity ladder pushing reviews toward bimodal high/low instead of
defaulting to medium
- ≤200-char description target (gold set averages ~186 chars; we were at ~436)
- mandatory docs / wiki / code lookup before flagging concurrency, security,
or perf — the three categories that dominated false positives
- "do not flag" list covering compiler/linter-catchable nits, speculative
claims without a concrete attacker/interleaving/scale, style preferences
the codebase doesn't share, and test-quality nits on non-test diffs
- smart file-selection guidance for large PRs (deprioritize generated /
vendored / pure-rename hunks)
Eval config switched to openai:gpt-5.5 + high reasoning effort for the next
benchmark run.
* trim prompt
* subagent prompting
* confidence ratings
* added medium
* enforce confidence threshold
* .
* reviewer: precision-tuned prompt + drop confidence gate
Rewrites the reviewer system prompt around a defensibility bar (anchor +
failure mode + maintainer wouldn't say "not a bug"), an explicit do-not-file
list (style nits, speculation, scope-policing, same-bug fan-out), and a
checklist of 10 bug archetypes drawn from a per-PR audit of the eval golden
set. The audit showed 145 FPs in the last eval split ~28% speculative, ~26%
style-nit, ~31% real-but-unscored (mostly same-archetype fan-out); the new
prompt targets each class directly.
Confidence is still recorded on every finding for post-hoc calibration but
no longer gates publication — the audit showed the gate was a no-op (agent
self-rated 65% of findings "high" regardless), and the prompt's defensibility
bar is the actual discipline. Drops CONFIDENCE_ORDER, CONFIDENCE_THRESHOLD,
the confidence_threshold kwarg on filter_findings_for_publish, the
confidence_filtered score_mode, and the min_confidence kwarg on the eval
target's _extract_comments — all dead once the gate is gone.
Also removes the "informational" severity tier from the Severity enum,
SEVERITY_ORDER, and all validators / tests / docstrings. It was reserved for
FYI observations the dataset never rewards.
* benchmax
* adding google provider
* slight steering
* tuning
* more tuning
* fix
* cleanup
* reducing overfitting
* Add per-repo review style profiles and inject them into the reviewer.
Dashboard users can analyze historical PR review feedback per repository,
edit the resulting style guide, and have it loaded from LangGraph Store at
reviewer runtime (including Martian eval runs) keyed by owner/name.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix review style job errors leaking exception details to clients.
Return generic dashboard messages while logging full stack traces server-side.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 11:35:00 -07:00
|
|
|
if severity_threshold not in {"low", "medium", "high", "critical"}:
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
return {"success": False, "error": f"Invalid severity_threshold: {severity_threshold}"}
|
2026-07-20 15:28:00 -04:00
|
|
|
if verdict is not None and verdict not in _VERDICT_EVENTS:
|
|
|
|
|
return {
|
|
|
|
|
"success": False,
|
|
|
|
|
"error": (
|
|
|
|
|
f"Invalid verdict: {verdict!r}. Use 'approve', 'request_changes', "
|
|
|
|
|
"or omit the parameter."
|
|
|
|
|
),
|
|
|
|
|
}
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
|
|
|
|
|
config = get_config()
|
2026-05-28 13:48:14 -07:00
|
|
|
raw_configurable = config.get("configurable", {}) if isinstance(config, dict) else {}
|
|
|
|
|
configurable = raw_configurable if isinstance(raw_configurable, dict) else {}
|
|
|
|
|
repo_config = configurable.get("repo")
|
|
|
|
|
pr_number = configurable.get("pr_number")
|
|
|
|
|
head_sha = configurable.get("head_sha")
|
|
|
|
|
is_re_review = bool(configurable.get("re_review"))
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
|
|
|
|
|
if (
|
|
|
|
|
not isinstance(repo_config, dict)
|
|
|
|
|
or not repo_config.get("owner")
|
|
|
|
|
or not repo_config.get("name")
|
|
|
|
|
):
|
|
|
|
|
return {"success": False, "error": "Missing repo info in run config"}
|
|
|
|
|
if not isinstance(pr_number, int):
|
|
|
|
|
return {"success": False, "error": "Missing pr_number in run config"}
|
|
|
|
|
if not isinstance(head_sha, str) or not head_sha:
|
|
|
|
|
return {"success": False, "error": "Missing head_sha in run config"}
|
|
|
|
|
|
2026-05-18 15:47:13 -07:00
|
|
|
if _is_reviewer_eval_mode(configurable):
|
2026-07-17 12:10:54 -04:00
|
|
|
eval_threshold = configurable.get("reviewer_eval_severity_threshold")
|
|
|
|
|
if isinstance(eval_threshold, str) and eval_threshold in {
|
|
|
|
|
"low",
|
|
|
|
|
"medium",
|
|
|
|
|
"high",
|
|
|
|
|
"critical",
|
|
|
|
|
}:
|
|
|
|
|
severity_threshold = eval_threshold
|
|
|
|
|
eval_cap = configurable.get("reviewer_eval_cap")
|
|
|
|
|
if not isinstance(eval_cap, int) or isinstance(eval_cap, bool) or eval_cap < 0:
|
|
|
|
|
eval_cap = REVIEW_FINDING_CAP
|
2026-06-10 10:44:13 -07:00
|
|
|
try:
|
chore: sync upstream/main, defer #1621 modular webhooks (#81)
* chore: bake sfw binary into sandbox image (#1611)
sfw only ships a launcher that fetches its real binary at first run and does a
daily update check against api.github.com/repos/SocketDev/sfw-free. Both fail in
the sandbox (restricted egress; the proxy injects the GitHub App installation
token, which lacks access to that repo), so `sfw yarn install` errors with
"could not fetch its binary". Pin sfw 2.0.6, warm + verify the binary cache at
build, and set SFW_SKIP_UPDATE_CHECK=1 so runs use the baked binary offline.
* feat: editable plan mode + fix review-plan banner overlap (#1610)
* feat: editable plan mode + fix review-plan banner overlap
Lets the thread owner edit the plan markdown by hand from the plan-review
page (Edit -> textarea -> Save) via a new PUT /dashboard/api/plan/{id}
endpoint that re-publishes the plan and mirrors it into the sandbox
plan.md, so approve hands the edited plan to the agent as the source of
truth. Also fixes the collapsed git-panel's floating expand button
covering the "Review plan ->" banner by reserving space for it.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: abort plan approval when the published plan read fails
get_plan_content() swallowed store errors and returned None, so a
transient failure during approve would still mark the plan approved and
dispatch the generic fallback text — silently dropping an owner's edited
plan. Read the plan strictly (raise_on_error=True) so approval aborts
instead, matching the comment read.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: show message timestamps (#1609)
* feat: show message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: suppress fallback message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: stable message + tool-call hover timestamps
Stamp a stable client-side arrival time per message and tool call (keyed
by id, persisted to localStorage). Messages render the timestamp inline;
tool rows reveal a dim timestamp chip on hover. Real backend created_at
still takes precedence when present.
* fix: hide client-stamped message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add PR trace resolution (#1612)
* feat: add PR trace resolution
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: inject reviewer trace context as JSON
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: address review on PR trace resolution
Use the documented LangSmith metadata filter syntax
(and(eq(metadata_key,...), eq(metadata_value,...))) instead of
has(metadata, '{...}'), which does not match runs — _list_thread_runs
was silently returning nothing. Bound full-text searches to a 90-day
window so they don't hit LangSmith's large-window rate limit.
Also folds in the best-effort branch->head-sha resolver (dropping the
weighted scoring/threshold + repo/file evidence + GitHub hydration),
sandbox JSON injection, and the admin "Resolve trace" dry-run endpoint.
The IDOR findings are moot: resolve_pr_to_threads/summarize_agent_session
were removed; resolution now runs deterministically from the trusted run
config with no model-controlled pr_url or thread_id.
* fix: scope branch trace search to the repo
Branch names like fix-tests aren't unique across repos (or older PRs) in
a shared tracing project, so an unscoped branch hit could resolve to an
unrelated thread and write its runs into the reviewer sandbox. Require
the repo slug to co-occur with the branch in matched runs; the full head
SHA stays unscoped since it is globally unique. Addresses open-swe review
on PR #1612.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: include plan links in PR descriptions (#1613)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: gate workflow pushes with approval (#1614)
* feat: gate workflow pushes with approval
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve proxy refresh test compatibility
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: bind workflow approvals to pushed ref
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: recover thread work as patch (#1615)
* feat: recover thread work as patch
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: search sandbox cwd for recovery patches
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: omit plan link in PR description when no plan exists (#1618)
Plan links in PR descriptions were always built from the thread id, so
runs that never produced a plan linked to an empty plan-review page.
Now the plan content store is consulted first; the link is only added
when a plan with non-empty markdown actually exists. A transient store
failure degrades gracefully (no link) rather than blocking PR creation.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add filter & grouping menu to agents threads sidebar (#1617)
Add a Cursor-style control to the agents sidebar that groups (None/Date/
Status/Project), filters (ownership, status, source, pull request, model,
repo, include-resolved), and compacts the threads list. All client-side over
already-fetched sidebar threads; preferences persist in localStorage.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: update langsmith sdk to 0.9.3 (#1616)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: clickable shared PR header in git panel and reviews (#1620)
* feat: clickable shared PR header in git panel and reviews
Replace the standalone "View PR" button in the agent git panel with a
clickable PR title, matching the reviews view. Extract a shared PrHeader
component reused by both the git panel and the review main body.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* refactor: drop PrHeader wrapper, use shared component directly
The review-side PrHeader was just a thin adapter mapping detail -> the
shared component's props. Inline it at the call site and use the shared
PrHeader directly so there's a single component.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* refactor: durable interrupt dispatch + completion webhook (#1621)
* wip(rebuild): core reliability spine
- remove PR-babysitting (ci_autofix + ci_monitor graph + webhook wiring)
- dispatch core: agent/dispatch.py with multitask_strategy=interrupt +
durability=sync + completion webhook; reroute all webhook + plan triggers;
drop the racy in-process lock + is_thread_active busy-check
- completion webhook: agent/completion.py + /webhooks/run-complete loopback
route for failure/timeout replies (idempotent)
Co-authored-by: open-swe[bot]
* feat(rebuild): async tools, reconcile, shared http timeouts, assembly tuning
Parallel batch on top of the reliability spine:
- async-ify all 24 tools (drop asyncio.run; requests->httpx); re-implement the
http_request/fetch_url SSRF + DNS-rebinding defense httpx-natively and harden
the IP check to 'not is_global' (+ IPv4-mapped unwrap)
- reconcile.py: stale pending-run sweep (threads.search -> per-thread runs.list
-> cancel_many), wired into the scheduler graph via task='reconcile'
- shared DEFAULT_HTTP_TIMEOUT (agent/utils/http.py) on every bare
httpx.AsyncClient() across utils/dashboard/webapp/middleware
- run budget: MODEL_CALL_RECURSION_LIMIT 5000->250
- fix stale OpenAI->Anthropic fallback id (claude-opus-4-5 -> 4-8)
- drop redundant custom repair middleware (deepagents auto-adds PatchToolCalls)
- confirm tool-result eviction + summarization auto-wired via backend
- slim system prompt ~8% (full harness-profile rewrite deferred)
Co-authored-by: open-swe[bot]
* feat(rebuild): harness-profile prompt + split webhooks out of webapp
- prompt.py: own the system prompt via a registered harness profile
(OPEN_SWE_SHARED_BASE, kept neutral so the read-only reviewer/analyzer that
share it stay safe), registered across all 4 providers; per-thread values
stay in construct_system_prompt. Assembled main-agent prompt ~6.8k -> ~3.1k
tokens (~55% smaller); de-duped PR/commit/suite/force-push guidance; dropped
ALL-CAPS markers.
- webapp.py 3325 -> 1890 LOC: moved 14 per-source handlers into
agent/webhooks/{linear,slack,github}.py; webapp re-exports them for the
routes + tests; moved handlers reach shared helpers via the webapp namespace
to preserve the test suite's monkeypatch targets.
Full suite: 1168 passing, lint clean.
Co-authored-by: open-swe[bot]
* Restore MODEL_CALL_RECURSION_LIMIT to 5000 for long-running tasks
Reverts the 250 cap from the run-budget change — long-running tasks legitimately
need many model calls. The notify_step_limit_reached safety net still fires if a
run does hit the cap, so runs end with a signal either way.
Co-authored-by: open-swe[bot]
* fix: address PR review (auth, SSRF, interrupted status, redirect headers)
- completion.py: drop `interrupted` from failure statuses — with
multitask_strategy=interrupt a follow-up ends the prior run as interrupted,
which is healthy, not a failure to report. [open-swe]
- /webhooks/run-complete: shared-secret auth — dispatch appends ?token= when
RUN_COMPLETE_WEBHOOK_SECRET is set; route verifies via hmac.compare_digest.
[corridor-security]
- SSRF: extract the URL validator to agent/utils/url_safety.py and apply it
before server-side image fetches in multimodal.fetch_image_block.
[corridor-security]
- http_request: preserve caller headers/extensions across redirect hops instead
of dropping them on the first hop. [open-swe]
Co-authored-by: open-swe[bot]
* chore: remove REBUILD_PLAN.md (planning doc, not needed in the repo)
Co-authored-by: open-swe[bot]
* fix: fail closed on run-complete webhook auth when secret unset
Corridor follow-up: verify_run_complete_token returns False (not True) when
RUN_COMPLETE_WEBHOOK_SECRET is unset, so the public route is never
unauthenticated. Logs a startup warning when the secret is absent, and dispatch
skips registering the webhook when there's no secret (no rejected callbacks).
Co-authored-by: open-swe[bot]
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: restore forced tool call to prevent premature run stops (#1622)
Restore the ensure_no_empty_msg middleware and the always-call-a-tool system-prompt instruction that #1535 removed. When the model emits a message with no tool call (and hasn't already messaged the user or confirmed completion), the middleware re-injects a no_op / confirming_completion tool call so the run continues instead of ending mid-task.
Shipping to test whether it fixes runs that stop halfway through.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore(deps): bump langgraph-checkpoint from 4.1.0 to 4.1.1 (#1619)
Bumps [langgraph-checkpoint](https://github.com/langchain-ai/langgraph) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/langchain-ai/langgraph/releases)
- [Commits](https://github.com/langchain-ai/langgraph/compare/checkpoint==4.1.0...checkpoint==4.1.1)
---
updated-dependencies:
- dependency-name: langgraph-checkpoint
dependency-version: 4.1.1
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix: post reviewer resolution notes verbatim (#1624)
* fix: post reviewer resolution notes verbatim
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: stabilize dashboard follow-up e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve dashboard attribution in e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: make e2e attribution marker durable
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: only echo found e2e attribution
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: check live dashboard attribution in e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* hotfix: stop prompting agent/reviewer to wrap installs in sfw (#1625)
Installs hung when prefixed with sfw inside the sandbox (trace 019f0608
stalled on a pending `sfw npm install` execute, never returned). Strip the
Socket Firewall guidance from the agent and reviewer prompts so installs run
through the project's package manager directly. sfw stays in the Docker image;
nothing invokes it now.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: make plan view mobile friendly (#1636)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: fall back to vision model for image threads (#1626)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: surface Slack thread errors (#1627)
* fix: surface Slack thread errors
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: don't set failure_reply_posted on Slack preprocessing errors
The preprocessing error handler was setting failure_reply_posted=True,
the same idempotency flag handle_run_completion checks to suppress
duplicate run-failure replies. Since preprocessing failures happen
before any run exists but the flag persists on the thread, a subsequent
run failure on the same thread would be silently ignored.
The preprocessing handler already posts its own Slack reply, so the
run-completion idempotency flag should not be set here.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: avoid recapping Slack replies (#1629)
* chore: avoid recapping Slack replies
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: simplify Slack reply prompt wording
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: update Slack trace reply on web handoff (#1630)
* fix: update Slack trace reply on web handoff
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: trigger web handoff on dashboard starts
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: format web handoff as contextual fragment
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve trace_message_ts when overwriting Slack run mapping
When store_slack_run_mapping is called without trace_message_ts (e.g. on
follow-up Slack mentions), it was unconditionally overwriting the
thread-level mapping and clobbering the timestamp captured from the
initial trace reply. After that, _notify_slack_web_handoff could not find
the original message, so a subsequent move to Web silently skipped the
Slack trace update.
Now, when trace_message_ts is not passed, the existing thread mapping is
read first and its trace_message_ts is preserved.
* style: ruff format
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* fix: pre-bundle shiki/@pierre deps to stop dev dynamic-import failures (#1643)
* fix(ui): pre-bundle shiki/@pierre deps to stop dev dynamic-import failures
shiki lazy-imports a grammar per language and these libs only live inside
lazy route components, so Vite's startup scanner never sees them. They get
discovered on first thread navigation, triggering a dep re-optimize +
force-reload that aborts the in-flight route-chunk import, surfacing as
"Failed to fetch dynamically imported module: .../$threadId.tsx".
Pre-bundle them (and the github themes + common code-block languages) via
optimizeDeps.include so the optimize happens once at startup. Dev-only;
production bundles are unaffected.
* fix: pre-bundle canonical shiki docker/make langs instead of aliases
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: show queued dashboard follow-ups (#1631)
* feat: show queued dashboard follow-ups
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: de-dupe queued follow-ups while streaming
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat: notify Slack on plan approval (#1632)
* feat: notify Slack on plan approval
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: post Slack approval notice after successful dispatch
Move the _maybe_post_plan_approved_to_slack call until after
_dispatch_followup succeeds so the Slack thread is not told
implementation is beginning before the LangGraph run is created.
Addresses PR review comment.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat: include Slack channel context in prompts (#1633)
Add cached Slack channel metadata enrichment for Slack-triggered runs so prompts can include channel names and descriptions without duplicate conversations.info calls.\n\nCo-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
* chore: keep plan guidance high-level (#1634)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: publish plans from sandbox files (#1635)
* feat: publish plans from sandbox files
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: avoid fixed plan filenames
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: virtualize local sandbox file paths
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve plan_file_path across set_plan_status
set_plan_status was rewriting the content record with only markdown
and status, dropping plan_file_path. After a reject, the owner's
dashboard edit would mirror to a different file than the agent's
original, and the next save_plan could republish the stale file.
Preserve plan_file_path when updating status.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: return to thread after plan approval (#1637)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add Slack breakout thread tool (#1638)
* feat: add Slack breakout thread tool
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: make fake LLM scripts declarative
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: exclude slack_start_new_thread from plan mode
The breakout tool can dispatch a fresh agent run that starts outside the
current plan-mode state, bypassing the approval flow. Add it to
PLAN_MODE_EXCLUDED_TOOLS so it's hidden alongside the other mutating
tools while planning.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: require bun for ui agent work (#1639)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: request actions read for sandbox logs (#1642)
* fix: request actions read for sandbox logs
Request optional Actions read permission for sandbox proxy tokens, with fallback for installations that have not approved it yet. Update setup docs and prompt guidance for safe GitHub Actions log usage.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: restore actions:read scope after workflow push
After an approved workflow push, the guard was restoring the proxy with
BASE_RUNTIME_PROXY_TOKEN_PERMISSIONS, which excludes the actions: read
scope this PR adds. Restore with RUNTIME_PROXY_TOKEN_PERMISSIONS (which
includes actions: read) and fall back to BASE if the install hasn't
granted Actions read — mirroring the pattern in _create_sandbox_with_proxy.
Addresses review comment on PR #1642.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: widen split review diffs (#1647)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: install missing deps before verification (#1646)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: switch ui to pnpm (#1645)
* chore: require pnpm for ui agent work
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: switch ui to pnpm
Replace Bun and Yarn lockfiles with pnpm lockfile and update UI/Vercel commands to use pnpm.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* ci: use corepack for ui pnpm e2e build
Run pnpm through Corepack in the E2E global setup so CI can use the pinned package manager without a separate pnpm install step.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add Sonnet 5 to model picker (#1651)
* chore: update Sonnet examples to Sonnet 5
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: add Sonnet 5 to model picker
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* Remove dead breakout-thread e2e scenario after dropping the tool
The merge resolution deferred upstream's Slack breakout-thread tool
(slack_start_new_thread, #1638) since it depends on the #1621 dispatch
module, but the e2e harness still scripted it. Removing the tool name
from fake_llm.py's _tool_step call left a malformed scenario, crashing
the langgraph-dev web server at import (TypeError: _tool_step() missing
'call_id') and failing Playwright E2E.
Drop the "breakout" script scenario, its _is_breakout_request helper +
ScriptRule, and the corresponding full_flow.spec.ts test.
* Revert upstream pnpm switch; keep bun for the UI build
The merge auto-adopted upstream's pnpm switch (#1645) in tests/e2e/
global-setup.ts and ui/package.json, but our fork builds the UI with
bun (vercel.json + the E2E workflow's setup-bun). That left the
Playwright globalSetup running `corepack pnpm install --frozen-lockfile`
with no pnpm-lock.yaml, failing E2E at UI build time.
Revert global-setup.ts and ui/package.json to the dev (bun) baseline,
drop the merge-added ui/pnpm-lock.yaml, and remove the re-added
ui/AGENTS.md (our fork had deleted it).
* Align plan-review e2e + UI with the HEAD (pre-#1635) backend
The merge left a split plan vertical: the backend save_plan/plan_api are
HEAD (we deferred the editable-plan/sandbox-publish features #1610/#1635/
#1637 per #80), but the plan UI and e2e harness were upstream's. The
fake_llm scenario called save_plan(plan_file_path=...) — upstream's
file-based #1635 contract — while HEAD save_plan takes plan_markdown,
so the plan never saved and PlanReview never rendered (E2E failure on
the plan-review locator).
Pass plan_markdown to save_plan, and revert PlanReview.tsx / plan.ts /
$threadId_.plan.tsx / plan_review.spec.ts to the dev baseline so the
whole plan flow (save -> render -> approve -> implement) is consistent
with the HEAD backend.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Ramon Nogueira <ramon.nogueira@langchain.dev>
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: Caroline di Vittorio <43390382+carolinedivittorio@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
Co-authored-by: Mukil Loganathan <mukil@langchain.dev>
2026-06-30 16:45:19 -04:00
|
|
|
return await _publish_review_eval_dry_run_async(
|
|
|
|
|
head_sha=head_sha,
|
|
|
|
|
severity_threshold=_cast_severity(severity_threshold),
|
2026-07-17 12:10:54 -04:00
|
|
|
cap=eval_cap,
|
2026-05-18 15:47:13 -07:00
|
|
|
)
|
2026-06-10 10:44:13 -07:00
|
|
|
except ReviewerThreadMissingError as exc:
|
|
|
|
|
return thread_missing_tool_result(exc)
|
2026-05-18 15:47:13 -07:00
|
|
|
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
token = get_github_token()
|
|
|
|
|
if not token:
|
|
|
|
|
return {"success": False, "error": "No GitHub token available"}
|
|
|
|
|
|
2026-07-31 13:02:07 -04:00
|
|
|
if configurable.get("verdict_requested") is True:
|
|
|
|
|
verdict_authorization = "requested"
|
|
|
|
|
elif configurable.get("verdict_authorized") is True:
|
|
|
|
|
verdict_authorization = "consistent"
|
|
|
|
|
else:
|
|
|
|
|
verdict_authorization = "none"
|
2026-07-20 15:28:00 -04:00
|
|
|
|
2026-05-08 22:57:01 +00:00
|
|
|
try:
|
2026-07-20 15:28:00 -04:00
|
|
|
result = await _publish_review_async(
|
chore: sync upstream/main, defer #1621 modular webhooks (#81)
* chore: bake sfw binary into sandbox image (#1611)
sfw only ships a launcher that fetches its real binary at first run and does a
daily update check against api.github.com/repos/SocketDev/sfw-free. Both fail in
the sandbox (restricted egress; the proxy injects the GitHub App installation
token, which lacks access to that repo), so `sfw yarn install` errors with
"could not fetch its binary". Pin sfw 2.0.6, warm + verify the binary cache at
build, and set SFW_SKIP_UPDATE_CHECK=1 so runs use the baked binary offline.
* feat: editable plan mode + fix review-plan banner overlap (#1610)
* feat: editable plan mode + fix review-plan banner overlap
Lets the thread owner edit the plan markdown by hand from the plan-review
page (Edit -> textarea -> Save) via a new PUT /dashboard/api/plan/{id}
endpoint that re-publishes the plan and mirrors it into the sandbox
plan.md, so approve hands the edited plan to the agent as the source of
truth. Also fixes the collapsed git-panel's floating expand button
covering the "Review plan ->" banner by reserving space for it.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: abort plan approval when the published plan read fails
get_plan_content() swallowed store errors and returned None, so a
transient failure during approve would still mark the plan approved and
dispatch the generic fallback text — silently dropping an owner's edited
plan. Read the plan strictly (raise_on_error=True) so approval aborts
instead, matching the comment read.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: show message timestamps (#1609)
* feat: show message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: suppress fallback message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: stable message + tool-call hover timestamps
Stamp a stable client-side arrival time per message and tool call (keyed
by id, persisted to localStorage). Messages render the timestamp inline;
tool rows reveal a dim timestamp chip on hover. Real backend created_at
still takes precedence when present.
* fix: hide client-stamped message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add PR trace resolution (#1612)
* feat: add PR trace resolution
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: inject reviewer trace context as JSON
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: address review on PR trace resolution
Use the documented LangSmith metadata filter syntax
(and(eq(metadata_key,...), eq(metadata_value,...))) instead of
has(metadata, '{...}'), which does not match runs — _list_thread_runs
was silently returning nothing. Bound full-text searches to a 90-day
window so they don't hit LangSmith's large-window rate limit.
Also folds in the best-effort branch->head-sha resolver (dropping the
weighted scoring/threshold + repo/file evidence + GitHub hydration),
sandbox JSON injection, and the admin "Resolve trace" dry-run endpoint.
The IDOR findings are moot: resolve_pr_to_threads/summarize_agent_session
were removed; resolution now runs deterministically from the trusted run
config with no model-controlled pr_url or thread_id.
* fix: scope branch trace search to the repo
Branch names like fix-tests aren't unique across repos (or older PRs) in
a shared tracing project, so an unscoped branch hit could resolve to an
unrelated thread and write its runs into the reviewer sandbox. Require
the repo slug to co-occur with the branch in matched runs; the full head
SHA stays unscoped since it is globally unique. Addresses open-swe review
on PR #1612.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: include plan links in PR descriptions (#1613)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: gate workflow pushes with approval (#1614)
* feat: gate workflow pushes with approval
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve proxy refresh test compatibility
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: bind workflow approvals to pushed ref
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: recover thread work as patch (#1615)
* feat: recover thread work as patch
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: search sandbox cwd for recovery patches
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: omit plan link in PR description when no plan exists (#1618)
Plan links in PR descriptions were always built from the thread id, so
runs that never produced a plan linked to an empty plan-review page.
Now the plan content store is consulted first; the link is only added
when a plan with non-empty markdown actually exists. A transient store
failure degrades gracefully (no link) rather than blocking PR creation.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add filter & grouping menu to agents threads sidebar (#1617)
Add a Cursor-style control to the agents sidebar that groups (None/Date/
Status/Project), filters (ownership, status, source, pull request, model,
repo, include-resolved), and compacts the threads list. All client-side over
already-fetched sidebar threads; preferences persist in localStorage.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: update langsmith sdk to 0.9.3 (#1616)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: clickable shared PR header in git panel and reviews (#1620)
* feat: clickable shared PR header in git panel and reviews
Replace the standalone "View PR" button in the agent git panel with a
clickable PR title, matching the reviews view. Extract a shared PrHeader
component reused by both the git panel and the review main body.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* refactor: drop PrHeader wrapper, use shared component directly
The review-side PrHeader was just a thin adapter mapping detail -> the
shared component's props. Inline it at the call site and use the shared
PrHeader directly so there's a single component.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* refactor: durable interrupt dispatch + completion webhook (#1621)
* wip(rebuild): core reliability spine
- remove PR-babysitting (ci_autofix + ci_monitor graph + webhook wiring)
- dispatch core: agent/dispatch.py with multitask_strategy=interrupt +
durability=sync + completion webhook; reroute all webhook + plan triggers;
drop the racy in-process lock + is_thread_active busy-check
- completion webhook: agent/completion.py + /webhooks/run-complete loopback
route for failure/timeout replies (idempotent)
Co-authored-by: open-swe[bot]
* feat(rebuild): async tools, reconcile, shared http timeouts, assembly tuning
Parallel batch on top of the reliability spine:
- async-ify all 24 tools (drop asyncio.run; requests->httpx); re-implement the
http_request/fetch_url SSRF + DNS-rebinding defense httpx-natively and harden
the IP check to 'not is_global' (+ IPv4-mapped unwrap)
- reconcile.py: stale pending-run sweep (threads.search -> per-thread runs.list
-> cancel_many), wired into the scheduler graph via task='reconcile'
- shared DEFAULT_HTTP_TIMEOUT (agent/utils/http.py) on every bare
httpx.AsyncClient() across utils/dashboard/webapp/middleware
- run budget: MODEL_CALL_RECURSION_LIMIT 5000->250
- fix stale OpenAI->Anthropic fallback id (claude-opus-4-5 -> 4-8)
- drop redundant custom repair middleware (deepagents auto-adds PatchToolCalls)
- confirm tool-result eviction + summarization auto-wired via backend
- slim system prompt ~8% (full harness-profile rewrite deferred)
Co-authored-by: open-swe[bot]
* feat(rebuild): harness-profile prompt + split webhooks out of webapp
- prompt.py: own the system prompt via a registered harness profile
(OPEN_SWE_SHARED_BASE, kept neutral so the read-only reviewer/analyzer that
share it stay safe), registered across all 4 providers; per-thread values
stay in construct_system_prompt. Assembled main-agent prompt ~6.8k -> ~3.1k
tokens (~55% smaller); de-duped PR/commit/suite/force-push guidance; dropped
ALL-CAPS markers.
- webapp.py 3325 -> 1890 LOC: moved 14 per-source handlers into
agent/webhooks/{linear,slack,github}.py; webapp re-exports them for the
routes + tests; moved handlers reach shared helpers via the webapp namespace
to preserve the test suite's monkeypatch targets.
Full suite: 1168 passing, lint clean.
Co-authored-by: open-swe[bot]
* Restore MODEL_CALL_RECURSION_LIMIT to 5000 for long-running tasks
Reverts the 250 cap from the run-budget change — long-running tasks legitimately
need many model calls. The notify_step_limit_reached safety net still fires if a
run does hit the cap, so runs end with a signal either way.
Co-authored-by: open-swe[bot]
* fix: address PR review (auth, SSRF, interrupted status, redirect headers)
- completion.py: drop `interrupted` from failure statuses — with
multitask_strategy=interrupt a follow-up ends the prior run as interrupted,
which is healthy, not a failure to report. [open-swe]
- /webhooks/run-complete: shared-secret auth — dispatch appends ?token= when
RUN_COMPLETE_WEBHOOK_SECRET is set; route verifies via hmac.compare_digest.
[corridor-security]
- SSRF: extract the URL validator to agent/utils/url_safety.py and apply it
before server-side image fetches in multimodal.fetch_image_block.
[corridor-security]
- http_request: preserve caller headers/extensions across redirect hops instead
of dropping them on the first hop. [open-swe]
Co-authored-by: open-swe[bot]
* chore: remove REBUILD_PLAN.md (planning doc, not needed in the repo)
Co-authored-by: open-swe[bot]
* fix: fail closed on run-complete webhook auth when secret unset
Corridor follow-up: verify_run_complete_token returns False (not True) when
RUN_COMPLETE_WEBHOOK_SECRET is unset, so the public route is never
unauthenticated. Logs a startup warning when the secret is absent, and dispatch
skips registering the webhook when there's no secret (no rejected callbacks).
Co-authored-by: open-swe[bot]
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: restore forced tool call to prevent premature run stops (#1622)
Restore the ensure_no_empty_msg middleware and the always-call-a-tool system-prompt instruction that #1535 removed. When the model emits a message with no tool call (and hasn't already messaged the user or confirmed completion), the middleware re-injects a no_op / confirming_completion tool call so the run continues instead of ending mid-task.
Shipping to test whether it fixes runs that stop halfway through.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore(deps): bump langgraph-checkpoint from 4.1.0 to 4.1.1 (#1619)
Bumps [langgraph-checkpoint](https://github.com/langchain-ai/langgraph) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/langchain-ai/langgraph/releases)
- [Commits](https://github.com/langchain-ai/langgraph/compare/checkpoint==4.1.0...checkpoint==4.1.1)
---
updated-dependencies:
- dependency-name: langgraph-checkpoint
dependency-version: 4.1.1
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix: post reviewer resolution notes verbatim (#1624)
* fix: post reviewer resolution notes verbatim
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: stabilize dashboard follow-up e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve dashboard attribution in e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: make e2e attribution marker durable
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: only echo found e2e attribution
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: check live dashboard attribution in e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* hotfix: stop prompting agent/reviewer to wrap installs in sfw (#1625)
Installs hung when prefixed with sfw inside the sandbox (trace 019f0608
stalled on a pending `sfw npm install` execute, never returned). Strip the
Socket Firewall guidance from the agent and reviewer prompts so installs run
through the project's package manager directly. sfw stays in the Docker image;
nothing invokes it now.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: make plan view mobile friendly (#1636)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: fall back to vision model for image threads (#1626)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: surface Slack thread errors (#1627)
* fix: surface Slack thread errors
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: don't set failure_reply_posted on Slack preprocessing errors
The preprocessing error handler was setting failure_reply_posted=True,
the same idempotency flag handle_run_completion checks to suppress
duplicate run-failure replies. Since preprocessing failures happen
before any run exists but the flag persists on the thread, a subsequent
run failure on the same thread would be silently ignored.
The preprocessing handler already posts its own Slack reply, so the
run-completion idempotency flag should not be set here.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: avoid recapping Slack replies (#1629)
* chore: avoid recapping Slack replies
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: simplify Slack reply prompt wording
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: update Slack trace reply on web handoff (#1630)
* fix: update Slack trace reply on web handoff
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: trigger web handoff on dashboard starts
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: format web handoff as contextual fragment
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve trace_message_ts when overwriting Slack run mapping
When store_slack_run_mapping is called without trace_message_ts (e.g. on
follow-up Slack mentions), it was unconditionally overwriting the
thread-level mapping and clobbering the timestamp captured from the
initial trace reply. After that, _notify_slack_web_handoff could not find
the original message, so a subsequent move to Web silently skipped the
Slack trace update.
Now, when trace_message_ts is not passed, the existing thread mapping is
read first and its trace_message_ts is preserved.
* style: ruff format
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* fix: pre-bundle shiki/@pierre deps to stop dev dynamic-import failures (#1643)
* fix(ui): pre-bundle shiki/@pierre deps to stop dev dynamic-import failures
shiki lazy-imports a grammar per language and these libs only live inside
lazy route components, so Vite's startup scanner never sees them. They get
discovered on first thread navigation, triggering a dep re-optimize +
force-reload that aborts the in-flight route-chunk import, surfacing as
"Failed to fetch dynamically imported module: .../$threadId.tsx".
Pre-bundle them (and the github themes + common code-block languages) via
optimizeDeps.include so the optimize happens once at startup. Dev-only;
production bundles are unaffected.
* fix: pre-bundle canonical shiki docker/make langs instead of aliases
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: show queued dashboard follow-ups (#1631)
* feat: show queued dashboard follow-ups
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: de-dupe queued follow-ups while streaming
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat: notify Slack on plan approval (#1632)
* feat: notify Slack on plan approval
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: post Slack approval notice after successful dispatch
Move the _maybe_post_plan_approved_to_slack call until after
_dispatch_followup succeeds so the Slack thread is not told
implementation is beginning before the LangGraph run is created.
Addresses PR review comment.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat: include Slack channel context in prompts (#1633)
Add cached Slack channel metadata enrichment for Slack-triggered runs so prompts can include channel names and descriptions without duplicate conversations.info calls.\n\nCo-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
* chore: keep plan guidance high-level (#1634)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: publish plans from sandbox files (#1635)
* feat: publish plans from sandbox files
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: avoid fixed plan filenames
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: virtualize local sandbox file paths
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve plan_file_path across set_plan_status
set_plan_status was rewriting the content record with only markdown
and status, dropping plan_file_path. After a reject, the owner's
dashboard edit would mirror to a different file than the agent's
original, and the next save_plan could republish the stale file.
Preserve plan_file_path when updating status.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: return to thread after plan approval (#1637)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add Slack breakout thread tool (#1638)
* feat: add Slack breakout thread tool
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: make fake LLM scripts declarative
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: exclude slack_start_new_thread from plan mode
The breakout tool can dispatch a fresh agent run that starts outside the
current plan-mode state, bypassing the approval flow. Add it to
PLAN_MODE_EXCLUDED_TOOLS so it's hidden alongside the other mutating
tools while planning.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: require bun for ui agent work (#1639)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: request actions read for sandbox logs (#1642)
* fix: request actions read for sandbox logs
Request optional Actions read permission for sandbox proxy tokens, with fallback for installations that have not approved it yet. Update setup docs and prompt guidance for safe GitHub Actions log usage.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: restore actions:read scope after workflow push
After an approved workflow push, the guard was restoring the proxy with
BASE_RUNTIME_PROXY_TOKEN_PERMISSIONS, which excludes the actions: read
scope this PR adds. Restore with RUNTIME_PROXY_TOKEN_PERMISSIONS (which
includes actions: read) and fall back to BASE if the install hasn't
granted Actions read — mirroring the pattern in _create_sandbox_with_proxy.
Addresses review comment on PR #1642.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: widen split review diffs (#1647)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: install missing deps before verification (#1646)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: switch ui to pnpm (#1645)
* chore: require pnpm for ui agent work
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: switch ui to pnpm
Replace Bun and Yarn lockfiles with pnpm lockfile and update UI/Vercel commands to use pnpm.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* ci: use corepack for ui pnpm e2e build
Run pnpm through Corepack in the E2E global setup so CI can use the pinned package manager without a separate pnpm install step.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add Sonnet 5 to model picker (#1651)
* chore: update Sonnet examples to Sonnet 5
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: add Sonnet 5 to model picker
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* Remove dead breakout-thread e2e scenario after dropping the tool
The merge resolution deferred upstream's Slack breakout-thread tool
(slack_start_new_thread, #1638) since it depends on the #1621 dispatch
module, but the e2e harness still scripted it. Removing the tool name
from fake_llm.py's _tool_step call left a malformed scenario, crashing
the langgraph-dev web server at import (TypeError: _tool_step() missing
'call_id') and failing Playwright E2E.
Drop the "breakout" script scenario, its _is_breakout_request helper +
ScriptRule, and the corresponding full_flow.spec.ts test.
* Revert upstream pnpm switch; keep bun for the UI build
The merge auto-adopted upstream's pnpm switch (#1645) in tests/e2e/
global-setup.ts and ui/package.json, but our fork builds the UI with
bun (vercel.json + the E2E workflow's setup-bun). That left the
Playwright globalSetup running `corepack pnpm install --frozen-lockfile`
with no pnpm-lock.yaml, failing E2E at UI build time.
Revert global-setup.ts and ui/package.json to the dev (bun) baseline,
drop the merge-added ui/pnpm-lock.yaml, and remove the re-added
ui/AGENTS.md (our fork had deleted it).
* Align plan-review e2e + UI with the HEAD (pre-#1635) backend
The merge left a split plan vertical: the backend save_plan/plan_api are
HEAD (we deferred the editable-plan/sandbox-publish features #1610/#1635/
#1637 per #80), but the plan UI and e2e harness were upstream's. The
fake_llm scenario called save_plan(plan_file_path=...) — upstream's
file-based #1635 contract — while HEAD save_plan takes plan_markdown,
so the plan never saved and PlanReview never rendered (E2E failure on
the plan-review locator).
Pass plan_markdown to save_plan, and revert PlanReview.tsx / plan.ts /
$threadId_.plan.tsx / plan_review.spec.ts to the dev baseline so the
whole plan flow (save -> render -> approve -> implement) is consistent
with the HEAD backend.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Ramon Nogueira <ramon.nogueira@langchain.dev>
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: Caroline di Vittorio <43390382+carolinedivittorio@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
Co-authored-by: Mukil Loganathan <mukil@langchain.dev>
2026-06-30 16:45:19 -04:00
|
|
|
owner=str(repo_config["owner"]),
|
|
|
|
|
repo=str(repo_config["name"]),
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
head_sha=head_sha,
|
|
|
|
|
token=token,
|
|
|
|
|
severity_threshold=_cast_severity(severity_threshold),
|
2026-07-17 12:10:54 -04:00
|
|
|
cap=REVIEW_FINDING_CAP,
|
chore: sync upstream/main, defer #1621 modular webhooks (#81)
* chore: bake sfw binary into sandbox image (#1611)
sfw only ships a launcher that fetches its real binary at first run and does a
daily update check against api.github.com/repos/SocketDev/sfw-free. Both fail in
the sandbox (restricted egress; the proxy injects the GitHub App installation
token, which lacks access to that repo), so `sfw yarn install` errors with
"could not fetch its binary". Pin sfw 2.0.6, warm + verify the binary cache at
build, and set SFW_SKIP_UPDATE_CHECK=1 so runs use the baked binary offline.
* feat: editable plan mode + fix review-plan banner overlap (#1610)
* feat: editable plan mode + fix review-plan banner overlap
Lets the thread owner edit the plan markdown by hand from the plan-review
page (Edit -> textarea -> Save) via a new PUT /dashboard/api/plan/{id}
endpoint that re-publishes the plan and mirrors it into the sandbox
plan.md, so approve hands the edited plan to the agent as the source of
truth. Also fixes the collapsed git-panel's floating expand button
covering the "Review plan ->" banner by reserving space for it.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: abort plan approval when the published plan read fails
get_plan_content() swallowed store errors and returned None, so a
transient failure during approve would still mark the plan approved and
dispatch the generic fallback text — silently dropping an owner's edited
plan. Read the plan strictly (raise_on_error=True) so approval aborts
instead, matching the comment read.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: show message timestamps (#1609)
* feat: show message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: suppress fallback message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: stable message + tool-call hover timestamps
Stamp a stable client-side arrival time per message and tool call (keyed
by id, persisted to localStorage). Messages render the timestamp inline;
tool rows reveal a dim timestamp chip on hover. Real backend created_at
still takes precedence when present.
* fix: hide client-stamped message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add PR trace resolution (#1612)
* feat: add PR trace resolution
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: inject reviewer trace context as JSON
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: address review on PR trace resolution
Use the documented LangSmith metadata filter syntax
(and(eq(metadata_key,...), eq(metadata_value,...))) instead of
has(metadata, '{...}'), which does not match runs — _list_thread_runs
was silently returning nothing. Bound full-text searches to a 90-day
window so they don't hit LangSmith's large-window rate limit.
Also folds in the best-effort branch->head-sha resolver (dropping the
weighted scoring/threshold + repo/file evidence + GitHub hydration),
sandbox JSON injection, and the admin "Resolve trace" dry-run endpoint.
The IDOR findings are moot: resolve_pr_to_threads/summarize_agent_session
were removed; resolution now runs deterministically from the trusted run
config with no model-controlled pr_url or thread_id.
* fix: scope branch trace search to the repo
Branch names like fix-tests aren't unique across repos (or older PRs) in
a shared tracing project, so an unscoped branch hit could resolve to an
unrelated thread and write its runs into the reviewer sandbox. Require
the repo slug to co-occur with the branch in matched runs; the full head
SHA stays unscoped since it is globally unique. Addresses open-swe review
on PR #1612.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: include plan links in PR descriptions (#1613)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: gate workflow pushes with approval (#1614)
* feat: gate workflow pushes with approval
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve proxy refresh test compatibility
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: bind workflow approvals to pushed ref
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: recover thread work as patch (#1615)
* feat: recover thread work as patch
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: search sandbox cwd for recovery patches
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: omit plan link in PR description when no plan exists (#1618)
Plan links in PR descriptions were always built from the thread id, so
runs that never produced a plan linked to an empty plan-review page.
Now the plan content store is consulted first; the link is only added
when a plan with non-empty markdown actually exists. A transient store
failure degrades gracefully (no link) rather than blocking PR creation.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add filter & grouping menu to agents threads sidebar (#1617)
Add a Cursor-style control to the agents sidebar that groups (None/Date/
Status/Project), filters (ownership, status, source, pull request, model,
repo, include-resolved), and compacts the threads list. All client-side over
already-fetched sidebar threads; preferences persist in localStorage.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: update langsmith sdk to 0.9.3 (#1616)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: clickable shared PR header in git panel and reviews (#1620)
* feat: clickable shared PR header in git panel and reviews
Replace the standalone "View PR" button in the agent git panel with a
clickable PR title, matching the reviews view. Extract a shared PrHeader
component reused by both the git panel and the review main body.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* refactor: drop PrHeader wrapper, use shared component directly
The review-side PrHeader was just a thin adapter mapping detail -> the
shared component's props. Inline it at the call site and use the shared
PrHeader directly so there's a single component.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* refactor: durable interrupt dispatch + completion webhook (#1621)
* wip(rebuild): core reliability spine
- remove PR-babysitting (ci_autofix + ci_monitor graph + webhook wiring)
- dispatch core: agent/dispatch.py with multitask_strategy=interrupt +
durability=sync + completion webhook; reroute all webhook + plan triggers;
drop the racy in-process lock + is_thread_active busy-check
- completion webhook: agent/completion.py + /webhooks/run-complete loopback
route for failure/timeout replies (idempotent)
Co-authored-by: open-swe[bot]
* feat(rebuild): async tools, reconcile, shared http timeouts, assembly tuning
Parallel batch on top of the reliability spine:
- async-ify all 24 tools (drop asyncio.run; requests->httpx); re-implement the
http_request/fetch_url SSRF + DNS-rebinding defense httpx-natively and harden
the IP check to 'not is_global' (+ IPv4-mapped unwrap)
- reconcile.py: stale pending-run sweep (threads.search -> per-thread runs.list
-> cancel_many), wired into the scheduler graph via task='reconcile'
- shared DEFAULT_HTTP_TIMEOUT (agent/utils/http.py) on every bare
httpx.AsyncClient() across utils/dashboard/webapp/middleware
- run budget: MODEL_CALL_RECURSION_LIMIT 5000->250
- fix stale OpenAI->Anthropic fallback id (claude-opus-4-5 -> 4-8)
- drop redundant custom repair middleware (deepagents auto-adds PatchToolCalls)
- confirm tool-result eviction + summarization auto-wired via backend
- slim system prompt ~8% (full harness-profile rewrite deferred)
Co-authored-by: open-swe[bot]
* feat(rebuild): harness-profile prompt + split webhooks out of webapp
- prompt.py: own the system prompt via a registered harness profile
(OPEN_SWE_SHARED_BASE, kept neutral so the read-only reviewer/analyzer that
share it stay safe), registered across all 4 providers; per-thread values
stay in construct_system_prompt. Assembled main-agent prompt ~6.8k -> ~3.1k
tokens (~55% smaller); de-duped PR/commit/suite/force-push guidance; dropped
ALL-CAPS markers.
- webapp.py 3325 -> 1890 LOC: moved 14 per-source handlers into
agent/webhooks/{linear,slack,github}.py; webapp re-exports them for the
routes + tests; moved handlers reach shared helpers via the webapp namespace
to preserve the test suite's monkeypatch targets.
Full suite: 1168 passing, lint clean.
Co-authored-by: open-swe[bot]
* Restore MODEL_CALL_RECURSION_LIMIT to 5000 for long-running tasks
Reverts the 250 cap from the run-budget change — long-running tasks legitimately
need many model calls. The notify_step_limit_reached safety net still fires if a
run does hit the cap, so runs end with a signal either way.
Co-authored-by: open-swe[bot]
* fix: address PR review (auth, SSRF, interrupted status, redirect headers)
- completion.py: drop `interrupted` from failure statuses — with
multitask_strategy=interrupt a follow-up ends the prior run as interrupted,
which is healthy, not a failure to report. [open-swe]
- /webhooks/run-complete: shared-secret auth — dispatch appends ?token= when
RUN_COMPLETE_WEBHOOK_SECRET is set; route verifies via hmac.compare_digest.
[corridor-security]
- SSRF: extract the URL validator to agent/utils/url_safety.py and apply it
before server-side image fetches in multimodal.fetch_image_block.
[corridor-security]
- http_request: preserve caller headers/extensions across redirect hops instead
of dropping them on the first hop. [open-swe]
Co-authored-by: open-swe[bot]
* chore: remove REBUILD_PLAN.md (planning doc, not needed in the repo)
Co-authored-by: open-swe[bot]
* fix: fail closed on run-complete webhook auth when secret unset
Corridor follow-up: verify_run_complete_token returns False (not True) when
RUN_COMPLETE_WEBHOOK_SECRET is unset, so the public route is never
unauthenticated. Logs a startup warning when the secret is absent, and dispatch
skips registering the webhook when there's no secret (no rejected callbacks).
Co-authored-by: open-swe[bot]
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: restore forced tool call to prevent premature run stops (#1622)
Restore the ensure_no_empty_msg middleware and the always-call-a-tool system-prompt instruction that #1535 removed. When the model emits a message with no tool call (and hasn't already messaged the user or confirmed completion), the middleware re-injects a no_op / confirming_completion tool call so the run continues instead of ending mid-task.
Shipping to test whether it fixes runs that stop halfway through.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore(deps): bump langgraph-checkpoint from 4.1.0 to 4.1.1 (#1619)
Bumps [langgraph-checkpoint](https://github.com/langchain-ai/langgraph) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/langchain-ai/langgraph/releases)
- [Commits](https://github.com/langchain-ai/langgraph/compare/checkpoint==4.1.0...checkpoint==4.1.1)
---
updated-dependencies:
- dependency-name: langgraph-checkpoint
dependency-version: 4.1.1
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix: post reviewer resolution notes verbatim (#1624)
* fix: post reviewer resolution notes verbatim
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: stabilize dashboard follow-up e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve dashboard attribution in e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: make e2e attribution marker durable
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: only echo found e2e attribution
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: check live dashboard attribution in e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* hotfix: stop prompting agent/reviewer to wrap installs in sfw (#1625)
Installs hung when prefixed with sfw inside the sandbox (trace 019f0608
stalled on a pending `sfw npm install` execute, never returned). Strip the
Socket Firewall guidance from the agent and reviewer prompts so installs run
through the project's package manager directly. sfw stays in the Docker image;
nothing invokes it now.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: make plan view mobile friendly (#1636)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: fall back to vision model for image threads (#1626)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: surface Slack thread errors (#1627)
* fix: surface Slack thread errors
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: don't set failure_reply_posted on Slack preprocessing errors
The preprocessing error handler was setting failure_reply_posted=True,
the same idempotency flag handle_run_completion checks to suppress
duplicate run-failure replies. Since preprocessing failures happen
before any run exists but the flag persists on the thread, a subsequent
run failure on the same thread would be silently ignored.
The preprocessing handler already posts its own Slack reply, so the
run-completion idempotency flag should not be set here.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: avoid recapping Slack replies (#1629)
* chore: avoid recapping Slack replies
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: simplify Slack reply prompt wording
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: update Slack trace reply on web handoff (#1630)
* fix: update Slack trace reply on web handoff
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: trigger web handoff on dashboard starts
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: format web handoff as contextual fragment
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve trace_message_ts when overwriting Slack run mapping
When store_slack_run_mapping is called without trace_message_ts (e.g. on
follow-up Slack mentions), it was unconditionally overwriting the
thread-level mapping and clobbering the timestamp captured from the
initial trace reply. After that, _notify_slack_web_handoff could not find
the original message, so a subsequent move to Web silently skipped the
Slack trace update.
Now, when trace_message_ts is not passed, the existing thread mapping is
read first and its trace_message_ts is preserved.
* style: ruff format
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* fix: pre-bundle shiki/@pierre deps to stop dev dynamic-import failures (#1643)
* fix(ui): pre-bundle shiki/@pierre deps to stop dev dynamic-import failures
shiki lazy-imports a grammar per language and these libs only live inside
lazy route components, so Vite's startup scanner never sees them. They get
discovered on first thread navigation, triggering a dep re-optimize +
force-reload that aborts the in-flight route-chunk import, surfacing as
"Failed to fetch dynamically imported module: .../$threadId.tsx".
Pre-bundle them (and the github themes + common code-block languages) via
optimizeDeps.include so the optimize happens once at startup. Dev-only;
production bundles are unaffected.
* fix: pre-bundle canonical shiki docker/make langs instead of aliases
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: show queued dashboard follow-ups (#1631)
* feat: show queued dashboard follow-ups
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: de-dupe queued follow-ups while streaming
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat: notify Slack on plan approval (#1632)
* feat: notify Slack on plan approval
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: post Slack approval notice after successful dispatch
Move the _maybe_post_plan_approved_to_slack call until after
_dispatch_followup succeeds so the Slack thread is not told
implementation is beginning before the LangGraph run is created.
Addresses PR review comment.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat: include Slack channel context in prompts (#1633)
Add cached Slack channel metadata enrichment for Slack-triggered runs so prompts can include channel names and descriptions without duplicate conversations.info calls.\n\nCo-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
* chore: keep plan guidance high-level (#1634)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: publish plans from sandbox files (#1635)
* feat: publish plans from sandbox files
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: avoid fixed plan filenames
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: virtualize local sandbox file paths
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve plan_file_path across set_plan_status
set_plan_status was rewriting the content record with only markdown
and status, dropping plan_file_path. After a reject, the owner's
dashboard edit would mirror to a different file than the agent's
original, and the next save_plan could republish the stale file.
Preserve plan_file_path when updating status.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: return to thread after plan approval (#1637)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add Slack breakout thread tool (#1638)
* feat: add Slack breakout thread tool
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: make fake LLM scripts declarative
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: exclude slack_start_new_thread from plan mode
The breakout tool can dispatch a fresh agent run that starts outside the
current plan-mode state, bypassing the approval flow. Add it to
PLAN_MODE_EXCLUDED_TOOLS so it's hidden alongside the other mutating
tools while planning.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: require bun for ui agent work (#1639)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: request actions read for sandbox logs (#1642)
* fix: request actions read for sandbox logs
Request optional Actions read permission for sandbox proxy tokens, with fallback for installations that have not approved it yet. Update setup docs and prompt guidance for safe GitHub Actions log usage.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: restore actions:read scope after workflow push
After an approved workflow push, the guard was restoring the proxy with
BASE_RUNTIME_PROXY_TOKEN_PERMISSIONS, which excludes the actions: read
scope this PR adds. Restore with RUNTIME_PROXY_TOKEN_PERMISSIONS (which
includes actions: read) and fall back to BASE if the install hasn't
granted Actions read — mirroring the pattern in _create_sandbox_with_proxy.
Addresses review comment on PR #1642.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: widen split review diffs (#1647)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: install missing deps before verification (#1646)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: switch ui to pnpm (#1645)
* chore: require pnpm for ui agent work
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: switch ui to pnpm
Replace Bun and Yarn lockfiles with pnpm lockfile and update UI/Vercel commands to use pnpm.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* ci: use corepack for ui pnpm e2e build
Run pnpm through Corepack in the E2E global setup so CI can use the pinned package manager without a separate pnpm install step.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add Sonnet 5 to model picker (#1651)
* chore: update Sonnet examples to Sonnet 5
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: add Sonnet 5 to model picker
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* Remove dead breakout-thread e2e scenario after dropping the tool
The merge resolution deferred upstream's Slack breakout-thread tool
(slack_start_new_thread, #1638) since it depends on the #1621 dispatch
module, but the e2e harness still scripted it. Removing the tool name
from fake_llm.py's _tool_step call left a malformed scenario, crashing
the langgraph-dev web server at import (TypeError: _tool_step() missing
'call_id') and failing Playwright E2E.
Drop the "breakout" script scenario, its _is_breakout_request helper +
ScriptRule, and the corresponding full_flow.spec.ts test.
* Revert upstream pnpm switch; keep bun for the UI build
The merge auto-adopted upstream's pnpm switch (#1645) in tests/e2e/
global-setup.ts and ui/package.json, but our fork builds the UI with
bun (vercel.json + the E2E workflow's setup-bun). That left the
Playwright globalSetup running `corepack pnpm install --frozen-lockfile`
with no pnpm-lock.yaml, failing E2E at UI build time.
Revert global-setup.ts and ui/package.json to the dev (bun) baseline,
drop the merge-added ui/pnpm-lock.yaml, and remove the re-added
ui/AGENTS.md (our fork had deleted it).
* Align plan-review e2e + UI with the HEAD (pre-#1635) backend
The merge left a split plan vertical: the backend save_plan/plan_api are
HEAD (we deferred the editable-plan/sandbox-publish features #1610/#1635/
#1637 per #80), but the plan UI and e2e harness were upstream's. The
fake_llm scenario called save_plan(plan_file_path=...) — upstream's
file-based #1635 contract — while HEAD save_plan takes plan_markdown,
so the plan never saved and PlanReview never rendered (E2E failure on
the plan-review locator).
Pass plan_markdown to save_plan, and revert PlanReview.tsx / plan.ts /
$threadId_.plan.tsx / plan_review.spec.ts to the dev baseline so the
whole plan flow (save -> render -> approve -> implement) is consistent
with the HEAD backend.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Ramon Nogueira <ramon.nogueira@langchain.dev>
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: Caroline di Vittorio <43390382+carolinedivittorio@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
Co-authored-by: Mukil Loganathan <mukil@langchain.dev>
2026-06-30 16:45:19 -04:00
|
|
|
is_re_review=is_re_review,
|
|
|
|
|
langgraph_run_id=_current_run_id(config),
|
|
|
|
|
trace_link_config_override=configurable.get("review_trace_link_enabled"),
|
2026-07-20 15:28:00 -04:00
|
|
|
verdict=verdict,
|
|
|
|
|
verdict_requester=str(configurable.get("github_login") or ""),
|
2026-07-31 13:02:07 -04:00
|
|
|
verdict_authorization=verdict_authorization,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
)
|
2026-07-20 15:28:00 -04:00
|
|
|
return result
|
2026-06-10 10:44:13 -07:00
|
|
|
except ReviewerThreadMissingError as exc:
|
|
|
|
|
return thread_missing_tool_result(exc)
|
2026-05-08 22:57:01 +00:00
|
|
|
except GitHubAuthError as exc:
|
|
|
|
|
thread_id = get_thread_id_from_runtime()
|
|
|
|
|
if thread_id:
|
chore: sync upstream/main, defer #1621 modular webhooks (#81)
* chore: bake sfw binary into sandbox image (#1611)
sfw only ships a launcher that fetches its real binary at first run and does a
daily update check against api.github.com/repos/SocketDev/sfw-free. Both fail in
the sandbox (restricted egress; the proxy injects the GitHub App installation
token, which lacks access to that repo), so `sfw yarn install` errors with
"could not fetch its binary". Pin sfw 2.0.6, warm + verify the binary cache at
build, and set SFW_SKIP_UPDATE_CHECK=1 so runs use the baked binary offline.
* feat: editable plan mode + fix review-plan banner overlap (#1610)
* feat: editable plan mode + fix review-plan banner overlap
Lets the thread owner edit the plan markdown by hand from the plan-review
page (Edit -> textarea -> Save) via a new PUT /dashboard/api/plan/{id}
endpoint that re-publishes the plan and mirrors it into the sandbox
plan.md, so approve hands the edited plan to the agent as the source of
truth. Also fixes the collapsed git-panel's floating expand button
covering the "Review plan ->" banner by reserving space for it.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: abort plan approval when the published plan read fails
get_plan_content() swallowed store errors and returned None, so a
transient failure during approve would still mark the plan approved and
dispatch the generic fallback text — silently dropping an owner's edited
plan. Read the plan strictly (raise_on_error=True) so approval aborts
instead, matching the comment read.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: show message timestamps (#1609)
* feat: show message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: suppress fallback message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: stable message + tool-call hover timestamps
Stamp a stable client-side arrival time per message and tool call (keyed
by id, persisted to localStorage). Messages render the timestamp inline;
tool rows reveal a dim timestamp chip on hover. Real backend created_at
still takes precedence when present.
* fix: hide client-stamped message timestamps
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add PR trace resolution (#1612)
* feat: add PR trace resolution
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: inject reviewer trace context as JSON
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: address review on PR trace resolution
Use the documented LangSmith metadata filter syntax
(and(eq(metadata_key,...), eq(metadata_value,...))) instead of
has(metadata, '{...}'), which does not match runs — _list_thread_runs
was silently returning nothing. Bound full-text searches to a 90-day
window so they don't hit LangSmith's large-window rate limit.
Also folds in the best-effort branch->head-sha resolver (dropping the
weighted scoring/threshold + repo/file evidence + GitHub hydration),
sandbox JSON injection, and the admin "Resolve trace" dry-run endpoint.
The IDOR findings are moot: resolve_pr_to_threads/summarize_agent_session
were removed; resolution now runs deterministically from the trusted run
config with no model-controlled pr_url or thread_id.
* fix: scope branch trace search to the repo
Branch names like fix-tests aren't unique across repos (or older PRs) in
a shared tracing project, so an unscoped branch hit could resolve to an
unrelated thread and write its runs into the reviewer sandbox. Require
the repo slug to co-occur with the branch in matched runs; the full head
SHA stays unscoped since it is globally unique. Addresses open-swe review
on PR #1612.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: include plan links in PR descriptions (#1613)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: gate workflow pushes with approval (#1614)
* feat: gate workflow pushes with approval
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve proxy refresh test compatibility
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: bind workflow approvals to pushed ref
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: recover thread work as patch (#1615)
* feat: recover thread work as patch
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: search sandbox cwd for recovery patches
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: omit plan link in PR description when no plan exists (#1618)
Plan links in PR descriptions were always built from the thread id, so
runs that never produced a plan linked to an empty plan-review page.
Now the plan content store is consulted first; the link is only added
when a plan with non-empty markdown actually exists. A transient store
failure degrades gracefully (no link) rather than blocking PR creation.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add filter & grouping menu to agents threads sidebar (#1617)
Add a Cursor-style control to the agents sidebar that groups (None/Date/
Status/Project), filters (ownership, status, source, pull request, model,
repo, include-resolved), and compacts the threads list. All client-side over
already-fetched sidebar threads; preferences persist in localStorage.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: update langsmith sdk to 0.9.3 (#1616)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: clickable shared PR header in git panel and reviews (#1620)
* feat: clickable shared PR header in git panel and reviews
Replace the standalone "View PR" button in the agent git panel with a
clickable PR title, matching the reviews view. Extract a shared PrHeader
component reused by both the git panel and the review main body.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* refactor: drop PrHeader wrapper, use shared component directly
The review-side PrHeader was just a thin adapter mapping detail -> the
shared component's props. Inline it at the call site and use the shared
PrHeader directly so there's a single component.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* refactor: durable interrupt dispatch + completion webhook (#1621)
* wip(rebuild): core reliability spine
- remove PR-babysitting (ci_autofix + ci_monitor graph + webhook wiring)
- dispatch core: agent/dispatch.py with multitask_strategy=interrupt +
durability=sync + completion webhook; reroute all webhook + plan triggers;
drop the racy in-process lock + is_thread_active busy-check
- completion webhook: agent/completion.py + /webhooks/run-complete loopback
route for failure/timeout replies (idempotent)
Co-authored-by: open-swe[bot]
* feat(rebuild): async tools, reconcile, shared http timeouts, assembly tuning
Parallel batch on top of the reliability spine:
- async-ify all 24 tools (drop asyncio.run; requests->httpx); re-implement the
http_request/fetch_url SSRF + DNS-rebinding defense httpx-natively and harden
the IP check to 'not is_global' (+ IPv4-mapped unwrap)
- reconcile.py: stale pending-run sweep (threads.search -> per-thread runs.list
-> cancel_many), wired into the scheduler graph via task='reconcile'
- shared DEFAULT_HTTP_TIMEOUT (agent/utils/http.py) on every bare
httpx.AsyncClient() across utils/dashboard/webapp/middleware
- run budget: MODEL_CALL_RECURSION_LIMIT 5000->250
- fix stale OpenAI->Anthropic fallback id (claude-opus-4-5 -> 4-8)
- drop redundant custom repair middleware (deepagents auto-adds PatchToolCalls)
- confirm tool-result eviction + summarization auto-wired via backend
- slim system prompt ~8% (full harness-profile rewrite deferred)
Co-authored-by: open-swe[bot]
* feat(rebuild): harness-profile prompt + split webhooks out of webapp
- prompt.py: own the system prompt via a registered harness profile
(OPEN_SWE_SHARED_BASE, kept neutral so the read-only reviewer/analyzer that
share it stay safe), registered across all 4 providers; per-thread values
stay in construct_system_prompt. Assembled main-agent prompt ~6.8k -> ~3.1k
tokens (~55% smaller); de-duped PR/commit/suite/force-push guidance; dropped
ALL-CAPS markers.
- webapp.py 3325 -> 1890 LOC: moved 14 per-source handlers into
agent/webhooks/{linear,slack,github}.py; webapp re-exports them for the
routes + tests; moved handlers reach shared helpers via the webapp namespace
to preserve the test suite's monkeypatch targets.
Full suite: 1168 passing, lint clean.
Co-authored-by: open-swe[bot]
* Restore MODEL_CALL_RECURSION_LIMIT to 5000 for long-running tasks
Reverts the 250 cap from the run-budget change — long-running tasks legitimately
need many model calls. The notify_step_limit_reached safety net still fires if a
run does hit the cap, so runs end with a signal either way.
Co-authored-by: open-swe[bot]
* fix: address PR review (auth, SSRF, interrupted status, redirect headers)
- completion.py: drop `interrupted` from failure statuses — with
multitask_strategy=interrupt a follow-up ends the prior run as interrupted,
which is healthy, not a failure to report. [open-swe]
- /webhooks/run-complete: shared-secret auth — dispatch appends ?token= when
RUN_COMPLETE_WEBHOOK_SECRET is set; route verifies via hmac.compare_digest.
[corridor-security]
- SSRF: extract the URL validator to agent/utils/url_safety.py and apply it
before server-side image fetches in multimodal.fetch_image_block.
[corridor-security]
- http_request: preserve caller headers/extensions across redirect hops instead
of dropping them on the first hop. [open-swe]
Co-authored-by: open-swe[bot]
* chore: remove REBUILD_PLAN.md (planning doc, not needed in the repo)
Co-authored-by: open-swe[bot]
* fix: fail closed on run-complete webhook auth when secret unset
Corridor follow-up: verify_run_complete_token returns False (not True) when
RUN_COMPLETE_WEBHOOK_SECRET is unset, so the public route is never
unauthenticated. Logs a startup warning when the secret is absent, and dispatch
skips registering the webhook when there's no secret (no rejected callbacks).
Co-authored-by: open-swe[bot]
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: restore forced tool call to prevent premature run stops (#1622)
Restore the ensure_no_empty_msg middleware and the always-call-a-tool system-prompt instruction that #1535 removed. When the model emits a message with no tool call (and hasn't already messaged the user or confirmed completion), the middleware re-injects a no_op / confirming_completion tool call so the run continues instead of ending mid-task.
Shipping to test whether it fixes runs that stop halfway through.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore(deps): bump langgraph-checkpoint from 4.1.0 to 4.1.1 (#1619)
Bumps [langgraph-checkpoint](https://github.com/langchain-ai/langgraph) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/langchain-ai/langgraph/releases)
- [Commits](https://github.com/langchain-ai/langgraph/compare/checkpoint==4.1.0...checkpoint==4.1.1)
---
updated-dependencies:
- dependency-name: langgraph-checkpoint
dependency-version: 4.1.1
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix: post reviewer resolution notes verbatim (#1624)
* fix: post reviewer resolution notes verbatim
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: stabilize dashboard follow-up e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve dashboard attribution in e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: make e2e attribution marker durable
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: only echo found e2e attribution
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: check live dashboard attribution in e2e
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* hotfix: stop prompting agent/reviewer to wrap installs in sfw (#1625)
Installs hung when prefixed with sfw inside the sandbox (trace 019f0608
stalled on a pending `sfw npm install` execute, never returned). Strip the
Socket Firewall guidance from the agent and reviewer prompts so installs run
through the project's package manager directly. sfw stays in the Docker image;
nothing invokes it now.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: make plan view mobile friendly (#1636)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: fall back to vision model for image threads (#1626)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: surface Slack thread errors (#1627)
* fix: surface Slack thread errors
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: don't set failure_reply_posted on Slack preprocessing errors
The preprocessing error handler was setting failure_reply_posted=True,
the same idempotency flag handle_run_completion checks to suppress
duplicate run-failure replies. Since preprocessing failures happen
before any run exists but the flag persists on the thread, a subsequent
run failure on the same thread would be silently ignored.
The preprocessing handler already posts its own Slack reply, so the
run-completion idempotency flag should not be set here.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: avoid recapping Slack replies (#1629)
* chore: avoid recapping Slack replies
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: simplify Slack reply prompt wording
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: update Slack trace reply on web handoff (#1630)
* fix: update Slack trace reply on web handoff
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: trigger web handoff on dashboard starts
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: format web handoff as contextual fragment
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve trace_message_ts when overwriting Slack run mapping
When store_slack_run_mapping is called without trace_message_ts (e.g. on
follow-up Slack mentions), it was unconditionally overwriting the
thread-level mapping and clobbering the timestamp captured from the
initial trace reply. After that, _notify_slack_web_handoff could not find
the original message, so a subsequent move to Web silently skipped the
Slack trace update.
Now, when trace_message_ts is not passed, the existing thread mapping is
read first and its trace_message_ts is preserved.
* style: ruff format
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* fix: pre-bundle shiki/@pierre deps to stop dev dynamic-import failures (#1643)
* fix(ui): pre-bundle shiki/@pierre deps to stop dev dynamic-import failures
shiki lazy-imports a grammar per language and these libs only live inside
lazy route components, so Vite's startup scanner never sees them. They get
discovered on first thread navigation, triggering a dep re-optimize +
force-reload that aborts the in-flight route-chunk import, surfacing as
"Failed to fetch dynamically imported module: .../$threadId.tsx".
Pre-bundle them (and the github themes + common code-block languages) via
optimizeDeps.include so the optimize happens once at startup. Dev-only;
production bundles are unaffected.
* fix: pre-bundle canonical shiki docker/make langs instead of aliases
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: show queued dashboard follow-ups (#1631)
* feat: show queued dashboard follow-ups
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: de-dupe queued follow-ups while streaming
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat: notify Slack on plan approval (#1632)
* feat: notify Slack on plan approval
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: post Slack approval notice after successful dispatch
Move the _maybe_post_plan_approved_to_slack call until after
_dispatch_followup succeeds so the Slack thread is not told
implementation is beginning before the LangGraph run is created.
Addresses PR review comment.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat: include Slack channel context in prompts (#1633)
Add cached Slack channel metadata enrichment for Slack-triggered runs so prompts can include channel names and descriptions without duplicate conversations.info calls.\n\nCo-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
* chore: keep plan guidance high-level (#1634)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: publish plans from sandbox files (#1635)
* feat: publish plans from sandbox files
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: avoid fixed plan filenames
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: virtualize local sandbox file paths
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve plan_file_path across set_plan_status
set_plan_status was rewriting the content record with only markdown
and status, dropping plan_file_path. After a reject, the owner's
dashboard edit would mirror to a different file than the agent's
original, and the next save_plan could republish the stale file.
Preserve plan_file_path when updating status.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: return to thread after plan approval (#1637)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add Slack breakout thread tool (#1638)
* feat: add Slack breakout thread tool
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: make fake LLM scripts declarative
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: exclude slack_start_new_thread from plan mode
The breakout tool can dispatch a fresh agent run that starts outside the
current plan-mode state, bypassing the approval flow. Add it to
PLAN_MODE_EXCLUDED_TOOLS so it's hidden alongside the other mutating
tools while planning.
---------
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: require bun for ui agent work (#1639)
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: request actions read for sandbox logs (#1642)
* fix: request actions read for sandbox logs
Request optional Actions read permission for sandbox proxy tokens, with fallback for installations that have not approved it yet. Update setup docs and prompt guidance for safe GitHub Actions log usage.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: restore actions:read scope after workflow push
After an approved workflow push, the guard was restoring the proxy with
BASE_RUNTIME_PROXY_TOKEN_PERMISSIONS, which excludes the actions: read
scope this PR adds. Restore with RUNTIME_PROXY_TOKEN_PERMISSIONS (which
includes actions: read) and fall back to BASE if the install hasn't
granted Actions read — mirroring the pattern in _create_sandbox_with_proxy.
Addresses review comment on PR #1642.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: widen split review diffs (#1647)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: install missing deps before verification (#1646)
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: switch ui to pnpm (#1645)
* chore: require pnpm for ui agent work
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: switch ui to pnpm
Replace Bun and Yarn lockfiles with pnpm lockfile and update UI/Vercel commands to use pnpm.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* ci: use corepack for ui pnpm e2e build
Run pnpm through Corepack in the E2E global setup so CI can use the pinned package manager without a separate pnpm install step.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: add Sonnet 5 to model picker (#1651)
* chore: update Sonnet examples to Sonnet 5
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* chore: add Sonnet 5 to model picker
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* Remove dead breakout-thread e2e scenario after dropping the tool
The merge resolution deferred upstream's Slack breakout-thread tool
(slack_start_new_thread, #1638) since it depends on the #1621 dispatch
module, but the e2e harness still scripted it. Removing the tool name
from fake_llm.py's _tool_step call left a malformed scenario, crashing
the langgraph-dev web server at import (TypeError: _tool_step() missing
'call_id') and failing Playwright E2E.
Drop the "breakout" script scenario, its _is_breakout_request helper +
ScriptRule, and the corresponding full_flow.spec.ts test.
* Revert upstream pnpm switch; keep bun for the UI build
The merge auto-adopted upstream's pnpm switch (#1645) in tests/e2e/
global-setup.ts and ui/package.json, but our fork builds the UI with
bun (vercel.json + the E2E workflow's setup-bun). That left the
Playwright globalSetup running `corepack pnpm install --frozen-lockfile`
with no pnpm-lock.yaml, failing E2E at UI build time.
Revert global-setup.ts and ui/package.json to the dev (bun) baseline,
drop the merge-added ui/pnpm-lock.yaml, and remove the re-added
ui/AGENTS.md (our fork had deleted it).
* Align plan-review e2e + UI with the HEAD (pre-#1635) backend
The merge left a split plan vertical: the backend save_plan/plan_api are
HEAD (we deferred the editable-plan/sandbox-publish features #1610/#1635/
#1637 per #80), but the plan UI and e2e harness were upstream's. The
fake_llm scenario called save_plan(plan_file_path=...) — upstream's
file-based #1635 contract — while HEAD save_plan takes plan_markdown,
so the plan never saved and PlanReview never rendered (E2E failure on
the plan-review locator).
Pass plan_markdown to save_plan, and revert PlanReview.tsx / plan.ts /
$threadId_.plan.tsx / plan_review.spec.ts to the dev baseline so the
whole plan flow (save -> render -> approve -> implement) is consistent
with the HEAD backend.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Ramon Nogueira <ramon.nogueira@langchain.dev>
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: Caroline di Vittorio <43390382+carolinedivittorio@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
Co-authored-by: Mukil Loganathan <mukil@langchain.dev>
2026-06-30 16:45:19 -04:00
|
|
|
await invalidate_cached_github_token(thread_id)
|
2026-05-08 22:57:01 +00:00
|
|
|
return {
|
|
|
|
|
"success": False,
|
|
|
|
|
"error": (
|
|
|
|
|
"GitHub returned 401 — the cached OAuth token is invalid or revoked. "
|
|
|
|
|
"Please re-authenticate and trigger the review again."
|
|
|
|
|
),
|
|
|
|
|
"auth_error": str(exc),
|
|
|
|
|
}
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
def _cast_severity(value: str) -> Severity:
|
|
|
|
|
return value # type: ignore[return-value]
|
|
|
|
|
|
|
|
|
|
|
2026-05-28 13:48:14 -07:00
|
|
|
async def _resolve_review_trace_url(thread_id: str, config_override: object) -> str | None:
|
|
|
|
|
if config_override is False:
|
|
|
|
|
return None
|
|
|
|
|
if not await get_team_review_trace_links_enabled():
|
|
|
|
|
return None
|
|
|
|
|
if not thread_id:
|
|
|
|
|
return None
|
2026-06-13 09:37:15 -07:00
|
|
|
return get_langsmith_trace_url(thread_id, project_name=REVIEW_TRACING_PROJECT)
|
2026-05-28 13:48:14 -07:00
|
|
|
|
|
|
|
|
|
2026-05-18 15:47:13 -07:00
|
|
|
def _is_reviewer_eval_mode(configurable: dict[str, Any]) -> bool:
|
|
|
|
|
return configurable.get("reviewer_eval") is True or configurable.get("eval") is True
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def _publish_review_eval_dry_run_async(
|
|
|
|
|
*,
|
|
|
|
|
head_sha: str,
|
|
|
|
|
severity_threshold: Severity,
|
|
|
|
|
cap: int,
|
|
|
|
|
) -> dict[str, Any]:
|
|
|
|
|
"""Simulate publish_review for benchmark runs without posting to GitHub."""
|
|
|
|
|
thread_id = get_thread_id_from_runtime()
|
|
|
|
|
findings = await list_findings_async(thread_id)
|
2026-05-27 10:52:30 -07:00
|
|
|
unpublished_findings = [f for f in findings if not _has_publication_identity(f)]
|
2026-05-18 15:47:13 -07:00
|
|
|
open_unpublished = [f for f in unpublished_findings if f.get("status", "open") == "open"]
|
2026-06-12 10:19:30 -07:00
|
|
|
# Out-of-diff findings are disabled: only in-diff findings are surfaced.
|
2026-06-05 10:41:48 -07:00
|
|
|
in_diff_unpublished = [f for f in unpublished_findings if f.get("in_diff", True)]
|
2026-05-18 15:47:13 -07:00
|
|
|
eligible = filter_findings_for_publish(
|
2026-06-05 10:41:48 -07:00
|
|
|
in_diff_unpublished,
|
|
|
|
|
severity_threshold=severity_threshold,
|
|
|
|
|
cap=cap,
|
|
|
|
|
)
|
2026-07-17 12:10:54 -04:00
|
|
|
eligible_with_payload = [
|
|
|
|
|
(finding, payload)
|
2026-05-18 15:47:13 -07:00
|
|
|
for finding in eligible
|
|
|
|
|
if (payload := render_inline_comment_payload(finding)) is not None
|
|
|
|
|
]
|
2026-07-17 12:10:54 -04:00
|
|
|
finding_ids = [
|
|
|
|
|
finding["id"]
|
|
|
|
|
for finding, _payload in eligible_with_payload
|
|
|
|
|
if isinstance(finding.get("id"), str)
|
|
|
|
|
]
|
|
|
|
|
publication = {
|
|
|
|
|
"finding_ids": finding_ids,
|
|
|
|
|
"severity_threshold": severity_threshold,
|
|
|
|
|
"cap": cap,
|
|
|
|
|
}
|
2026-05-18 15:47:13 -07:00
|
|
|
|
2026-07-17 12:10:54 -04:00
|
|
|
await set_reviewer_thread_metadata(
|
|
|
|
|
thread_id,
|
|
|
|
|
last_reviewed_sha=head_sha,
|
|
|
|
|
extra={REVIEWER_EVAL_PUBLICATION_KEY: publication},
|
|
|
|
|
)
|
2026-05-18 15:47:13 -07:00
|
|
|
|
|
|
|
|
return {
|
|
|
|
|
"success": True,
|
|
|
|
|
"dry_run": True,
|
|
|
|
|
"review_id": None,
|
2026-07-17 12:10:54 -04:00
|
|
|
"surfaced_count": len(eligible_with_payload),
|
|
|
|
|
"hidden_count": max(len(open_unpublished) - len(eligible_with_payload), 0),
|
2026-05-18 15:47:13 -07:00
|
|
|
"resolved_thread_count": 0,
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
async def _publish_review_async(
|
|
|
|
|
*,
|
|
|
|
|
owner: str,
|
|
|
|
|
repo: str,
|
|
|
|
|
pr_number: int,
|
|
|
|
|
head_sha: str,
|
|
|
|
|
token: str,
|
|
|
|
|
severity_threshold: Severity,
|
|
|
|
|
cap: int,
|
2026-05-07 16:11:36 -07:00
|
|
|
is_re_review: bool,
|
2026-05-26 16:24:34 -07:00
|
|
|
langgraph_run_id: str | None = None,
|
2026-05-28 13:48:14 -07:00
|
|
|
trace_link_config_override: object = None,
|
2026-07-20 15:28:00 -04:00
|
|
|
verdict: str | None = None,
|
|
|
|
|
verdict_requester: str = "",
|
2026-07-31 13:02:07 -04:00
|
|
|
verdict_authorization: str = "requested",
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
) -> dict[str, Any]:
|
|
|
|
|
thread_id = get_thread_id_from_runtime()
|
2026-07-20 15:28:00 -04:00
|
|
|
verdict_ignored_reason: str | None = None
|
2026-07-31 13:02:07 -04:00
|
|
|
verdict_attempted = verdict is not None
|
2026-07-20 15:28:00 -04:00
|
|
|
reviewed_head_sha = head_sha
|
fix: reviewer publishes against stale head_sha on mid-run re-review (#1393)
* fix: resolve reviewer head_sha from thread metadata, not frozen run config
A push that lands while a reviewer run is in flight is delivered as a
queued message into that run. The run's configurable is frozen at
creation, so its head_sha still names the commit the run was created for
— not the commit just pushed. publish_review then anchored the GitHub
review to the stale commit and regressed last_reviewed_sha to it, and
add_finding/update_finding stamped findings with the stale SHA.
Persist the current head in thread metadata at every reviewer dispatch
(both the ready-for-review and push paths, before they branch to create
a run or queue a message), and add resolve_review_head_sha() which
prefers the metadata head over the run config. Wire it into
publish_review (review commit_id + last_reviewed_sha), add_finding
(first_seen_sha) and update_finding (last_confirmed_sha). Falls back to
the run config when metadata carries no head (first review, eval, tests).
* fix: persist head_sha in manual review dispatch (trigger_pr_review_from_ref)
resolve_review_head_sha prefers metadata[head_sha] over the run config,
and the push/ready dispatchers write it — but trigger_pr_review_from_ref
(Slack/GitHub @open-swe review, request_pr_review tool) created a run
with a freshly-fetched config head while leaving metadata's head stale
from a prior dispatch. A manual re-review at a newer commit would then
resolve to the old head and publish/advance findings against it.
Persist head_sha in that dispatch's metadata write too, so every
run-creating reviewer dispatch keeps metadata in sync with the head its
run targets. Caught by the Open SWE reviewer on this PR.
2026-06-03 11:38:56 -07:00
|
|
|
# The run config's head_sha is frozen at run creation; a push that arrived
|
|
|
|
|
# mid-run updated the live head in thread metadata. Prefer that so the
|
|
|
|
|
# review anchors to (and last_reviewed_sha advances to) the commit actually
|
|
|
|
|
# reviewed, not the stale one this run was created for.
|
|
|
|
|
head_sha = await resolve_review_head_sha(thread_id, {"head_sha": head_sha})
|
2026-07-20 15:28:00 -04:00
|
|
|
|
2026-05-27 10:52:30 -07:00
|
|
|
findings = await _backfill_findings_from_pr_threads(
|
|
|
|
|
thread_id=thread_id,
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
token=token,
|
|
|
|
|
)
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
|
2026-07-21 20:18:25 -04:00
|
|
|
review_trace_url = await _resolve_review_trace_url(thread_id, trace_link_config_override)
|
|
|
|
|
review_ui_url = dashboard_review_url(owner, repo, pr_number)
|
|
|
|
|
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
# Re-reviews only post NEW findings. Anything with a github_review_comment_id
|
|
|
|
|
# already lives on GitHub from a prior publish — reposting would create
|
|
|
|
|
# duplicate inline comments and break the resolve-on-fix flow (only
|
|
|
|
|
# whichever duplicate id we'd cache last would resolve later).
|
|
|
|
|
unpublished_findings = [
|
|
|
|
|
f for f in findings if not isinstance(f.get("github_review_comment_id"), int)
|
|
|
|
|
]
|
2026-05-27 10:52:30 -07:00
|
|
|
if is_re_review:
|
|
|
|
|
unpublished_findings = [
|
|
|
|
|
f for f in unpublished_findings if f.get("first_seen_sha") == head_sha
|
|
|
|
|
]
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
open_unpublished = [f for f in unpublished_findings if f.get("status", "open") == "open"]
|
2026-06-12 10:19:30 -07:00
|
|
|
# In-diff findings become inline comments. Out-of-diff findings are disabled:
|
|
|
|
|
# they are never surfaced on the PR (any legacy in-state ones are treated as
|
|
|
|
|
# hidden).
|
2026-06-05 10:41:48 -07:00
|
|
|
in_diff_unpublished = [f for f in unpublished_findings if f.get("in_diff", True)]
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
eligible = filter_findings_for_publish(
|
2026-06-05 10:41:48 -07:00
|
|
|
in_diff_unpublished, severity_threshold=severity_threshold, cap=cap
|
|
|
|
|
)
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
|
2026-06-18 11:15:42 -07:00
|
|
|
severity_rank = SEVERITY_ORDER[severity_threshold]
|
|
|
|
|
eligible_ids = {f.get("id") for f in eligible}
|
|
|
|
|
additional_findings_count = sum(
|
|
|
|
|
1
|
|
|
|
|
for f in in_diff_unpublished
|
|
|
|
|
if f.get("id") not in eligible_ids
|
|
|
|
|
and f.get("status", "open") == "open"
|
|
|
|
|
and SEVERITY_ORDER.get(f.get("severity", "low"), 0) < severity_rank
|
|
|
|
|
)
|
|
|
|
|
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
inline_comments: list[dict[str, Any]] = []
|
|
|
|
|
eligible_with_payload: list[tuple[dict[str, Any], dict[str, Any]]] = []
|
|
|
|
|
for finding in eligible:
|
|
|
|
|
payload = render_inline_comment_payload(finding)
|
|
|
|
|
if payload is None:
|
|
|
|
|
continue
|
|
|
|
|
inline_comments.append(payload)
|
|
|
|
|
eligible_with_payload.append((dict(finding), payload))
|
|
|
|
|
|
2026-06-03 10:30:13 -07:00
|
|
|
# With nothing new to surface, skip the "no issues found" summary if Open
|
|
|
|
|
# SWE has already reviewed this PR — the user already saw the previous
|
|
|
|
|
# result, and posting another summary on every push is noise. We can't rely
|
|
|
|
|
# on the static re_review flag alone: a push that lands mid-run is delivered
|
|
|
|
|
# as a queued message into the still-running first-review run, whose
|
|
|
|
|
# configurable still says re_review=False, so that path would post a
|
|
|
|
|
# duplicate "No issues found". Key off the actual PR state (an existing Open
|
|
|
|
|
# SWE review summary) instead. Still resolve threads for findings that just
|
|
|
|
|
# moved to resolved, and advance last_reviewed_sha so subsequent pushes
|
|
|
|
|
# don't redo the same diff.
|
2026-07-20 15:28:00 -04:00
|
|
|
# A pending verdict must always reach GitHub — "approve if clean" with zero
|
|
|
|
|
# findings still posts an APPROVE review — so the skip only applies to
|
|
|
|
|
# plain comment publishes.
|
|
|
|
|
if (
|
|
|
|
|
not inline_comments
|
2026-07-31 13:02:07 -04:00
|
|
|
and not verdict_attempted
|
2026-07-20 15:28:00 -04:00
|
|
|
and await _open_swe_already_reviewed(
|
|
|
|
|
thread_id=thread_id,
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
token=token,
|
|
|
|
|
is_re_review=is_re_review,
|
|
|
|
|
)
|
2026-06-03 10:30:13 -07:00
|
|
|
):
|
2026-05-22 13:36:14 -07:00
|
|
|
resolved_thread_count = await _resolve_threads_for_resolved_findings(
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
token=token,
|
|
|
|
|
findings=findings,
|
|
|
|
|
)
|
|
|
|
|
await set_reviewer_thread_metadata(thread_id, last_reviewed_sha=head_sha)
|
2026-06-07 01:17:09 -04:00
|
|
|
await clear_review_started_comment(thread_id=thread_id, owner=owner, repo=repo, token=token)
|
2026-07-20 15:28:00 -04:00
|
|
|
skip_result: dict[str, Any] = {
|
2026-05-22 13:36:14 -07:00
|
|
|
"success": True,
|
|
|
|
|
"review_id": None,
|
|
|
|
|
"surfaced_count": 0,
|
|
|
|
|
"hidden_count": max(len(open_unpublished), 0),
|
|
|
|
|
"resolved_thread_count": resolved_thread_count,
|
|
|
|
|
"skipped_empty_re_review": True,
|
2026-07-31 13:24:32 -04:00
|
|
|
"blocking_finding_count": sum(
|
|
|
|
|
1 for finding in findings if _finding_blocks_verdict(finding)
|
|
|
|
|
),
|
|
|
|
|
"verdict_authorization": verdict_authorization,
|
2026-05-22 13:36:14 -07:00
|
|
|
}
|
2026-07-31 13:24:32 -04:00
|
|
|
conclusion, check_title, check_summary = review_check_conclusion(skip_result)
|
|
|
|
|
await settle_review_check_run(
|
|
|
|
|
thread_id=thread_id,
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
token=token,
|
|
|
|
|
conclusion=conclusion,
|
|
|
|
|
title=check_title,
|
|
|
|
|
summary=check_summary,
|
|
|
|
|
)
|
2026-07-20 15:28:00 -04:00
|
|
|
return skip_result
|
|
|
|
|
|
2026-07-31 13:02:07 -04:00
|
|
|
review_body = render_review_body(
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
surfaced_count=len(inline_comments),
|
|
|
|
|
trace_url=review_trace_url,
|
|
|
|
|
ui_url=review_ui_url,
|
|
|
|
|
additional_findings_count=additional_findings_count,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
)
|
2026-07-31 13:02:07 -04:00
|
|
|
review_response, event, verdict_ignored_reason, findings = await _post_review_guarded(
|
|
|
|
|
thread_id=thread_id,
|
2026-05-07 15:16:20 -07:00
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
head_sha=head_sha,
|
|
|
|
|
body=review_body,
|
|
|
|
|
inline_comments=inline_comments,
|
|
|
|
|
token=token,
|
2026-07-31 13:02:07 -04:00
|
|
|
verdict=verdict,
|
|
|
|
|
verdict_authorization=verdict_authorization,
|
|
|
|
|
verdict_requester=verdict_requester,
|
|
|
|
|
reviewed_head_sha=reviewed_head_sha,
|
2026-05-07 15:16:20 -07:00
|
|
|
)
|
fix: publish_review HTTP 422 "Path/Line could not be resolved" — agent retries with identical args instead of dropping unresolvable findings (#1338)
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-05-26 18:37:56 -07:00
|
|
|
# If GitHub rejected the batch because one or more inline comments anchor
|
|
|
|
|
# to a file/line that's not in the PR diff, drop just those findings and
|
|
|
|
|
# retry once. Returning the bare 422 to the agent only invites it to
|
|
|
|
|
# retry publish_review with byte-identical args until findings drain.
|
|
|
|
|
unresolvable_findings: list[str] = []
|
|
|
|
|
if (
|
|
|
|
|
isinstance(review_response, dict)
|
|
|
|
|
and review_response.get("_error_kind") == "unresolved_anchor"
|
|
|
|
|
):
|
|
|
|
|
valid_with_payload, dropped_ids = await _filter_against_pr_diff(
|
|
|
|
|
eligible_with_payload,
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
token=token,
|
|
|
|
|
)
|
|
|
|
|
if dropped_ids and valid_with_payload:
|
|
|
|
|
retry_inline = [p for _, p in valid_with_payload]
|
2026-07-31 13:02:07 -04:00
|
|
|
retry_body = render_review_body(
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
surfaced_count=len(retry_inline),
|
|
|
|
|
trace_url=review_trace_url,
|
|
|
|
|
ui_url=review_ui_url,
|
|
|
|
|
additional_findings_count=additional_findings_count,
|
2026-05-28 13:48:14 -07:00
|
|
|
)
|
2026-07-31 13:02:07 -04:00
|
|
|
retry_response, event, verdict_ignored_reason, findings = await _post_review_guarded(
|
|
|
|
|
thread_id=thread_id,
|
fix: publish_review HTTP 422 "Path/Line could not be resolved" — agent retries with identical args instead of dropping unresolvable findings (#1338)
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-05-26 18:37:56 -07:00
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
head_sha=head_sha,
|
|
|
|
|
body=retry_body,
|
|
|
|
|
inline_comments=retry_inline,
|
|
|
|
|
token=token,
|
2026-07-31 13:02:07 -04:00
|
|
|
verdict=verdict,
|
|
|
|
|
verdict_authorization=verdict_authorization,
|
|
|
|
|
verdict_requester=verdict_requester,
|
|
|
|
|
reviewed_head_sha=reviewed_head_sha,
|
fix: publish_review HTTP 422 "Path/Line could not be resolved" — agent retries with identical args instead of dropping unresolvable findings (#1338)
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-05-26 18:37:56 -07:00
|
|
|
)
|
|
|
|
|
if isinstance(retry_response, dict) and "_error" not in retry_response:
|
|
|
|
|
review_response = retry_response
|
2026-07-31 13:02:07 -04:00
|
|
|
review_body = retry_body
|
fix: publish_review HTTP 422 "Path/Line could not be resolved" — agent retries with identical args instead of dropping unresolvable findings (#1338)
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-05-26 18:37:56 -07:00
|
|
|
inline_comments = retry_inline
|
|
|
|
|
eligible_with_payload = valid_with_payload
|
|
|
|
|
unresolvable_findings = dropped_ids
|
|
|
|
|
else:
|
|
|
|
|
retry_error = (
|
|
|
|
|
retry_response.get("_error", "unknown error")
|
|
|
|
|
if isinstance(retry_response, dict)
|
|
|
|
|
else "no response"
|
|
|
|
|
)
|
|
|
|
|
return {
|
|
|
|
|
"success": False,
|
|
|
|
|
"error": f"Failed to POST PR review: {retry_error}",
|
|
|
|
|
"unresolvable_findings": dropped_ids,
|
|
|
|
|
"hint": (
|
|
|
|
|
"Call update_finding(status='resolved') on these ids "
|
|
|
|
|
"or fix their file/line before retrying."
|
|
|
|
|
),
|
|
|
|
|
}
|
2026-07-20 15:28:00 -04:00
|
|
|
elif event != "COMMENT":
|
|
|
|
|
# A verdict is pending but every inline comment anchors outside the
|
|
|
|
|
# diff. GitHub accepts a bodied review with zero inline comments, so
|
|
|
|
|
# post the authorized verdict rather than dropping it — the verdict
|
|
|
|
|
# must land even when the findings can't be anchored.
|
2026-07-31 13:02:07 -04:00
|
|
|
verdict_only_body = render_review_body(
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
surfaced_count=0,
|
|
|
|
|
trace_url=review_trace_url,
|
|
|
|
|
ui_url=review_ui_url,
|
|
|
|
|
additional_findings_count=additional_findings_count,
|
2026-07-20 15:28:00 -04:00
|
|
|
)
|
2026-07-31 13:02:07 -04:00
|
|
|
(
|
|
|
|
|
verdict_only_response,
|
|
|
|
|
event,
|
|
|
|
|
verdict_ignored_reason,
|
|
|
|
|
findings,
|
|
|
|
|
) = await _post_review_guarded(
|
|
|
|
|
thread_id=thread_id,
|
2026-07-20 15:28:00 -04:00
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
head_sha=head_sha,
|
|
|
|
|
body=verdict_only_body,
|
|
|
|
|
inline_comments=[],
|
|
|
|
|
token=token,
|
2026-07-31 13:02:07 -04:00
|
|
|
verdict=verdict,
|
|
|
|
|
verdict_authorization=verdict_authorization,
|
|
|
|
|
verdict_requester=verdict_requester,
|
|
|
|
|
reviewed_head_sha=reviewed_head_sha,
|
2026-07-20 15:28:00 -04:00
|
|
|
)
|
|
|
|
|
if isinstance(verdict_only_response, dict) and "_error" not in verdict_only_response:
|
|
|
|
|
review_response = verdict_only_response
|
2026-07-31 13:02:07 -04:00
|
|
|
review_body = verdict_only_body
|
2026-07-20 15:28:00 -04:00
|
|
|
inline_comments = []
|
|
|
|
|
eligible_with_payload = []
|
|
|
|
|
unresolvable_findings = dropped_ids
|
|
|
|
|
else:
|
|
|
|
|
verdict_error = (
|
|
|
|
|
verdict_only_response.get("_error", "unknown error")
|
|
|
|
|
if isinstance(verdict_only_response, dict)
|
|
|
|
|
else "no response"
|
|
|
|
|
)
|
|
|
|
|
return {
|
|
|
|
|
"success": False,
|
|
|
|
|
"error": f"Failed to POST PR review: {verdict_error}",
|
|
|
|
|
"unresolvable_findings": dropped_ids,
|
|
|
|
|
}
|
fix: publish_review HTTP 422 "Path/Line could not be resolved" — agent retries with identical args instead of dropping unresolvable findings (#1338)
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-05-26 18:37:56 -07:00
|
|
|
else:
|
|
|
|
|
# Either nothing to drop (no diff_line_set available, so we can't
|
|
|
|
|
# tell which findings are bad) or everything would be dropped.
|
|
|
|
|
# Either way, do not retry — surface the structural signal so the
|
|
|
|
|
# agent stops retrying with the same args.
|
|
|
|
|
return {
|
|
|
|
|
"success": False,
|
|
|
|
|
"error": f"Failed to POST PR review: {review_response['_error']}",
|
|
|
|
|
"unresolvable_findings": dropped_ids,
|
|
|
|
|
"hint": (
|
|
|
|
|
"Call update_finding(status='resolved') on these ids "
|
|
|
|
|
"or fix their file/line before retrying."
|
|
|
|
|
),
|
|
|
|
|
}
|
2026-05-08 15:40:05 -07:00
|
|
|
if isinstance(review_response, dict) and "_error" in review_response:
|
|
|
|
|
return {
|
|
|
|
|
"success": False,
|
|
|
|
|
"error": f"Failed to POST PR review: {review_response['_error']}",
|
|
|
|
|
}
|
2026-05-12 13:55:39 -07:00
|
|
|
if review_response is None:
|
|
|
|
|
# Defensive guard: with the upstream change this should never happen,
|
|
|
|
|
# but keep a clear signal if it does so the agent doesn't retry blindly.
|
|
|
|
|
return {
|
|
|
|
|
"success": False,
|
|
|
|
|
"error": "Failed to POST PR review: no response from GitHub",
|
|
|
|
|
}
|
2026-05-07 15:16:20 -07:00
|
|
|
review_id = review_response.get("id") if isinstance(review_response, dict) else None
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
|
2026-07-31 13:02:07 -04:00
|
|
|
# Trust GitHub's recorded review state, not the event we asked for.
|
2026-07-20 15:28:00 -04:00
|
|
|
returned_state = review_response.get("state") if isinstance(review_response, dict) else None
|
2026-07-31 13:02:07 -04:00
|
|
|
recorded_state = returned_state.upper() if isinstance(returned_state, str) else ""
|
2026-07-20 15:28:00 -04:00
|
|
|
expected_state = _EVENT_TO_STATE.get(event)
|
2026-07-31 13:02:07 -04:00
|
|
|
verdict_submitted = bool(
|
|
|
|
|
verdict_attempted
|
|
|
|
|
and expected_state
|
|
|
|
|
and recorded_state == expected_state
|
|
|
|
|
and review_id is not None
|
|
|
|
|
)
|
|
|
|
|
if verdict_attempted and not verdict_submitted and verdict_ignored_reason is None:
|
|
|
|
|
verdict_ignored_reason = "github_state_mismatch"
|
2026-07-31 13:12:25 -04:00
|
|
|
body_update_failed = False
|
2026-07-31 13:17:35 -04:00
|
|
|
if verdict_submitted and isinstance(review_id, int) and recorded_state:
|
2026-07-31 13:12:25 -04:00
|
|
|
body_updated = await update_pull_request_review_body(
|
2026-07-31 13:02:07 -04:00
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
review_id=review_id,
|
|
|
|
|
body=_decorate_recorded_review_body(
|
|
|
|
|
review_body,
|
|
|
|
|
recorded_state=recorded_state,
|
|
|
|
|
verdict=verdict,
|
|
|
|
|
verdict_requester=verdict_requester,
|
2026-07-31 13:12:25 -04:00
|
|
|
verdict_authorization=verdict_authorization,
|
2026-07-31 13:02:07 -04:00
|
|
|
verdict_submitted=verdict_submitted,
|
|
|
|
|
verdict_ignored_reason=verdict_ignored_reason,
|
|
|
|
|
),
|
|
|
|
|
token=token,
|
|
|
|
|
)
|
2026-07-31 13:12:25 -04:00
|
|
|
body_update_failed = body_updated is not True
|
2026-07-20 15:28:00 -04:00
|
|
|
await _reconcile_last_verdict(
|
|
|
|
|
thread_id=thread_id,
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
token=token,
|
|
|
|
|
event=event,
|
|
|
|
|
review_id=review_id if isinstance(review_id, int) else None,
|
|
|
|
|
head_sha=head_sha,
|
|
|
|
|
surfaced_count=len(inline_comments),
|
|
|
|
|
verdict_submitted=verdict_submitted,
|
|
|
|
|
)
|
|
|
|
|
|
2026-06-12 10:19:30 -07:00
|
|
|
if review_id is not None and inline_comments:
|
2026-06-10 10:17:31 -07:00
|
|
|
# Record the GitHub review id AND inline comment ids in a single
|
|
|
|
|
# findings write. Previously these were three separate read-replace
|
|
|
|
|
# cycles (out-of-diff review id, inline review id, comment ids); each
|
|
|
|
|
# extra write widened the window where a crash could leave findings
|
|
|
|
|
# half-stamped — surfaced on GitHub but with no recorded comment id, so
|
|
|
|
|
# a later resolve-on-fix couldn't find the thread.
|
|
|
|
|
comment_records: list[dict[str, Any]] = []
|
|
|
|
|
if inline_comments:
|
|
|
|
|
comment_records = await fetch_review_comments(
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
review_id=review_id,
|
|
|
|
|
token=token,
|
|
|
|
|
)
|
|
|
|
|
if langgraph_run_id is None:
|
|
|
|
|
metadata = await get_thread_metadata(thread_id)
|
|
|
|
|
current_run_id = metadata.get("current_reviewer_run_id")
|
|
|
|
|
if isinstance(current_run_id, str) and current_run_id:
|
|
|
|
|
langgraph_run_id = current_run_id
|
|
|
|
|
await _record_review_publication(
|
2026-05-27 10:52:30 -07:00
|
|
|
thread_id=thread_id,
|
|
|
|
|
review_id=review_id,
|
2026-06-10 10:17:31 -07:00
|
|
|
inline_with_payload=eligible_with_payload,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
comment_records=comment_records,
|
2026-05-26 16:24:34 -07:00
|
|
|
langgraph_run_id=langgraph_run_id,
|
|
|
|
|
)
|
2026-06-10 10:17:31 -07:00
|
|
|
|
|
|
|
|
if review_id is not None and inline_comments:
|
2026-05-27 10:52:30 -07:00
|
|
|
current_findings = await list_findings_async(thread_id)
|
|
|
|
|
if _missing_comment_ids_for_published_findings(current_findings, eligible_with_payload):
|
|
|
|
|
await _backfill_findings_from_pr_threads(
|
|
|
|
|
thread_id=thread_id,
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
token=token,
|
|
|
|
|
)
|
2026-05-26 16:24:34 -07:00
|
|
|
await _store_thread_ids_on_findings(
|
|
|
|
|
thread_id=thread_id,
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
token=token,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
resolved_thread_count = await _resolve_threads_for_resolved_findings(
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
token=token,
|
|
|
|
|
findings=await list_findings_async(thread_id),
|
|
|
|
|
)
|
|
|
|
|
|
2026-05-07 16:11:36 -07:00
|
|
|
if not is_re_review:
|
|
|
|
|
await _maybe_post_slack_completion_reply(
|
|
|
|
|
thread_id=thread_id,
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
review_id=review_id,
|
|
|
|
|
surfaced_count=len(inline_comments),
|
|
|
|
|
)
|
|
|
|
|
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
await set_reviewer_thread_metadata(thread_id, last_reviewed_sha=head_sha)
|
2026-06-07 01:17:09 -04:00
|
|
|
await clear_review_started_comment(thread_id=thread_id, owner=owner, repo=repo, token=token)
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
|
fix: publish_review HTTP 422 "Path/Line could not be resolved" — agent retries with identical args instead of dropping unresolvable findings (#1338)
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-05-26 18:37:56 -07:00
|
|
|
result: dict[str, Any] = {
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
"success": True,
|
|
|
|
|
"review_id": review_id,
|
|
|
|
|
"surfaced_count": len(inline_comments),
|
2026-06-12 10:19:30 -07:00
|
|
|
"hidden_count": max(len(open_unpublished) - len(inline_comments), 0),
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
"resolved_thread_count": resolved_thread_count,
|
2026-07-31 13:24:32 -04:00
|
|
|
"blocking_finding_count": sum(
|
|
|
|
|
1 for finding in findings if _finding_blocks_verdict(finding)
|
|
|
|
|
),
|
|
|
|
|
"verdict_authorization": verdict_authorization,
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
}
|
2026-07-31 13:02:07 -04:00
|
|
|
if recorded_state:
|
|
|
|
|
result["review_state"] = recorded_state
|
2026-07-20 15:28:00 -04:00
|
|
|
if verdict_submitted:
|
|
|
|
|
result["verdict_submitted"] = True
|
|
|
|
|
result["verdict_event"] = event
|
|
|
|
|
elif verdict_ignored_reason:
|
|
|
|
|
result["verdict_submitted"] = False
|
|
|
|
|
result["verdict_ignored"] = True
|
|
|
|
|
result["verdict_ignored_reason"] = verdict_ignored_reason
|
2026-07-31 13:12:25 -04:00
|
|
|
if body_update_failed:
|
|
|
|
|
result["body_update_failed"] = True
|
|
|
|
|
result["body_update_message"] = (
|
|
|
|
|
"GitHub recorded the review state, but updating the review body failed. "
|
|
|
|
|
"The original body remains neutral and does not claim a verdict."
|
|
|
|
|
)
|
fix: publish_review HTTP 422 "Path/Line could not be resolved" — agent retries with identical args instead of dropping unresolvable findings (#1338)
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-05-26 18:37:56 -07:00
|
|
|
if unresolvable_findings:
|
|
|
|
|
result["unresolvable_findings"] = unresolvable_findings
|
|
|
|
|
result["hint"] = (
|
|
|
|
|
"Some findings had anchors not in the PR diff; "
|
|
|
|
|
"call update_finding to fix or resolve them."
|
|
|
|
|
)
|
2026-07-31 13:24:32 -04:00
|
|
|
conclusion, check_title, check_summary = review_check_conclusion(result)
|
|
|
|
|
await settle_review_check_run(
|
|
|
|
|
thread_id=thread_id,
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
token=token,
|
|
|
|
|
conclusion=conclusion,
|
|
|
|
|
title=check_title,
|
|
|
|
|
summary=check_summary,
|
|
|
|
|
)
|
fix: publish_review HTTP 422 "Path/Line could not be resolved" — agent retries with identical args instead of dropping unresolvable findings (#1338)
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-05-26 18:37:56 -07:00
|
|
|
return result
|
|
|
|
|
|
|
|
|
|
|
2026-07-31 13:02:07 -04:00
|
|
|
def _finding_blocks_verdict(finding: Finding) -> bool:
|
|
|
|
|
status = finding.get("status", "open")
|
|
|
|
|
if status in {"open", "needs_reassessment"}:
|
|
|
|
|
return True
|
|
|
|
|
interactions = finding.get("interactions")
|
|
|
|
|
if not isinstance(interactions, list) or not interactions:
|
|
|
|
|
return False
|
|
|
|
|
latest = interactions[-1]
|
|
|
|
|
return isinstance(latest, dict) and latest.get("needs_reassessment") is True
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def _post_review_guarded(
|
|
|
|
|
*,
|
|
|
|
|
thread_id: str,
|
|
|
|
|
owner: str,
|
|
|
|
|
repo: str,
|
|
|
|
|
pr_number: int,
|
|
|
|
|
head_sha: str,
|
|
|
|
|
body: str,
|
|
|
|
|
inline_comments: list[dict[str, Any]],
|
|
|
|
|
token: str,
|
|
|
|
|
verdict: str | None,
|
|
|
|
|
verdict_authorization: str,
|
|
|
|
|
verdict_requester: str,
|
|
|
|
|
reviewed_head_sha: str,
|
|
|
|
|
) -> tuple[dict[str, Any] | None, str, str | None, list[Finding]]:
|
|
|
|
|
if verdict is None:
|
|
|
|
|
response = await post_pull_request_review(
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
head_sha=head_sha,
|
|
|
|
|
body=body,
|
|
|
|
|
inline_comments=inline_comments,
|
|
|
|
|
token=token,
|
|
|
|
|
event="COMMENT",
|
|
|
|
|
)
|
|
|
|
|
return response, "COMMENT", None, await list_findings_async(thread_id)
|
|
|
|
|
|
|
|
|
|
async with _finding_mutation_lock(thread_id):
|
|
|
|
|
metadata = await _get_thread_metadata_strict(thread_id)
|
|
|
|
|
findings = _coerce_findings_list(metadata.get("findings"))
|
|
|
|
|
submitted_verdict = verdict
|
|
|
|
|
ignored_reason: str | None = None
|
|
|
|
|
post_head_sha = head_sha
|
|
|
|
|
|
|
|
|
|
if verdict_authorization == "none":
|
|
|
|
|
submitted_verdict = None
|
|
|
|
|
ignored_reason = "verdict_not_requested"
|
|
|
|
|
elif verdict_authorization == "consistent":
|
|
|
|
|
open_count = sum(1 for finding in findings if _finding_blocks_verdict(finding))
|
|
|
|
|
if verdict == "approve" and open_count:
|
|
|
|
|
submitted_verdict = None
|
|
|
|
|
ignored_reason = "approve_with_open_findings"
|
|
|
|
|
elif verdict == "request_changes" and not open_count:
|
|
|
|
|
submitted_verdict = None
|
|
|
|
|
ignored_reason = "request_changes_without_open_findings"
|
|
|
|
|
|
|
|
|
|
if submitted_verdict is not None:
|
|
|
|
|
pr_author = _pr_author_from_thread(metadata)
|
|
|
|
|
bot_logins = {login.casefold() for login in INTERNAL_BOT_LOGINS}
|
|
|
|
|
if not pr_author:
|
|
|
|
|
submitted_verdict = None
|
|
|
|
|
ignored_reason = "author_unknown"
|
|
|
|
|
elif pr_author.casefold() in bot_logins:
|
|
|
|
|
submitted_verdict = None
|
|
|
|
|
ignored_reason = "self_review"
|
|
|
|
|
|
|
|
|
|
if submitted_verdict is not None and head_sha != reviewed_head_sha:
|
|
|
|
|
submitted_verdict = None
|
|
|
|
|
ignored_reason = "head_moved"
|
|
|
|
|
|
|
|
|
|
if submitted_verdict is not None:
|
|
|
|
|
live_head_sha = await fetch_pull_request_head_sha(
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
token=token,
|
|
|
|
|
)
|
2026-07-31 13:12:25 -04:00
|
|
|
if live_head_sha is None:
|
|
|
|
|
submitted_verdict = None
|
|
|
|
|
ignored_reason = "head_check_failed"
|
|
|
|
|
elif live_head_sha != reviewed_head_sha:
|
2026-07-31 13:02:07 -04:00
|
|
|
submitted_verdict = None
|
|
|
|
|
ignored_reason = "head_moved"
|
|
|
|
|
post_head_sha = live_head_sha
|
|
|
|
|
|
|
|
|
|
event = _VERDICT_EVENTS.get(submitted_verdict or "", "COMMENT")
|
|
|
|
|
decorated_body = _decorate_review_body(
|
|
|
|
|
body,
|
|
|
|
|
verdict=submitted_verdict,
|
2026-07-31 13:12:25 -04:00
|
|
|
verdict_authorization=verdict_authorization,
|
2026-07-31 13:02:07 -04:00
|
|
|
verdict_requester=verdict_requester,
|
|
|
|
|
verdict_ignored_reason=ignored_reason,
|
|
|
|
|
)
|
|
|
|
|
response = await post_pull_request_review(
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
head_sha=post_head_sha,
|
|
|
|
|
body=decorated_body,
|
|
|
|
|
inline_comments=inline_comments,
|
|
|
|
|
token=token,
|
|
|
|
|
event=event,
|
|
|
|
|
)
|
|
|
|
|
return response, event, ignored_reason, findings
|
|
|
|
|
|
|
|
|
|
|
2026-06-10 10:17:31 -07:00
|
|
|
async def _open_swe_already_reviewed(
|
|
|
|
|
*,
|
|
|
|
|
thread_id: str,
|
|
|
|
|
owner: str,
|
|
|
|
|
repo: str,
|
|
|
|
|
pr_number: int,
|
|
|
|
|
token: str,
|
|
|
|
|
is_re_review: bool,
|
|
|
|
|
) -> bool:
|
|
|
|
|
"""Decide whether to suppress a duplicate empty "no issues found" summary.
|
|
|
|
|
|
|
|
|
|
Suppress only when we are *certain* a prior Open SWE review exists, so a
|
|
|
|
|
transient GitHub failure never causes a double-post:
|
|
|
|
|
|
|
|
|
|
- ``is_re_review`` is a durable signal (the dispatching webhook set it from
|
|
|
|
|
the persisted ``last_reviewed_sha``), so trust it outright.
|
|
|
|
|
- Otherwise consult durable reviewer state (``last_reviewed_sha`` on thread
|
|
|
|
|
metadata): a non-empty value means this thread already published once.
|
|
|
|
|
- Only as a last resort hit the GitHub reviews API. That call is tri-state:
|
|
|
|
|
``True``/``False`` are authoritative, but ``None`` means "unknown"
|
|
|
|
|
(pagination or the request failed). On ``None`` we do NOT suppress — a
|
|
|
|
|
possible duplicate summary is better than silently swallowing the only
|
|
|
|
|
review the user will ever see, and re-posting is the safe failure mode.
|
|
|
|
|
"""
|
|
|
|
|
if is_re_review:
|
|
|
|
|
return True
|
|
|
|
|
metadata = await get_thread_metadata(thread_id)
|
|
|
|
|
if get_thread_last_reviewed_sha(metadata):
|
|
|
|
|
return True
|
|
|
|
|
exists = await open_swe_review_exists(owner=owner, repo=repo, pr_number=pr_number, token=token)
|
|
|
|
|
return exists is True
|
|
|
|
|
|
|
|
|
|
|
2026-07-20 15:28:00 -04:00
|
|
|
def _pr_author_from_thread(metadata: dict[str, Any]) -> str:
|
|
|
|
|
pr_meta = get_thread_pr_meta(metadata)
|
|
|
|
|
if pr_meta is None:
|
|
|
|
|
return ""
|
|
|
|
|
author = pr_meta.get("author")
|
|
|
|
|
return author if isinstance(author, str) else ""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _decorate_review_body(
|
|
|
|
|
body: str,
|
|
|
|
|
*,
|
|
|
|
|
verdict: str | None,
|
2026-07-31 13:12:25 -04:00
|
|
|
verdict_authorization: str,
|
2026-07-20 15:28:00 -04:00
|
|
|
verdict_requester: str,
|
|
|
|
|
verdict_ignored_reason: str | None,
|
|
|
|
|
) -> str:
|
|
|
|
|
"""Append verdict attribution / downgrade context to the review body."""
|
|
|
|
|
if verdict is not None:
|
2026-07-31 13:12:25 -04:00
|
|
|
if verdict_authorization == "consistent":
|
|
|
|
|
return (
|
|
|
|
|
f"{body}\n\nAutomatic verdict evaluation (`{verdict}`) pending "
|
|
|
|
|
"based on authoritative finding state."
|
|
|
|
|
)
|
|
|
|
|
requester = f"@{verdict_requester}" if verdict_requester else "an explicit human request"
|
|
|
|
|
return f"{body}\n\nVerdict (`{verdict}`) pending for {requester}."
|
2026-07-31 13:02:07 -04:00
|
|
|
if verdict_ignored_reason:
|
|
|
|
|
reason = verdict_ignored_reason.replace("_", " ")
|
|
|
|
|
return f"{body}\n\n> Verdict withheld ({reason}). Published as a comment review."
|
|
|
|
|
return body
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _decorate_recorded_review_body(
|
|
|
|
|
body: str,
|
|
|
|
|
*,
|
|
|
|
|
recorded_state: str,
|
|
|
|
|
verdict: str | None,
|
|
|
|
|
verdict_requester: str,
|
2026-07-31 13:12:25 -04:00
|
|
|
verdict_authorization: str,
|
2026-07-31 13:02:07 -04:00
|
|
|
verdict_submitted: bool,
|
|
|
|
|
verdict_ignored_reason: str | None,
|
|
|
|
|
) -> str:
|
|
|
|
|
if verdict_submitted and verdict is not None:
|
2026-07-31 13:12:25 -04:00
|
|
|
if verdict_authorization == "consistent":
|
|
|
|
|
return (
|
|
|
|
|
f"{body}\n\nReview outcome: **{recorded_state}**. "
|
|
|
|
|
f"Automatic verdict (`{verdict}`) recorded based on authoritative finding state."
|
|
|
|
|
)
|
|
|
|
|
requester = f"@{verdict_requester}" if verdict_requester else "an explicit human request"
|
2026-07-20 15:28:00 -04:00
|
|
|
return (
|
2026-07-31 13:02:07 -04:00
|
|
|
f"{body}\n\nReview outcome: **{recorded_state}**. "
|
|
|
|
|
f"Verdict (`{verdict}`) recorded for {requester}."
|
2026-07-20 15:28:00 -04:00
|
|
|
)
|
2026-07-31 13:02:07 -04:00
|
|
|
reason = (verdict_ignored_reason or "github_state_mismatch").replace("_", " ")
|
2026-07-31 13:17:35 -04:00
|
|
|
if verdict_authorization == "consistent":
|
|
|
|
|
return (
|
|
|
|
|
f"{body}\n\nReview outcome: **{recorded_state}**. "
|
|
|
|
|
f"The automatic verdict evaluation was withheld ({reason})."
|
|
|
|
|
)
|
2026-07-31 13:02:07 -04:00
|
|
|
return (
|
|
|
|
|
f"{body}\n\nReview outcome: **{recorded_state}**. "
|
|
|
|
|
f"The requested verdict was withheld ({reason})."
|
|
|
|
|
)
|
2026-07-20 15:28:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
async def _reconcile_last_verdict(
|
|
|
|
|
*,
|
|
|
|
|
thread_id: str,
|
|
|
|
|
owner: str,
|
|
|
|
|
repo: str,
|
|
|
|
|
pr_number: int,
|
|
|
|
|
token: str,
|
|
|
|
|
event: str,
|
|
|
|
|
review_id: int | None,
|
|
|
|
|
head_sha: str,
|
|
|
|
|
surfaced_count: int,
|
|
|
|
|
verdict_submitted: bool,
|
|
|
|
|
) -> None:
|
|
|
|
|
"""Record a submitted verdict and dismiss a stale prior APPROVE.
|
|
|
|
|
|
|
|
|
|
A previously recorded APPROVE goes stale the moment a later publish
|
|
|
|
|
surfaces new findings without re-approving — leave it standing and the PR
|
|
|
|
|
keeps an approved state that no longer reflects the reviewer's opinion.
|
|
|
|
|
Entirely best-effort: failures are logged and never block the publish.
|
|
|
|
|
"""
|
|
|
|
|
if not verdict_submitted and surfaced_count == 0:
|
|
|
|
|
return
|
|
|
|
|
try:
|
|
|
|
|
metadata = await get_thread_metadata(thread_id)
|
|
|
|
|
last_verdict = metadata.get("last_verdict")
|
|
|
|
|
stale_approval_id: int | None = None
|
|
|
|
|
if (
|
|
|
|
|
isinstance(last_verdict, dict)
|
|
|
|
|
and last_verdict.get("event") == "APPROVE"
|
|
|
|
|
and isinstance(last_verdict.get("review_id"), int)
|
|
|
|
|
and event != "APPROVE"
|
|
|
|
|
and surfaced_count > 0
|
|
|
|
|
):
|
|
|
|
|
stale_approval_id = last_verdict["review_id"]
|
|
|
|
|
await dismiss_pull_request_review(
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
review_id=stale_approval_id,
|
|
|
|
|
message=(
|
|
|
|
|
"Dismissing stale approval: a later Open SWE review surfaced new "
|
|
|
|
|
"findings on this pull request."
|
|
|
|
|
),
|
|
|
|
|
token=token,
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
if verdict_submitted and review_id is not None:
|
|
|
|
|
await set_reviewer_thread_metadata(
|
|
|
|
|
thread_id,
|
|
|
|
|
extra={"last_verdict": {"event": event, "review_id": review_id, "sha": head_sha}},
|
|
|
|
|
)
|
|
|
|
|
elif stale_approval_id is not None:
|
|
|
|
|
await set_reviewer_thread_metadata(thread_id, extra={"last_verdict": None})
|
|
|
|
|
except Exception: # noqa: BLE001 — advisory bookkeeping must not fail the publish
|
|
|
|
|
logger.exception("Failed to reconcile last_verdict for %s/%s#%s", owner, repo, pr_number)
|
|
|
|
|
|
|
|
|
|
|
2026-05-27 10:52:30 -07:00
|
|
|
def _has_publication_identity(finding: Finding) -> bool:
|
|
|
|
|
return isinstance(finding.get("github_review_comment_id"), int) or isinstance(
|
|
|
|
|
finding.get("github_review_id"), int
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _int_list(value: Any) -> list[int]:
|
|
|
|
|
if not isinstance(value, list):
|
|
|
|
|
return []
|
|
|
|
|
return [item for item in value if isinstance(item, int)]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _str_list(value: Any) -> list[str]:
|
|
|
|
|
if not isinstance(value, list):
|
|
|
|
|
return []
|
|
|
|
|
return [item for item in value if isinstance(item, str) and item]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _comment_ids_for_finding(finding: dict[str, Any]) -> list[int]:
|
|
|
|
|
comment_ids = _int_list(finding.get("github_review_comment_ids"))
|
|
|
|
|
comment_id = finding.get("github_review_comment_id")
|
|
|
|
|
if isinstance(comment_id, int) and comment_id not in comment_ids:
|
|
|
|
|
comment_ids.insert(0, comment_id)
|
|
|
|
|
return comment_ids
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _thread_ids_for_finding(finding: dict[str, Any]) -> list[str]:
|
|
|
|
|
thread_ids = _str_list(finding.get("github_review_thread_ids"))
|
|
|
|
|
thread_id = finding.get("github_review_thread_id")
|
|
|
|
|
if isinstance(thread_id, str) and thread_id and thread_id not in thread_ids:
|
|
|
|
|
thread_ids.insert(0, thread_id)
|
|
|
|
|
return thread_ids
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def _backfill_findings_from_pr_threads(
|
|
|
|
|
*,
|
|
|
|
|
thread_id: str,
|
|
|
|
|
owner: str,
|
|
|
|
|
repo: str,
|
|
|
|
|
pr_number: int,
|
|
|
|
|
token: str,
|
|
|
|
|
) -> list[Finding]:
|
|
|
|
|
findings = await list_findings_async(thread_id)
|
|
|
|
|
if not findings:
|
|
|
|
|
return findings
|
|
|
|
|
review_threads = await fetch_pr_review_threads(
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
token=token,
|
|
|
|
|
)
|
|
|
|
|
if not review_threads:
|
|
|
|
|
return findings
|
|
|
|
|
return await reconcile_findings_with_review_threads(thread_id, review_threads)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _missing_comment_ids_for_published_findings(
|
|
|
|
|
findings: list[Finding],
|
|
|
|
|
eligible_with_payload: list[tuple[dict[str, Any], dict[str, Any]]],
|
|
|
|
|
) -> bool:
|
|
|
|
|
finding_ids = {
|
|
|
|
|
finding.get("id")
|
|
|
|
|
for finding, _payload in eligible_with_payload
|
|
|
|
|
if isinstance(finding.get("id"), str)
|
|
|
|
|
}
|
|
|
|
|
for finding in findings:
|
|
|
|
|
if finding.get("id") in finding_ids and not isinstance(
|
|
|
|
|
finding.get("github_review_comment_id"), int
|
|
|
|
|
):
|
|
|
|
|
return True
|
|
|
|
|
return False
|
|
|
|
|
|
|
|
|
|
|
2026-06-10 10:17:31 -07:00
|
|
|
def _apply_review_id(
|
2026-05-27 10:52:30 -07:00
|
|
|
findings: list[Finding],
|
2026-06-10 10:17:31 -07:00
|
|
|
*,
|
|
|
|
|
finding_ids: set[str],
|
2026-05-27 10:52:30 -07:00
|
|
|
review_id: int,
|
2026-06-10 10:17:31 -07:00
|
|
|
) -> bool:
|
2026-05-27 10:52:30 -07:00
|
|
|
updated = False
|
|
|
|
|
for finding in findings:
|
|
|
|
|
if finding.get("id") in finding_ids and finding.get("github_review_id") != review_id:
|
|
|
|
|
finding["github_review_id"] = review_id
|
2026-05-27 17:26:08 -07:00
|
|
|
if isinstance(finding.get("id"), str):
|
|
|
|
|
surface = _coerce_surface(finding, str(finding["id"]))
|
|
|
|
|
surface["github_review_id"] = review_id
|
|
|
|
|
finding["surface"] = surface
|
|
|
|
|
updated = True
|
2026-06-10 10:17:31 -07:00
|
|
|
return updated
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _apply_comment_ids(
|
|
|
|
|
findings: list[Finding],
|
|
|
|
|
*,
|
|
|
|
|
comment_id_by_finding_id: dict[str, int],
|
|
|
|
|
langgraph_run_id: str | None,
|
|
|
|
|
) -> bool:
|
|
|
|
|
updated = False
|
|
|
|
|
for finding in findings:
|
|
|
|
|
finding_id = finding.get("id")
|
|
|
|
|
if not isinstance(finding_id, str):
|
|
|
|
|
continue
|
|
|
|
|
comment_id = comment_id_by_finding_id.get(finding_id)
|
|
|
|
|
if comment_id is None:
|
|
|
|
|
continue
|
|
|
|
|
finding["github_review_comment_id"] = comment_id
|
|
|
|
|
comment_ids = _int_list(finding.get("github_review_comment_ids"))
|
|
|
|
|
if comment_id not in comment_ids:
|
|
|
|
|
comment_ids.append(comment_id)
|
|
|
|
|
finding["github_review_comment_ids"] = comment_ids
|
|
|
|
|
surface = _coerce_surface(finding, finding_id)
|
|
|
|
|
surface["state"] = "surfaced"
|
|
|
|
|
surface["github_review_comment_id"] = comment_id
|
|
|
|
|
surface["severity_threshold_at_publish"] = finding.get("severity")
|
|
|
|
|
surface["surfaced_at_sha"] = finding.get("last_confirmed_sha") or finding.get(
|
|
|
|
|
"first_seen_sha"
|
|
|
|
|
)
|
|
|
|
|
finding["surface"] = surface
|
|
|
|
|
if langgraph_run_id:
|
|
|
|
|
finding["github_review_run_id"] = langgraph_run_id
|
|
|
|
|
updated = True
|
|
|
|
|
return updated
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _comment_id_by_finding_id(
|
|
|
|
|
eligible_with_payload: list[tuple[dict[str, Any], dict[str, Any]]],
|
|
|
|
|
comment_records: list[dict[str, Any]],
|
|
|
|
|
) -> dict[str, int]:
|
|
|
|
|
"""Map each surfaced finding id to its GitHub comment id via the marker.
|
|
|
|
|
|
|
|
|
|
The embedded Open SWE marker is the *only* source of truth. Every comment
|
|
|
|
|
this reviewer posts carries a ``<!-- open-swe-review-comment {...} -->``
|
|
|
|
|
marker keyed by finding id (see ``render_inline_comment_body``), so the
|
|
|
|
|
match is exact. The old ``(path, line, body)`` fallback collided whenever
|
|
|
|
|
two findings shared a path/line/body — it cached the same comment id on
|
|
|
|
|
both, which corrupts resolve-on-fix (resolving one would target the wrong
|
|
|
|
|
thread). Findings whose comment lacks a parseable marker are left out here;
|
|
|
|
|
``_backfill_findings_from_pr_threads`` recovers them via the same marker
|
|
|
|
|
against the PR's review threads.
|
|
|
|
|
"""
|
|
|
|
|
by_marker_id: dict[str, int] = {}
|
|
|
|
|
for record in comment_records:
|
|
|
|
|
body = record.get("body", "")
|
|
|
|
|
comment_id = record.get("id")
|
|
|
|
|
if isinstance(body, str) and isinstance(comment_id, int):
|
|
|
|
|
marker = parse_review_comment_marker(body)
|
|
|
|
|
if marker is not None:
|
|
|
|
|
by_marker_id[marker["id"]] = comment_id
|
|
|
|
|
|
|
|
|
|
out: dict[str, int] = {}
|
|
|
|
|
for finding_snapshot, _payload in eligible_with_payload:
|
|
|
|
|
finding_id = finding_snapshot.get("id")
|
|
|
|
|
if isinstance(finding_id, str) and finding_id in by_marker_id:
|
|
|
|
|
out[finding_id] = by_marker_id[finding_id]
|
|
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def _record_review_publication(
|
|
|
|
|
*,
|
|
|
|
|
thread_id: str,
|
|
|
|
|
review_id: int,
|
|
|
|
|
inline_with_payload: list[tuple[dict[str, Any], dict[str, Any]]],
|
|
|
|
|
comment_records: list[dict[str, Any]],
|
|
|
|
|
langgraph_run_id: str | None,
|
|
|
|
|
) -> None:
|
|
|
|
|
"""Stamp the review id and inline comment ids onto findings in one write.
|
|
|
|
|
|
|
|
|
|
Collapsing the review-id and comment-id updates into a single
|
|
|
|
|
read-modify-write keeps publication identity atomic: a finding is never
|
|
|
|
|
persisted carrying a review id without also carrying whatever comment id
|
|
|
|
|
GitHub returned for it in the same record.
|
|
|
|
|
"""
|
|
|
|
|
review_finding_ids = {
|
|
|
|
|
finding.get("id")
|
|
|
|
|
for finding, _payload in inline_with_payload
|
|
|
|
|
if isinstance(finding.get("id"), str)
|
|
|
|
|
}
|
|
|
|
|
comment_id_by_finding_id = _comment_id_by_finding_id(inline_with_payload, comment_records)
|
|
|
|
|
|
|
|
|
|
latest = await list_findings_async(thread_id)
|
|
|
|
|
changed = _apply_review_id(
|
|
|
|
|
latest,
|
|
|
|
|
finding_ids={fid for fid in review_finding_ids if isinstance(fid, str)},
|
|
|
|
|
review_id=review_id,
|
|
|
|
|
)
|
|
|
|
|
changed = (
|
|
|
|
|
_apply_comment_ids(
|
|
|
|
|
latest,
|
|
|
|
|
comment_id_by_finding_id=comment_id_by_finding_id,
|
|
|
|
|
langgraph_run_id=langgraph_run_id,
|
|
|
|
|
)
|
|
|
|
|
or changed
|
|
|
|
|
)
|
|
|
|
|
if changed:
|
|
|
|
|
await replace_findings(thread_id, latest)
|
2026-05-27 10:52:30 -07:00
|
|
|
|
|
|
|
|
|
fix: publish_review HTTP 422 "Path/Line could not be resolved" — agent retries with identical args instead of dropping unresolvable findings (#1338)
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-05-26 18:37:56 -07:00
|
|
|
async def _resolve_diff_line_set(
|
|
|
|
|
*,
|
|
|
|
|
owner: str,
|
|
|
|
|
repo: str,
|
|
|
|
|
pr_number: int,
|
|
|
|
|
token: str,
|
|
|
|
|
) -> dict[str, set[int]] | None:
|
|
|
|
|
"""Return the new-side line set for the PR diff, fetching it if needed.
|
|
|
|
|
|
|
|
|
|
Reviewer runs clear ``configurable['diff_line_set']`` before the agent
|
|
|
|
|
starts (so ``add_finding`` trusts the agent's anchors), which means the
|
|
|
|
|
publish-time retry path can't rely on it being populated. Fetch the PR's
|
|
|
|
|
unified diff from the GitHub REST API and recompute the line set on the
|
|
|
|
|
fly. Returns ``None`` if the fetch fails — caller treats that as "we
|
|
|
|
|
can't tell which finding is bad, don't retry blindly".
|
|
|
|
|
"""
|
|
|
|
|
config = get_config()
|
|
|
|
|
configurable = config.get("configurable", {}) if isinstance(config, dict) else {}
|
|
|
|
|
cached = configurable.get("diff_line_set") if isinstance(configurable, dict) else None
|
|
|
|
|
if isinstance(cached, dict):
|
|
|
|
|
return cached
|
|
|
|
|
|
2026-05-27 10:03:33 -07:00
|
|
|
diff_text = await fetch_pr_diff(owner=owner, repo=repo, pr_number=pr_number, token=token)
|
|
|
|
|
if diff_text is None:
|
fix: publish_review HTTP 422 "Path/Line could not be resolved" — agent retries with identical args instead of dropping unresolvable findings (#1338)
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-05-26 18:37:56 -07:00
|
|
|
return None
|
2026-05-27 10:03:33 -07:00
|
|
|
return compute_diff_line_set(diff_text)
|
fix: publish_review HTTP 422 "Path/Line could not be resolved" — agent retries with identical args instead of dropping unresolvable findings (#1338)
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-05-26 18:37:56 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
async def _filter_against_pr_diff(
|
|
|
|
|
eligible_with_payload: list[tuple[dict[str, Any], dict[str, Any]]],
|
|
|
|
|
*,
|
|
|
|
|
owner: str,
|
|
|
|
|
repo: str,
|
|
|
|
|
pr_number: int,
|
|
|
|
|
token: str,
|
|
|
|
|
) -> tuple[list[tuple[dict[str, Any], dict[str, Any]]], list[str]]:
|
|
|
|
|
"""Drop findings whose path/line range is not in the current PR diff.
|
|
|
|
|
|
|
|
|
|
Returns ``(valid_with_payload, dropped_finding_ids)``. When the diff
|
|
|
|
|
cannot be resolved (fetch failed and no cached set), we return everything
|
|
|
|
|
unchanged and an empty drop list — the caller will then surface the
|
|
|
|
|
original error rather than retry blindly.
|
|
|
|
|
"""
|
|
|
|
|
diff_line_set = await _resolve_diff_line_set(
|
|
|
|
|
owner=owner, repo=repo, pr_number=pr_number, token=token
|
|
|
|
|
)
|
|
|
|
|
if diff_line_set is None:
|
|
|
|
|
return list(eligible_with_payload), []
|
|
|
|
|
|
|
|
|
|
valid: list[tuple[dict[str, Any], dict[str, Any]]] = []
|
|
|
|
|
dropped: list[str] = []
|
|
|
|
|
for finding, payload in eligible_with_payload:
|
|
|
|
|
path = payload.get("path")
|
|
|
|
|
# Prefer the finding's recorded range; fall back to the payload line.
|
|
|
|
|
start_line = finding.get("start_line")
|
|
|
|
|
end_line = finding.get("end_line")
|
|
|
|
|
if end_line is None:
|
|
|
|
|
payload_line = payload.get("line")
|
|
|
|
|
if isinstance(payload_line, int):
|
|
|
|
|
end_line = payload_line
|
|
|
|
|
if start_line is None:
|
|
|
|
|
start_line = payload_line
|
2026-05-27 10:03:33 -07:00
|
|
|
side = finding.get("side") if finding.get("side") in {"LEFT", "RIGHT"} else "RIGHT"
|
|
|
|
|
if isinstance(path, str) and is_range_in_diff(
|
|
|
|
|
diff_line_set, path, start_line, end_line, side=side
|
|
|
|
|
):
|
fix: publish_review HTTP 422 "Path/Line could not be resolved" — agent retries with identical args instead of dropping unresolvable findings (#1338)
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-05-26 18:37:56 -07:00
|
|
|
valid.append((finding, payload))
|
|
|
|
|
else:
|
|
|
|
|
finding_id = finding.get("id")
|
|
|
|
|
if isinstance(finding_id, str):
|
|
|
|
|
dropped.append(finding_id)
|
|
|
|
|
return valid, dropped
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
|
|
|
|
|
|
2026-05-07 16:11:36 -07:00
|
|
|
async def _maybe_post_slack_completion_reply(
|
|
|
|
|
*,
|
|
|
|
|
thread_id: str,
|
|
|
|
|
owner: str,
|
|
|
|
|
repo: str,
|
|
|
|
|
pr_number: int,
|
|
|
|
|
review_id: int | None,
|
|
|
|
|
surfaced_count: int,
|
|
|
|
|
) -> None:
|
|
|
|
|
"""Post a one-line completion summary to the Slack thread that started this review.
|
|
|
|
|
|
|
|
|
|
Only fires for first reviews (gated by the caller). No-op if the reviewer
|
|
|
|
|
thread has no ``slack_thread`` metadata — i.e. the review wasn't started
|
|
|
|
|
from Slack.
|
|
|
|
|
"""
|
|
|
|
|
metadata = await get_thread_metadata(thread_id)
|
|
|
|
|
slack_ref = get_thread_slack_ref(metadata)
|
|
|
|
|
if slack_ref is None:
|
|
|
|
|
return
|
|
|
|
|
|
|
|
|
|
if surfaced_count == 0:
|
|
|
|
|
headline = "*Open SWE Review*: No issues found."
|
|
|
|
|
else:
|
|
|
|
|
issue_word = "issue" if surfaced_count == 1 else "issues"
|
|
|
|
|
headline = f"*Open SWE Review* found {surfaced_count} potential {issue_word}."
|
|
|
|
|
|
|
|
|
|
review_url = f"https://github.com/{owner}/{repo}/pull/{pr_number}"
|
|
|
|
|
if isinstance(review_id, int):
|
|
|
|
|
review_url = f"{review_url}#pullrequestreview-{review_id}"
|
|
|
|
|
text = f"{headline} <{review_url}|View review>"
|
|
|
|
|
|
|
|
|
|
await post_slack_thread_reply(slack_ref["channel_id"], slack_ref["thread_ts"], text)
|
|
|
|
|
|
|
|
|
|
|
2026-05-26 16:24:34 -07:00
|
|
|
async def _store_thread_ids_on_findings(
|
|
|
|
|
*,
|
|
|
|
|
thread_id: str,
|
|
|
|
|
owner: str,
|
|
|
|
|
repo: str,
|
|
|
|
|
pr_number: int,
|
|
|
|
|
token: str,
|
|
|
|
|
) -> None:
|
|
|
|
|
findings = await list_findings_async(thread_id)
|
2026-05-27 10:52:30 -07:00
|
|
|
comment_ids_by_finding_id: dict[str, list[int]] = {}
|
2026-05-26 16:24:34 -07:00
|
|
|
for finding in findings:
|
|
|
|
|
finding_id = finding.get("id")
|
2026-05-27 10:52:30 -07:00
|
|
|
comment_ids = _comment_ids_for_finding(finding)
|
|
|
|
|
if isinstance(finding_id, str) and comment_ids and not _thread_ids_for_finding(finding):
|
|
|
|
|
comment_ids_by_finding_id[finding_id] = comment_ids
|
2026-05-26 16:24:34 -07:00
|
|
|
if not comment_ids_by_finding_id:
|
|
|
|
|
return
|
|
|
|
|
|
|
|
|
|
threads = await fetch_pr_review_threads(
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
token=token,
|
|
|
|
|
)
|
|
|
|
|
thread_id_by_comment_id: dict[int, str] = {}
|
|
|
|
|
for thread in threads:
|
|
|
|
|
github_thread_id = thread.get("id")
|
|
|
|
|
if not isinstance(github_thread_id, str) or not github_thread_id:
|
|
|
|
|
continue
|
|
|
|
|
for comment in thread.get("comments") or []:
|
|
|
|
|
if not isinstance(comment, dict):
|
|
|
|
|
continue
|
|
|
|
|
comment_id = comment.get("id")
|
|
|
|
|
if isinstance(comment_id, int):
|
|
|
|
|
thread_id_by_comment_id[comment_id] = github_thread_id
|
|
|
|
|
|
|
|
|
|
updated = False
|
|
|
|
|
for finding in findings:
|
|
|
|
|
finding_id = finding.get("id")
|
|
|
|
|
if not isinstance(finding_id, str):
|
|
|
|
|
continue
|
2026-05-27 10:52:30 -07:00
|
|
|
thread_ids = _thread_ids_for_finding(finding)
|
|
|
|
|
for comment_id in comment_ids_by_finding_id.get(finding_id, []):
|
|
|
|
|
github_thread_id = thread_id_by_comment_id.get(comment_id)
|
|
|
|
|
if not github_thread_id:
|
|
|
|
|
continue
|
|
|
|
|
if not isinstance(finding.get("github_review_thread_id"), str):
|
|
|
|
|
finding["github_review_thread_id"] = github_thread_id
|
|
|
|
|
updated = True
|
|
|
|
|
if github_thread_id not in thread_ids:
|
|
|
|
|
thread_ids.append(github_thread_id)
|
|
|
|
|
finding["github_review_thread_ids"] = thread_ids
|
|
|
|
|
updated = True
|
2026-05-27 17:26:08 -07:00
|
|
|
surface = _coerce_surface(finding, finding_id)
|
|
|
|
|
surface["state"] = "surfaced"
|
|
|
|
|
surface["github_review_thread_id"] = github_thread_id
|
|
|
|
|
finding["surface"] = surface
|
2026-05-26 16:24:34 -07:00
|
|
|
updated = True
|
|
|
|
|
|
|
|
|
|
if updated:
|
|
|
|
|
await replace_findings(thread_id, findings)
|
|
|
|
|
|
|
|
|
|
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
async def _resolve_threads_for_resolved_findings(
|
|
|
|
|
*,
|
|
|
|
|
owner: str,
|
|
|
|
|
repo: str,
|
|
|
|
|
pr_number: int,
|
|
|
|
|
token: str,
|
|
|
|
|
findings: list[dict[str, Any]],
|
|
|
|
|
) -> int:
|
|
|
|
|
"""Resolve GitHub review threads for findings that just transitioned to resolved.
|
|
|
|
|
|
2026-05-28 12:44:00 -07:00
|
|
|
Posts a resolution comment to the thread, then resolves it. Multiple threads
|
|
|
|
|
can exist when an earlier run duplicated a comment before publication identity
|
2026-05-27 10:52:30 -07:00
|
|
|
was backfilled.
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
"""
|
|
|
|
|
resolved_count = 0
|
|
|
|
|
mutated = False
|
|
|
|
|
for finding in findings:
|
2026-05-28 12:44:00 -07:00
|
|
|
status = finding.get("status")
|
|
|
|
|
if status not in {"resolved", "dismissed"}:
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
continue
|
2026-05-28 12:44:00 -07:00
|
|
|
|
2026-05-27 10:52:30 -07:00
|
|
|
thread_node_ids = _thread_ids_for_finding(finding)
|
2026-05-28 12:44:00 -07:00
|
|
|
comment_ids = _comment_ids_for_finding(finding)
|
|
|
|
|
|
|
|
|
|
for comment_id in comment_ids:
|
2026-05-26 16:24:34 -07:00
|
|
|
thread_node_id = await fetch_review_thread_id_for_comment(
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
review_comment_id=comment_id,
|
|
|
|
|
token=token,
|
|
|
|
|
)
|
2026-05-27 10:52:30 -07:00
|
|
|
if thread_node_id and thread_node_id not in thread_node_ids:
|
|
|
|
|
thread_node_ids.append(thread_node_id)
|
|
|
|
|
|
|
|
|
|
if not thread_node_ids:
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
continue
|
2026-05-27 10:52:30 -07:00
|
|
|
|
|
|
|
|
resolved_thread_ids = _str_list(finding.get("github_resolved_thread_ids"))
|
2026-05-28 12:44:00 -07:00
|
|
|
posted_resolution_comment_ids = _int_list(
|
|
|
|
|
finding.get("github_posted_resolution_comment_ids")
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
for idx, thread_node_id in enumerate(thread_node_ids):
|
2026-05-27 10:52:30 -07:00
|
|
|
if thread_node_id in resolved_thread_ids:
|
|
|
|
|
continue
|
2026-05-28 12:44:00 -07:00
|
|
|
|
|
|
|
|
primary_comment_id = comment_ids[idx] if idx < len(comment_ids) else None
|
2026-05-28 16:04:38 -07:00
|
|
|
resolution_body = render_resolution_comment(finding, status)
|
|
|
|
|
if (
|
|
|
|
|
primary_comment_id
|
|
|
|
|
and primary_comment_id not in posted_resolution_comment_ids
|
|
|
|
|
and resolution_body is not None
|
|
|
|
|
):
|
2026-05-28 12:44:00 -07:00
|
|
|
reply_response = await reply_to_review_comment(
|
|
|
|
|
owner=owner,
|
|
|
|
|
repo=repo,
|
|
|
|
|
pr_number=pr_number,
|
|
|
|
|
review_comment_id=primary_comment_id,
|
2026-05-28 16:04:38 -07:00
|
|
|
body=resolution_body,
|
2026-05-28 12:44:00 -07:00
|
|
|
token=token,
|
|
|
|
|
)
|
|
|
|
|
if reply_response and isinstance(reply_response.get("id"), int):
|
|
|
|
|
posted_resolution_comment_ids.append(primary_comment_id)
|
|
|
|
|
mutated = True
|
|
|
|
|
|
2026-05-27 10:52:30 -07:00
|
|
|
ok = await resolve_review_thread(thread_node_id=thread_node_id, token=token)
|
|
|
|
|
if ok:
|
|
|
|
|
resolved_thread_ids.append(thread_node_id)
|
|
|
|
|
resolved_count += 1
|
|
|
|
|
mutated = True
|
|
|
|
|
|
|
|
|
|
if resolved_thread_ids:
|
|
|
|
|
finding["github_resolved_thread_ids"] = resolved_thread_ids
|
2026-05-28 12:44:00 -07:00
|
|
|
if posted_resolution_comment_ids:
|
|
|
|
|
finding["github_posted_resolution_comment_ids"] = posted_resolution_comment_ids
|
2026-05-27 10:52:30 -07:00
|
|
|
if thread_node_ids:
|
|
|
|
|
finding["github_review_thread_ids"] = thread_node_ids
|
|
|
|
|
if not isinstance(finding.get("github_review_thread_id"), str):
|
|
|
|
|
finding["github_review_thread_id"] = thread_node_ids[0]
|
|
|
|
|
if thread_node_ids and all(
|
|
|
|
|
thread_id in resolved_thread_ids for thread_id in thread_node_ids
|
|
|
|
|
):
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
finding["github_thread_resolved"] = True
|
2026-05-27 17:26:08 -07:00
|
|
|
if isinstance(finding.get("id"), str):
|
|
|
|
|
surface = _coerce_surface(finding, str(finding["id"]))
|
|
|
|
|
surface["state"] = "resolved"
|
|
|
|
|
if thread_node_ids:
|
|
|
|
|
surface["github_review_thread_id"] = thread_node_ids[0]
|
|
|
|
|
finding["surface"] = surface
|
feat: implement reviewer findings, publish_review, and watch mode (#1253)
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
2026-05-07 14:48:43 -07:00
|
|
|
|
|
|
|
|
if mutated:
|
|
|
|
|
thread_id = get_thread_id_from_runtime()
|
|
|
|
|
await replace_findings(thread_id, findings)
|
|
|
|
|
|
|
|
|
|
return resolved_count
|
2026-05-26 16:24:34 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
def _current_run_id(config: dict[str, Any]) -> str | None:
|
|
|
|
|
candidates = [config.get("run_id")]
|
|
|
|
|
configurable = config.get("configurable")
|
|
|
|
|
if isinstance(configurable, dict):
|
|
|
|
|
candidates.append(configurable.get("run_id"))
|
|
|
|
|
for candidate in candidates:
|
|
|
|
|
if isinstance(candidate, str) and candidate:
|
|
|
|
|
return candidate
|
|
|
|
|
return None
|