* feat: AI-sorted PR review view with diff grouping
Group a PR's changed files into a logical top-to-bottom walkthrough via a best-effort structured-output LLM pass, kicked off concurrently with the reviewer run (~0 added latency) and persisted on reviewer thread metadata. Results render in the review UI behind an AI sorted / file tree toggle that persists across PRs; the view falls back to the file tree when groups are absent or stale.
Adds a grouping-model team default (inherits the Reviewer subagent model when unset) and the admin RolePicker for it.
* feat(reviews): richer AI-sorted explanations + sidebar polish
Sidebar group rows get Devin-style spacing (dividers, padding), a per-group file list (click a file to jump to its diff), inline-code chips in titles, and an accent Read explanation link.
Group explanations are now rich markdown: the grouping prompt feeds per-hunk line ranges and asks for inline code, a short code block, and [path:line](#loc=...) references. The Markdown renderer turns those #loc= links into in-page buttons that scroll the diff to the hunk and highlight the range, reusing the existing selectedLines path.
* fix(reviews): drop stale diff groups from the AI-sorted view
When groups were generated for a previous head, a persisted "ai" view in localStorage still rendered the outdated walkthrough. groupedView now returns null on diff_groups_stale, so the file-tree fallback is used and the view toggle hides until fresh groups arrive.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: disable out-of-diff reviewer findings
PR reviews were surfacing findings about code outside the PR's changed
lines, which read as random/off-topic noise. Reject out-of-diff findings
at add_finding and stop surfacing them in publish_review so only findings
anchored to changed lines reach the PR.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: align reviewer bar with out-of-diff rejection
The filing bar still permitted proven regressions in files absent from
the diff, but add_finding now rejects those, so a concrete regression
could be silently dropped after a failed tool call. Update the bar and
the "Do NOT file" list so the prompt only directs in-diff findings.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
prepare_review_repo is best-effort, but the system prompt unconditionally
told the agent the repo is checked out at the PR head. On a reused sandbox,
a dirty worktree (or a transient fetch failure) makes 'git checkout <sha>'
fail; prep returns False, the old checkout stays in place, and the reviewer
confidently reads stale code — observed as 'I rechecked the current head'
replies quoting pre-push code.
- checkout with --force so leftover worktree state can't block it, verify
HEAD matches the requested sha, tolerate 'git fetch --all' failures
- when prep fails, the prompt now warns the checkout may be stale and tells
the agent to re-fetch/checkout (or fall back to API file contents) before
trusting local files
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: refresh sandbox GitHub proxy token before mid-run expiry
GitHub App installation tokens expire after exactly 1 hour. The LangSmith
sandbox proxy was configured once at run start with a snapshot of that
token, so runs longer than ~1h hit 401s on every gh/git call. Record the
proxy token's expiry per thread and add a before-model hook that
re-configures the proxy with a fresh token when it nears expiry.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: preserve repo-scoped proxy token on mid-run refresh
Reviewer runs mint a repository-scoped installation token. Record the
repo scope per thread alongside the expiry so the before-model refresh
re-mints a token with the same scope instead of an installation-wide
token, avoiding privilege expansion on long reviewer runs.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: update passthrough stub for github_proxy_repositories param
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: report Open SWE Review as a PR check run
Auto-review dispatch now creates an in-progress 'Open SWE Review' check run
on the PR head SHA; publish_review completes it (neutral with findings,
success when clean). An after-agent hook fails the check if the run dies
before publishing. Requires the GitHub App's Checks: Read & write permission;
all calls are best-effort so a missing permission never breaks reviews.
* fix: address review feedback on check-run settling
Keep review_check_run_id when the completion PATCH fails so a later
publish or the after-agent hook can retry instead of hanging the check;
count out-of-diff findings toward the check conclusion.
* fix: retry failed check completion with the real publish conclusion
A transient PATCH failure after a successful publish previously left the
check id for the after-agent hook, which settled it as 'failure'. Persist
the intended result as review_check_pending_result and have the hook
prefer it over the generic failure fallback.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: honest publish_review reporting + structured thread-not-found errors
- Document skipped_empty_re_review and dry_run in the publish_review
docstring and add a closing-summary contract to the reviewer prompt so
the agent never claims a review was published when review_id is null.
- Raise ReviewerThreadMissingError from replace_findings on SDK
NotFoundError; add_finding/update_finding/publish_review return a
structured do-not-retry result instead of raising, so the agent reports
the blocker after one failure instead of retrying 10-30 times.
* fix: translate thread 404s across all reviewer tool boundaries
get_thread_metadata now raises ReviewerThreadMissingError instead of
swallowing a missing thread as {} (which produced misleading 'No finding
found' results), set_reviewer_thread_metadata translates the SDK 404 the
same way, and every reviewer tool entrypoint (add/update/list findings,
publish_review incl. eval dry-run, resolve/reply thread) returns the
structured do-not-retry result.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: prep reviewer repo at init and load repo skills
Clone + checkout the PR head during reviewer agent init so SkillsMiddleware
can discover the repo's .agents/skills and .claude/skills from disk at its
one-shot scan, and so the LLM no longer narrates the clone mid-run.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: load reviewer skills from trusted base sha and fetch PR pull ref
Address review findings: skills are now extracted from the PR base sha via
git archive into a dir outside the checkout (prevents PR-authored SKILL.md
prompt injection), and repo prep fetches refs/pull/<n>/head with a strict
checkout so fork PRs fail loudly instead of silently reviewing the default
branch.
* fix: drop ref from skill-extraction log to satisfy CodeQL
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Pull the api-standards skill from the LangSmith Context Hub at reviewer
run start and inject it into the system prompt, gated on the PR adding or
modifying an API surface. Best-effort: failures fall back to no supplement.
Co-authored-by: GowriH-1 <218394553+GowriH-1@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Adds an admin-managed, org-wide review guidelines field to team settings
that the reviewer injects into every PR review across all repos, alongside
the existing per-repo style prompt and AGENTS.md context. Repo-specific
rules take precedence when they conflict.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* fix: stop reviewer retrying out-of-diff findings
add_finding rejects findings anchored outside the PR diff, but the agent
retried the same finding 2-3x with adjacent line ranges before giving up,
burning a model turn each. Add a reviewer-prompt recovery block telling the
agent the rejection is authoritative (drop or re-anchor to a + line, don't
retry adjacent), and enrich the rejection payload with nearby in-diff line
ranges for the file so a single re-anchor needs no guessing.
* feat: surface out-of-diff findings in a collapsed dropdown
Instead of rejecting findings anchored outside the PR diff, accept them
(marked in_diff=false) and surface them in a collapsed <details> section of
the review summary, Devin-style. Inline comments stay reserved for in-diff
findings; out-of-diff are severity-gated and capped the same way.
Re-review normally suppresses the empty summary, but now makes an exception
when there are new out-of-diff findings to surface. Surfaced out-of-diff
findings carry a github_review_id so they aren't reposted on later pushes.
Supersedes the earlier 'drop/re-anchor out-of-diff' prompt guidance.
---------
Co-authored-by: open-swe[bot] <215916821+open-swe[bot]@users.noreply.github.com>
Reviewer/push runs execute in a worker process, but the GitHub token was
cached by the webhook handler in the API server process — a different
process — so the worker's in-process cache was always cold. resolve_github_token
then failed (User not authenticated / Unknown source: github_push) and only the
app-token fallback kept reviews working, noisily.
The reviewer always acts as the GitHub App (open-swe[bot]), so resolve the
installation token directly at run start, scoped to the repo. This also bypasses
org SAML enforcement that blocks user OAuth tokens. Drop the now-dead
cross-process cache writes in the webhook reviewer-dispatch handlers, and stop
leave_failure_comment raising on the github_push source.
* fix: scope public reviewer tokens
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* refactor: simplify reviewer token wiring; fix push re-scope + red test
- Remove the redundant _check_or_recreate_sandbox_for_proxy /
_refresh_github_proxy_or_recreate_for_proxy wrappers and call the
underlying functions directly (they already default the token to None).
- process_github_push_event: re-scope the GitHub App token when the push
payload lacked repo privacy/id but PR metadata reveals a public repo, so
reviewer.py never proxies a full-installation token for a public PR.
- Clarify the two-token sequence in trigger_pr_review_from_ref.
- Fix pre-existing failing test test_proxy_refresh_failure_recreates_sandbox
and add coverage for _reviewer_token_for_repo + push-event scoping.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* Inject PR title and body into reviewer context
The reviewer agent previously received the PR url, number, and SHAs but not
the PR title or description, so it sometimes missed the original intent of
the PR. Fetch the title/body fresh from the GitHub API on every run (never
cached) so edits to the title/description are reflected on re-reviews, and
inject them into the first-review, re-review, and finding-reply contexts as
an untrusted-data block (author-controlled text, guarded against prompt
injection).
* Harden PR overview escaping against whitespace-padded closing tags
* feat: structured reviewer comments + auto resolution comments
Restructure inline review comment bodies (severity emoji, bold title from
the first line, line reference, feedback footer) without duplicating the
first description line, and post an automatic resolution/dismissal comment
to the GitHub thread when a finding is resolved or dismissed.
- Centralize render_resolution_comment in reviewer_publish; fix a crash when
last_reconciliation_note is None and drop the misleading generic fallback.
- Post the resolution comment in resolve_finding_thread (the normal
update_finding path), not only in publish_review, so it actually fires on
re-review. Dedupe via github_posted_resolution_comment_ids.
- Add pytest coverage for rendering and the resolution-comment flow.
* fix: post resolution comment to every closed thread in resolve_finding_thread
Per-thread iteration (matching _resolve_threads_for_resolved_findings) so
duplicate threads after the first also receive the resolved/dismissed
explanation before being closed.
* feat: reconcile reviewer findings with PR threads
* fix: harden reviewer finding reply handling
* fix: queue reviewer finding reply body
Ensure review-comment replies that arrive during an active reviewer run include the sanitized reply body in the queued reassessment prompt.
* fix: apply reviewer reply formatting
Apply the repository formatter so the reviewer reply handling fix passes CI format checks.
* reviewer: fetch PR diff via GitHub API to re-enable add_finding validation
The previous hotfix in reviewer.py set diff_line_set=None because the
sandbox-based diff prep was sometimes producing empty diffs. That made
every bad anchor a publish-time 422 instead of a creation-time
rejection — the agent burned tokens producing unanchorable findings,
and we had to add a publish-time retry safety net (#1338) to clean up.
Fetch the PR's unified diff via the GitHub REST API at reviewer
startup and populate diff_text + diff_line_set so add_finding can
reject bad anchors immediately. The API path is reliable and is the
same diff GitHub validates against when posting inline review
comments. If the fetch fails, fall back to the previous behavior
(validation disabled, publish-time retry handles it).
Also extract the PR-diff fetch into reviewer_diff.fetch_pr_diff so
both reviewer.py and publish_review.py share one implementation
instead of two copies.
* reviewer: make diff_line_set validation side-aware
compute_diff_line_set previously returned only new-side line numbers,
so re-enabling add_finding's validation would wrongly reject findings
with side=LEFT (deleted-line bugs whose only anchor is an old-side
line). Return {file: {"RIGHT": {new_lines}, "LEFT": {old_lines}}}
instead, and have is_range_in_diff select the matching side from the
finding's recorded side. add_finding and publish_review's retry
filter both pass the finding's side through.
* publish_review: drop unresolvable findings and retry once on GitHub 422
GitHub returns 422 with 'Path could not be resolved' or 'Line could not be
resolved' when an inline comment anchors to a file/line not in the PR diff.
Previously the agent retried publish_review with byte-identical args
multiple times before draining to skipped_empty_re_review=true, silently
losing findings.
- reviewer_publish.post_pull_request_review: parse 422 body and tag with
_error_kind='unresolved_anchor' plus _raw_errors so callers can act.
- tools/publish_review._publish_review_async: when that signal fires,
cross-check each finding's range against the run config's diff_line_set,
drop the bad ones, and re-POST once with only the valid findings. Return
unresolvable_findings + hint so the agent calls update_finding instead of
retrying the same payload.
- reviewer.py: one-line prompt addendum telling the agent that
unresolvable_findings means update_finding, not retry.
- tests: cover 422 tagging (path + line), the drop-and-retry success path,
the retry-still-fails path, and the don't-blind-retry path when no
diff_line_set is available.
* publish_review: fetch PR diff on demand for 422 retry filter
Reviewer runs clear configurable['diff_line_set'] before the agent
starts, so the unresolved-anchor retry path had no diff data to filter
against — in the reachable production case it dropped nothing and
returned success=False with empty unresolvable_findings, losing the
otherwise-valid comments.
Fall back to fetching the PR's unified diff via the GitHub REST API
and recomputing the line set on the fly when no cached set is
available. The cached set is still preferred when present.
---------
Co-authored-by: issues-agent <issues-agent@langchain.dev>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* fix(reviewer): inject existing PR review threads into reviewer context
The reviewer agent was filing the same inline comment on every re-review
because it only saw findings recorded on its own thread metadata — not
the live PR review-thread state on GitHub. When a previous finding was
still open (code unchanged, or a human reply explained it), the agent
rediscovered the same defect on the next push and called `add_finding`
again, producing duplicate comments.
This change fetches the PR's review threads (across all reviewers, with
replies and isResolved status) via GraphQL and renders them into the
first-review and re-review contexts as a "Pre-existing PR review
threads" block. The system prompt now lists overlap with that block as
a hard "Do NOT file" rule, and treats threads addressed by a human
reply as resolved.
This also gives the reviewer comment-awareness on its very first run
on a PR, so it skips findings already raised by another reviewer or
bot.
* fix(reviewer): wrap PR review threads in untrusted-data XML block
Addresses the reviewer comment on this PR
(https://github.com/langchain-ai/open-swe/pull/1331#discussion_r3295497533):
PR review comment bodies are attacker-controlled (anyone who can comment
on the PR can put anything in them), and they were being concatenated
into the reviewer's system prompt with instruction-priority.
Switches the existing-threads section from a Markdown block to an XML
data block:
<pr_review_threads>
<thread location="path:line" status="open">
<comment author="open-swe[bot]">
<body>...</body>
</comment>
<comment author="romain-priour-lc">
<body>We added defaults in the template</body>
</comment>
</thread>
</pr_review_threads>
The system prompt now explicitly names the wrapper, tells the agent that
everything inside it is untrusted data from the PR (not instructions),
and that prompt-injection payloads inside bodies must be disregarded.
We keep the bodies so the agent can actually read engineer replies —
that's the whole point of comment-awareness — but they're delimited as
data, not concatenated as prose. Modern frontier models are well-trained
to honor this contract.
Additional defenses:
- Author logins are validated against the GitHub username grammar; any
unexpected value is rendered as "unknown" so the `author` attribute
can't smuggle freeform text.
- Literal closing tags (`</body>`, `</pr_review_threads>`, etc.) in
bodies are neutered so a body can't break out of its wrapper.
- Body length is capped at 4000 chars per comment to bound the prompt.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat(reviewer): inline AGENTS.md into reviewer system prompt
Fetches AGENTS.md from the PR's head_sha via the GitHub contents API
during reviewer setup and inlines it as a "Repository conventions"
block in the system prompt. Mirrors how the per-repo review style
prompt is already wired.
The main agent has long had a mandatory step to read AGENTS.md after
cloning, but the reviewer often skips cloning entirely (it can
`gh pr diff` directly), so it never saw the file. Loading it
deterministically means the reviewer judges findings against the
project's own conventions instead of relying on the model to fetch
the file itself.
* fix(reviewer): fetch AGENTS.md from base_sha, not head_sha
The reviewer inlines AGENTS.md into its system prompt. Reading from
head_sha means a PR author can edit AGENTS.md in the same PR being
reviewed and smuggle instructions like "ignore all bugs" / "publish
no findings" into the reviewer's prompt. Switch to base_sha (the
target branch's pre-PR state, which is trusted) and update the prompt
text to reflect that the contents come from the base, not the head.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
The base reviewer prompt had grown into a curated catalog of defect
patterns and grep recipes tailored to the Martian eval set (Optional.get,
forEach(async), React keys, ERB templates, ORM increment, X-Frame-Options,
etc.). That bias toward one dataset hurts generalization across the
repos the reviewer actually runs on. Trim it down to universal review
discipline (the bar, do-not-file rules, severity rubric, publish
discipline, a generic workflow) and rely on per-repo prompts in memory
to carry repo-specific patterns at runtime.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat(ui): add Agents chat UI ported from open-swe-app
Introduce a Cursor-style Agents surface separate from the dashboard, with ported chat/diff components and mock thread data until LangGraph APIs land.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(dashboard): wire Agents UI to LangGraph thread APIs
Add dashboard thread list/detail/run/message/stream endpoints with a LangGraph message adapter, dashboard OAuth auth for runs, and TanStack Query hooks replacing mock data.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(dashboard): single agent reply per turn in Agents UI
Use UUID thread IDs LangGraph accepts, skip confirming_completion for
dashboard threads, and merge adapter agent messages so duplicate bubbles
do not render.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(ui): polish Agents UI with floating prompt and layout cleanup
Remove no-op chrome (git panel, headers, sidebar search), port CloudPromptBar
from open-swe-app, and refine chat layout so messages scroll behind the input.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(agent): patch deepagents reducer for None messages on checkpoint replay
LangGraph thread state could 500 when cancelled runs left messages as None.
Apply the reducer guard before graph import, fall back to metadata in the
dashboard API, and adjust Agents prompt bar layout.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(ui): unify sidebar user menu and clean up Agents UI navigation
Extract SidebarUserMenu so the dashboard and Agents sidebars render the
same profile button, drop the redundant Agents nav row in favor of the
existing Back to Agents link, add the open-swe logo header to the Agents
sidebar, flatten the New Agent button, and cap the home screen run list
to keep the prompt input in view.
* feat(ui): resizable/collapsible sidebar shared across dashboard and Agents
Add a useSidebarLayout hook + SidebarFrame wrapper so both sidebars
share a persisted width (default 260px, drag to resize, 200-420 range)
and a collapse toggle that hides the panel and surfaces a floating
reopen button. Also adds a DELETE /threads/{id} endpoint and an X-on-
hover thread delete control in the Agents sidebar.
* feat(ui): instant user message and busy indicator on Agents transition
Stash submitted prompts in sessionStorage, pre-populate the new thread
detail cache, and merge pending prompts into the rendered message list
so the Agents page renders the user bubble plus the existing thinking
spinner immediately instead of flashing a skeleton and "Agent is
starting" while the run boots.
* feat(ui): token-stream agent replies in the Agents thread view
Opt the LangGraph runs into messages-tuple streaming and forward those
events through the existing SSE channel. The frontend now applies
AIMessageChunk deltas directly to the cached thread (cancelling any
in-flight refetch first so optimistic tokens are not clobbered) and
keeps positional pending prompts so the user bubble stays in the right
place while the agent streams its reply.
* fix(dashboard): await threads.join_stream before iterating
threads.join_stream is async def returning an AsyncIterator, so it must
be awaited before async for. The SSE endpoint was raising
TypeError: 'async for' requires an object with __aiter__ method, got
coroutine on every connection.
* fix(dashboard): drop messages-tuple stream_mode that broke thinking-mode tool turns
Setting stream_mode=["values","messages-tuple","updates"] on
runs.create forces langchain_anthropic into streaming, and on the
second model call (after tool execution) its serialized thinking
blocks come back malformed, so Anthropic rejects the request with
'messages.1.content.0.thinking.thinking: Field required'. Revert to
the default stream_mode so claude-opus thinking + tool use runs to
completion. The frontend keeps the messages-event handler in place
as a no-op fallback for when streaming is re-enabled.
* feat(agents): per-thread model picker wired through to the run
Add optional model_id/effort to the create-thread and send-message
request bodies, forward them as agent_model_id/agent_effort in the
LangGraph run configurable, and record the resolved choice in thread
metadata so the UI can show the model the run is actually using.
get_agent now picks the per-thread override last (highest priority over
team default + profile override) and falls back gracefully when it is
absent or unsupported.
The frontend prompt bar becomes a controlled component fed by a
shared useModelOptions hook (options + profile -> defaultSelection).
AgentsHome seeds the picker from the user's profile default; the
thread view seeds from the thread's recorded model/effort and lets
each follow-up retarget the run.
* refactor(ui): align Agents prompt bar layout with open-swe-app PromptBar
Drop the absolute-positioned send button, restore the original
px-4 py-3.5 min-h-[106px] flex-col container, and move the model
picker into a mt-auto pt-2 footer row so the placeholder text and
the model selector share the same horizontal padding.
* chore: fix lint/format CI failures
Remove unused imports and reformat two files flagged by ruff.
* fix(tests): stop messages-reducer patch tests from polluting the suite
Restore agent modules after reducer patch tests and import LangSmithSandbox
from agent.server in proxy refresh tests so isinstance checks stay valid.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(dashboard): restructure Open SWE Review tab + wire create_prs
Restructures the dashboard around two related changes the reviewer settings
have been asking for:
- Wire profile.create_prs. Defaults to true (opt-out); when off the system
prompt gets a `Pull Request Policy Override` section telling the agent
to push the branch and notify with the branch URL instead of opening a
PR. Removes the noop Slack Notifications / Allow Artifacts / First Name
/ Last Name controls and their schema fields.
- Repositories opt-in for Open SWE Review. New per-team enabled list
stored in the LangGraph Store (`["enabled_review_repos"]`). Every
reviewer webhook chokepoint now goes through `_is_repo_enabled_for_review`
which AND-combines the existing env allowlist with the dashboard list.
Default is empty (opt-in) — admins enable repos per-installation from
the new Repositories page nested under Open SWE Review.
- Open SWE Review tab now mirrors the Cursor "rules" pattern: main page
shows installation rows + a Rules entry; both drill into nested pages
(/review/repositories/$owner and /review/styles) with a back link.
- Adds the new logo/favicon assets shipped from sidebar + html head.
Tests pass with a new autouse fixture (`tests/conftest.py`) that defaults
`is_review_repo_enabled` to True for existing allowlist tests.
* fix(dashboard): make main content scroll independently of the sidebar
Outer flex container was min-h-svh, so it grew with main's content and the
whole page scrolled — sidebar moved with it. Pin to h-svh + overflow-hidden
so the sidebar stays put and only <main> scrolls.
* fix(dashboard): make disabled repo toggles obviously disabled
Switch's disabled state used opacity-50 against a muted background, so
the not-admin state looked nearly identical to the off state. Bump to
opacity-40 + grayscale, and wrap each repo toggle in a span carrying a
native hover tooltip explaining why it's disabled.
* fix(switch): handle base-ui's data-disabled state
base-ui's Switch.Root sets data-disabled (not the HTML disabled attribute)
when disabled, so Tailwind's disabled: variant never matches and the
button keeps its cursor-pointer + clickable look. Mirror the styling
under the data-[disabled] variant and add pointer-events-none so the
disabled state is both visible and actually unclickable.
* feat(dashboard): paginate per-installation repository list
20 repos per page with Prev / page X of Y / Next controls at the bottom.
Pager only renders when there are more than 20 repos. Page resets to 0
when navigating between installations.
* feat(dashboard): global default model selectors for Agent + Reviewer
Adds team-wide default model + reasoning effort for both agents in the
Admin tab so operators can switch models without redeploying.
Resolution chain:
Agent: hardcoded -> LLM_MODEL_ID env -> team default -> user profile
Reviewer: hardcoded -> LLM_MODEL_ID env -> team default -> per-call configurable
Team defaults live in team_settings and are validated against the
SUPPORTED_MODELS allowlist + the model's supported reasoning efforts.
'Inherit from env' clears the override and falls back to LLM_MODEL_ID.
* refactor(models): drop LLM_MODEL_ID env in favour of the team default
The team default is now the single source of truth for the runtime model
choice; per-user (agent) and per-call configurable (reviewer) selections
still win on top. When no admin has touched the team default, it surfaces
the hardcoded fallback (DEFAULT_MODEL_ID + its default effort), so the
admin UI's dropdown is always pre-populated with a sensible value.
The Admin UI loses the 'Inherit from env' option since there is no longer
an env layer to inherit from.
* chore(models): set hardcoded fallback to gpt-5.5 medium
Decouple the team-default boot value (gpt-5.5 / medium) from each model's
ProfileForm-suggested default_effort so we can change one without nudging
the other. The Opus xhigh default for new user profiles is unchanged.
* feat(dashboard): trigger-mode copy, Coming Soon badges, logout in My Settings
- Rename trigger mode 'ready_for_review' -> 'once_per_pr' with new
description copy that matches the screenshot. Legacy stored values
fall back to 'every_push' on read so the UI never shows an unknown
selection.
- Add a 'Coming soon' badge + greyed-out + disabled state on the
controls that don't have runtime consumers yet: Trigger Mode,
Autofix Mode, Autofix Severity Threshold, and Automatically fix CI
failures. SettingsRow grew a comingSoon prop to keep this consistent.
- My Settings drops the noop PR Preferences section and adds a Sign
Out button. preferred_pr_destination is removed from the profile
schema; old records get the field popped on next write.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: tune reviewer for precision — web/wiki tools + recalibrated prompt
Reviewer agent now has web_search, fetch_url, and http_request alongside the
finding tools, so it can verify library semantics and consult the DeepWiki
auto-generated wiki for public repos (https://deepwiki.com/<owner>/<repo>)
before flagging cross-file or architectural concerns.
Prompt rewritten to push precision over recall:
- explicit severity ladder pushing reviews toward bimodal high/low instead of
defaulting to medium
- ≤200-char description target (gold set averages ~186 chars; we were at ~436)
- mandatory docs / wiki / code lookup before flagging concurrency, security,
or perf — the three categories that dominated false positives
- "do not flag" list covering compiler/linter-catchable nits, speculative
claims without a concrete attacker/interleaving/scale, style preferences
the codebase doesn't share, and test-quality nits on non-test diffs
- smart file-selection guidance for large PRs (deprioritize generated /
vendored / pure-rename hunks)
Eval config switched to openai:gpt-5.5 + high reasoning effort for the next
benchmark run.
* trim prompt
* subagent prompting
* confidence ratings
* added medium
* enforce confidence threshold
* .
* reviewer: precision-tuned prompt + drop confidence gate
Rewrites the reviewer system prompt around a defensibility bar (anchor +
failure mode + maintainer wouldn't say "not a bug"), an explicit do-not-file
list (style nits, speculation, scope-policing, same-bug fan-out), and a
checklist of 10 bug archetypes drawn from a per-PR audit of the eval golden
set. The audit showed 145 FPs in the last eval split ~28% speculative, ~26%
style-nit, ~31% real-but-unscored (mostly same-archetype fan-out); the new
prompt targets each class directly.
Confidence is still recorded on every finding for post-hoc calibration but
no longer gates publication — the audit showed the gate was a no-op (agent
self-rated 65% of findings "high" regardless), and the prompt's defensibility
bar is the actual discipline. Drops CONFIDENCE_ORDER, CONFIDENCE_THRESHOLD,
the confidence_threshold kwarg on filter_findings_for_publish, the
confidence_filtered score_mode, and the min_confidence kwarg on the eval
target's _extract_comments — all dead once the gate is gone.
Also removes the "informational" severity tier from the Severity enum,
SEVERITY_ORDER, and all validators / tests / docstrings. It was reserved for
FYI observations the dataset never rewards.
* benchmax
* adding google provider
* slight steering
* tuning
* more tuning
* fix
* cleanup
* reducing overfitting
* Add per-repo review style profiles and inject them into the reviewer.
Dashboard users can analyze historical PR review feedback per repository,
edit the resulting style guide, and have it loaded from LangGraph Store at
reviewer runtime (including Martian eval runs) keyed by owner/name.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix review style job errors leaking exception details to clients.
Return generic dashboard messages while logging full stack traces server-side.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat: tighten reviewer eval workflow
Require the reviewer to verify and dedupe findings before recording them, and make benchmark runs safe to execute against deployed reviewer graphs without posting GitHub reviews.
* chore: move reviewer eval settings to config
Load reviewer benchmark settings from the default eval config file so deployed eval runs do not require a wide CLI surface.
* feat: allow reviewer eval model overrides
Pass reviewer model and reasoning effort from the eval config into reviewer runs so isolated benchmark deployments can test Opus 4.7 high thinking.
* fix: use adaptive thinking for Opus 4.7
Switch Opus 4.7 model overrides to Anthropic adaptive thinking with effort instead of the deprecated budgeted thinking payload rejected by the API.
* refactor: use latest Anthropic effort API
Remove legacy Anthropic budget-token thinking support and route Anthropic efforts through adaptive thinking plus effort.
* revert prompting
* feat: TTL and revocation handling for cached GitHub OAuth tokens [closes AB-2322]
Persist github_token_expires_at alongside github_token_encrypted, treat
expired cache entries as missing so we re-resolve before kicking off
runs, and invalidate the cached ciphertext on a downstream 401 so the
next invocation gets a fresh token instead of replaying a revoked one.
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
* webapp: forward installation-token expiry to reviewer cache writes
The three reviewer-thread persist sites in webapp.py were calling
get_github_app_installation_token() (no expiry) and persist_encrypted_github_token
without expires_at, so cached App tokens were treated as never-expiring even
though they actually expire in ~1 hour.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat: add optional Slack Assistants API typing status indicator
Mirrors OpenClaw's pragmatic approach: instead of rebuilding around
assistant_thread_started events, just opt into assistants.threads.setStatus
to show 'is thinking…' while the agent is working, and clear it when
post_slack_thread_reply lands. Gated behind SLACK_ASSISTANTS_API_ENABLED so
it can be toggled without touching code.
* fix(slack): drop redundant clear, add status heartbeat across model calls
- Slack auto-clears the typing indicator on bot post; remove the explicit
assistants.threads.setStatus("") call from post_slack_thread_reply.
- The indicator expires after ~2 minutes; add a before_model middleware
that refreshes it on every model tick so it stays visible across long
agent runs. Reuses the existing slack_thread.{channel_id,thread_ts}
configurable already plumbed for notify_step_limit.
- chat:write is sufficient on the bot token (assistant:write is on the
way out per Slack docs); no scope or app-config change required.
* feat(slack): contextual status text + rotating loading_messages
- set_slack_assistant_status now accepts an optional loading_messages list
(capped at 10 per Slack's API), surfaced via the assistants.threads.setStatus
payload so Slack rotates through them client-side.
- The heartbeat middleware derives a contextual status from the last
assistant message's tool calls (e.g. "searching the codebase…" after
grep, "running commands…" after execute), falling back to the default
"is thinking…" when no tool calls or unknown tool name.
- Adds a curated DEFAULT_LOADING_MESSAGES list passed alongside the
contextual status on each refresh.
* fix slack assistant status lifecycle
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat(open-swe): always anchor findings to a single line
GitHub renders multi-line review comment ranges as walls of context above
the comment, which buries the point. Match Devin Review's behavior and
always collapse `end_line` to `start_line` so each finding is anchored to
the single most relevant line.
Removes the (now unused) `MAX_FINDING_RANGE_LINES` cap and
`clip_finding_range` helper.
* fix(open-swe): collapse end_line before diff-range check
Anchoring a finding to a 25-line range (e.g. an entire function) makes
GitHub render the whole block as context above the comment, burying the
review text. Cap ranges at 10 lines and collapse to the start line when
exceeded; small ranges (≤10 lines) still render as a multi-line anchor.
Updated the reviewer prompt to anchor tightly rather than range-select
the whole function.
* feat(open-swe): cap reviewer suggestions at 4 lines
Long suggestion blocks read as the reviewer rewriting the code rather
than flagging an issue, which clutters PR comments. Steer the reviewer
toward description-only findings for non-trivial fixes, and enforce the
cap in `add_finding` / `update_finding` so suggestions over 4 lines are
dropped (the finding itself still publishes).
* fix(open-swe): don't clobber prior suggestion on over-cap update
`update_finding` was setting `suggestion=None` whenever clip_suggestion
dropped the input, which silently wiped any existing suggestion on the
finding. Distinguish the three cases: empty string clears, valid value
sets, over-cap value is rejected without touching the stored field.
The host-side prep (clone + git diff in the sandbox) was producing empty
diffs for some PRs. The agent then silently published a "no issues
found" review (e.g. langchainplus#24069, #24458). Fix is to rip out the
prep entirely and instruct the agent to fetch the diff itself with
\`gh pr diff <num> --repo <owner>/<repo>\` (or the compare API on
re-review). \`add_finding\` line-range validation is skipped when no
\`diff_line_set\` is provided in config — we trust the agent's anchors.
We can revisit a host-side diff path once we understand why the prep
was failing; this hotfix unblocks reviews on private PRs in the
meantime.
When `_ensure_repo_checked_out` or `compute_diff_in_sandbox` fail, the
reviewer used to log+continue and hand the agent an empty diff. The
agent would then emit a misleading "no issues found" review and the
underlying error stayed buried in server logs.
Now the helpers raise on non-zero exit codes (with the sandbox output)
and the prep block re-raises, so the failure shows up in the LangSmith
run trace and we can debug the real issue.
The reviewer agent was writing a 1–2 sentence "top-level take" as the
review body, which produced noisy paragraph-style summaries on PRs
("Reviewed the PR. The new ALLOWED_GITHUB_REPOS allowlist…"). Devin's
review comment is just a one-liner (`✅ No Issues Found` or
`**Devin Review** found N potential issue.`), and was preferred in the
internal A/B vs Graphite.
Drop the `summary` parameter from `publish_review`; render a fixed,
host-formatted body in `render_review_body` instead. Update the reviewer
prompt to forbid prose summaries.
* fix(reviewer): log every push/close early-return so 'silent ignore' is debuggable
Pushes to PRs that haven't had a first review fall through the watch
handler because the reviewer thread doesn't have kind=reviewer set.
Without log lines on the early-return paths, this scenario was
indistinguishable from 'webhook reached the handler at all' in the
hosted log stream.
Now every early-return logs at info or debug:
- info when a real PR exists but the reviewer thread isn't set up
(with a hint pointing at the trigger paths the user can use)
- info when the repo isn't in the reviewer allowlist
- debug for benign skips (non-branch refs, branch deletions,
already-reviewed head_sha)
* fix(reviewer): always post a summary review, even with no findings
The publish_review tool gated POSTing on `inline_comments or summary`,
so when the agent called publish_review() with no args on a clean PR
the result returned `success: true` but no GitHub review was posted —
the user got silence instead of a "no issues found" comment.
- Drop the gate so publish_review always POSTs.
- Friendlier no-findings render: `**No issues found.**` when the
findings list is empty, vs. `**No issues at or above \`<sev>\`
severity.**` with hidden count when only sub-threshold findings
exist. Agent summary renders below.
- Prompt now requires the agent to always pass a `summary` so the
body is meaningful; calls out specifically not to skip on a clean PR.
* feat: implement reviewer findings, publish_review, and watch mode
Build out the reviewer agent end-to-end against the design in
REVIEWER_DESIGN.md:
- Findings as first-class state on the reviewer thread metadata
(`agent/reviewer_findings.py`): Finding TypedDict with start_line/end_line
ranges, suggestion text for ```suggestion blocks, github_review_comment_id
for cross-run reconciliation, diff_hunk for UI rendering. Thread-level
metadata gets `kind=reviewer`, `pr`, `last_reviewed_sha`, `watch` so a
future frontend can list reviewer threads via the langgraph SDK.
- Diff utilities (`agent/reviewer_diff.py`): parse_unified_diff,
compute_diff_line_set for in-diff validation, extract_diff_hunk for
caching the hunk on a Finding, compute_diff_in_sandbox for SHA-to-SHA
diffs against the prepped repo.
- Tools: `add_finding` (validates against the diff line set so out-of-diff
ranges fail at creation, not at GitHub-publish), `update_finding`,
`list_findings`, `publish_review`. The reviewer agent's tool list is
swapped from `[]` (direct shell `gh api` calls) to these four.
- Publish path (`agent/reviewer_publish.py` + `agent/tools/publish_review.py`):
one POST /reviews call with body + inline comments + ```suggestion blocks,
per-comment IDs stored back on findings, GraphQL `resolveReviewThread`
fired for findings transitioning open->resolved on a re-review.
- Reviewer graph: deterministic clone-or-fetch + checkout in the factory
before the agent's first model call (warm- and cold-path symmetric);
computed diff and in-diff line set passed via runnable config; system
prompt rewritten for the single-evolving-findings model, severity ladder,
in-diff-only discipline, and watch-mode reconciliation flow.
- Watch mode in webapp.py: `push` event + `pull_request` closed/reopened
added to supported events. New `process_github_push_event` resolves the
open PR for the pushed branch, gates on the reviewer thread's `watch`
flag, builds a re-review configurable, and triggers a run on the same
canonical thread. `process_github_pr_close` toggles watch on
closed/reopened. `set_reviewer_thread_metadata` is called on first
review to install `kind=reviewer` + PR identity + watch=True.
- Eval harness: target.py now extracts `add_finding` calls (mapped to the
legacy {file, line, body, severity} shape the judge expects) and passes
the right configurable so the prep step has base/head SHAs.
- Tests: new unit suites for findings helpers, diff parsing, finding tools,
publish rendering + GraphQL resolve, and watch-mode webhook handlers
(push triggers re-review only when watching, idempotent on unchanged
head SHA, PR close disables watch). Updated existing reviewer-webhook
tests to mock `set_reviewer_thread_metadata`.
- REVIEWER_EVAL_PLAN.md removed per user request; folded relevant context
into REVIEWER_DESIGN.md.
* fix(reviewer): correct git diff flags, scope, dedup, and review-comments URL
Address PR #1253 review findings:
- compute_diff_in_sandbox dropped the invalid `--no-prefix=false` flag
(`option no-prefix takes no value` — every prep run was failing
silently and the agent saw an empty diff).
- compute_diff_in_sandbox grew a `merge_base` flag. First-review path
now uses three-dot `base...head` (the merge-base diff GitHub renders
on Files-changed) so we don't pick up changes that landed on the base
branch after the PR diverged. Re-review delta keeps two-dot
`last_reviewed_sha..head` since that's exactly the new commits.
- publish_review skips findings that already carry
`github_review_comment_id`. Without this, watched re-reviews
re-posted every previously surfaced finding, and only the most-recent
duplicate's id would later resolve when the issue got addressed.
- fetch_review_comments URL now includes `{pull_number}` —
`/repos/{owner}/{repo}/pulls/{pr_number}/reviews/{review_id}/comments`
is the canonical endpoint; the old form 404s, so comment ids were
never stored and watch-mode resolution couldn't run.
Three new tests cover: three-dot vs two-dot wiring, no `--no-prefix`
flag in the executed command, and that publish_review does not re-post
findings whose `github_review_comment_id` is set.
* fix(reviewer): default publish cap from 15 to 4
A clean PR with one critical issue padded out by three lower-severity
findings is fine; fifteen is review spam. The agent can override per
call when a PR genuinely warrants more.
* Use gh for reviewer inline comments
* Trigger reviewer on PR review requests
* Format reviewer webhook test
* Add GitHub repo allowlist
* Separate reviewer repo allowlist
* only implement allowlist for reviewer
* feat: add reviewer graph + eval target wiring
- New `reviewer` graph (`agent/reviewer.py`) registered in langgraph.json
alongside the main `agent` graph. Reuses the same sandbox lifecycle,
GH proxy auth, and middleware primitives from `agent.server`, but with
a narrower tool set, a reviewer-specific system prompt, no
commit/push, and the `task` (subagent) tool stripped via
`_ToolExclusionMiddleware` so review stays in one context.
- New `github_comment` tool: agents call it once per issue with
`(file, line, body, severity)` and the eval scores those calls
against golden comments.
- `ensure_no_empty_msg` middleware (the no_op nudge) is intentionally
*not* on the reviewer's stack — that middleware exists to enforce the
main agent's "always finalize via Slack/Linear/PR" contract, which
the reviewer doesn't have. The main agent's behavior is unchanged.
- `evals/reviewer/target.py`: send PR info as a user message, extract
every `github_comment` tool call (multiple expected per review) into
the run output.
- `evals/reviewer/judge.py`: per-example evaluator now returns a list
of metrics under `{"results": [...]}` so LangSmith averages each
numeric key (f1/precision/recall/tp/fp/fn) across the experiment in
the UI. Dropped the broken `aggregate_pr` summary evaluator that
reached for an attribute that doesn't exist on `RunTree`.
- `evals/reviewer/run_eval.py`: `--limit` now slices the dataset via
`client.list_examples(limit=N)` since `aevaluate` doesn't accept
`max_examples`.
- Makefile: `dev` and `run` targets now use `uv run` so they work
without an activated venv.
* resolve comments
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
