* feat: tune reviewer for precision — web/wiki tools + recalibrated prompt
Reviewer agent now has web_search, fetch_url, and http_request alongside the
finding tools, so it can verify library semantics and consult the DeepWiki
auto-generated wiki for public repos (https://deepwiki.com/<owner>/<repo>)
before flagging cross-file or architectural concerns.
Prompt rewritten to push precision over recall:
- explicit severity ladder pushing reviews toward bimodal high/low instead of
defaulting to medium
- ≤200-char description target (gold set averages ~186 chars; we were at ~436)
- mandatory docs / wiki / code lookup before flagging concurrency, security,
or perf — the three categories that dominated false positives
- "do not flag" list covering compiler/linter-catchable nits, speculative
claims without a concrete attacker/interleaving/scale, style preferences
the codebase doesn't share, and test-quality nits on non-test diffs
- smart file-selection guidance for large PRs (deprioritize generated /
vendored / pure-rename hunks)
Eval config switched to openai:gpt-5.5 + high reasoning effort for the next
benchmark run.
* trim prompt
* subagent prompting
* confidence ratings
* added medium
* enforce confidence threshold
* .
* reviewer: precision-tuned prompt + drop confidence gate
Rewrites the reviewer system prompt around a defensibility bar (anchor +
failure mode + maintainer wouldn't say "not a bug"), an explicit do-not-file
list (style nits, speculation, scope-policing, same-bug fan-out), and a
checklist of 10 bug archetypes drawn from a per-PR audit of the eval golden
set. The audit showed 145 FPs in the last eval split ~28% speculative, ~26%
style-nit, ~31% real-but-unscored (mostly same-archetype fan-out); the new
prompt targets each class directly.
Confidence is still recorded on every finding for post-hoc calibration but
no longer gates publication — the audit showed the gate was a no-op (agent
self-rated 65% of findings "high" regardless), and the prompt's defensibility
bar is the actual discipline. Drops CONFIDENCE_ORDER, CONFIDENCE_THRESHOLD,
the confidence_threshold kwarg on filter_findings_for_publish, the
confidence_filtered score_mode, and the min_confidence kwarg on the eval
target's _extract_comments — all dead once the gate is gone.
Also removes the "informational" severity tier from the Severity enum,
SEVERITY_ORDER, and all validators / tests / docstrings. It was reserved for
FYI observations the dataset never rewards.
* benchmax
* adding google provider
* slight steering
* tuning
* more tuning
* fix
* cleanup
* reducing overfitting
* Add per-repo review style profiles and inject them into the reviewer.
Dashboard users can analyze historical PR review feedback per repository,
edit the resulting style guide, and have it loaded from LangGraph Store at
reviewer runtime (including Martian eval runs) keyed by owner/name.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix review style job errors leaking exception details to clients.
Return generic dashboard messages while logging full stack traces server-side.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Avoid treating regex-parsed Slack text as authoritative routing or allowlist input so repository mentions remain model context and access is bounded by installation permissions.
Drops the add_slack_reaction call in process_slack_mention; the Slack
assistant status indicator and the agent's first reply still signal
acknowledgement. Linear and GitHub eyes reactions are unchanged.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Threat model T6: Fernet token-encryption key had no rotation path. Rotating
TOKEN_ENCRYPTION_KEY immediately invalidated every github_token_encrypted
value in thread metadata, forcing re-auth or bot-token re-resolution.
Switch to cryptography.fernet.MultiFernet and parse TOKEN_ENCRYPTION_KEY as a
comma- or newline-separated ordered list (most-recent-first). New writes
encrypt under the first key; reads try every key. Deployers can prepend a new
key, let active threads roll over, then drop the old key.
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
* feat: add ALLOWED_GITHUB_REPOS env var for owner/repo-level webhook filtering
Previously only org-level filtering was supported via ALLOWED_GITHUB_ORGS.
This adds ALLOWED_GITHUB_REPOS for finer-grained control, allowing specific
owner/repo pairs to be allowlisted independently of org membership.
* fix: update test patches for _is_repo_org_allowed rename
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
* feat: move github workflows to gh cli
Use LangSmith proxy auth to support gh-driven GitHub workflows while removing custom GitHub wrapper tools.
* docker ignore + snapshot and docker image updates
* updated image and instructions
* removing open_pr if needed after agent call
When setting up the OAuth Provider in LangSmith using `github-oauth-provider` results in the error:
> Error: Provider ID 'github-oauth-provider' is reserved for Fleet public OAuth. Choose another name.
* feat: migrate LangSmith sandbox creation to snapshot API
Replaces the template-based sandbox flow (DEFAULT_SANDBOX_TEMPLATE_NAME /
DEFAULT_SANDBOX_TEMPLATE_IMAGE) with the new snapshot-based flow.
- New required env var DEFAULT_SANDBOX_SNAPSHOT_ID (UUID of a pre-built
LangSmith snapshot; build out-of-band via UI or SandboxClient.create_snapshot)
- Optional DEFAULT_SANDBOX_SNAPSHOT_FS_CAPACITY_BYTES overrides the root FS
size at boot (default 32 GiB)
- Startup-time validation via a FastAPI lifespan hook: the server refuses
to boot with a clear ValueError if SANDBOX_TYPE=langsmith and
DEFAULT_SANDBOX_SNAPSHOT_ID is unset, so failures surface in boot logs
rather than on the first thread
- Reconnect-to-existing-sandbox path unchanged
- Docs (INSTALLATION.md, CUSTOMIZATION.md) updated to describe the new
snapshot workflow
* fix: format create_sandbox_snapshot.py to pass ruff
---------
Co-authored-by: aran-yogesh <yogesh.mahendran@langchain.dev>
* fix: add Exa web search tool
* chore: update uv.lock for exa-py dependency
* linting
* chore: remove web_search from system prompt
* chore: drop search_type and category params from web_search
* feat: extract repo parsing into shared util and add linear comment repo override
Moves the repo extraction regex logic (repo:, repo , GitHub URL) into
agent/utils/repo.py so it can be reused. Updates the Linear webhook
handler to check the comment body for a custom repo first, falling back
to the team/project mapping when none is specified.
* chore: document repo extraction util and default org configuration
* feat: add generic DEFAULT_REPO_OWNER/DEFAULT_REPO_NAME env vars replacing Slack-only defaults
* chore: remove deprecated SLACK_REPO_OWNER/NAME env vars from docs
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
* feat: send LangSmith trace URL on run trigger from Slack and Linear
* fix: check LANGSMITH_PROJECT before LANGSMITH_PROJECT_PROD for project name lookup
* fix: pass LangSmith API key explicitly to Client and remove global variable
* Update agent/utils/langsmith.py
Co-authored-by: Brace Sproul <braceasproul@gmail.com>
* refactor: move trace notification helpers out of webapp and use env vars for LangSmith URL base
* docs: add LangSmith tenant and project ID env vars to installation guide
* fix: remove unused comment_on_linear_issue import from webapp
* nit: remove lru_cache, make get_langsmith_trace_url sync, and move langsmith import to top level
* Update INSTALLATION.md
Co-authored-by: Brace Sproul <braceasproul@gmail.com>
* Update INSTALLATION.md
Co-authored-by: Brace Sproul <braceasproul@gmail.com>
* Update INSTALLATION.md
Co-authored-by: Brace Sproul <braceasproul@gmail.com>
---------
Co-authored-by: Brace Sproul <braceasproul@gmail.com>
* feat: add GitHub PR comment trigger and reply support
* refactor: improve readability of GitHub integration
* linting
* fix: resolve github token from thread metadata and improve PR trigger flow
* fix: fall back to OAuth for GitHub webhook when no token in thread metadata
* give me commit message github integeration working without a breaking
* auth.py refactor
* fix: validate cached GitHub token before use to handle expiry
* liniting
* ci unitest formatting
* feat: post PR comments as GitHub App bot instead of user OAuth token
* resolved comments
* slack resolveed comments
* Refactor docstring and comments in get_slack_repo_config
Removed unnecessary comments and cleaned up docstring formatting.
* cr
* cr
* cr
* cr
---------
Co-authored-by: bracesproul <braceasproul@gmail.com>
