An Open-Source Asynchronous Coding Agent
Find a file
Johannes du Plessis 82852f9eda
feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312)
* feat: tune reviewer for precision — web/wiki tools + recalibrated prompt

Reviewer agent now has web_search, fetch_url, and http_request alongside the
finding tools, so it can verify library semantics and consult the DeepWiki
auto-generated wiki for public repos (https://deepwiki.com/<owner>/<repo>)
before flagging cross-file or architectural concerns.

Prompt rewritten to push precision over recall:
- explicit severity ladder pushing reviews toward bimodal high/low instead of
  defaulting to medium
- ≤200-char description target (gold set averages ~186 chars; we were at ~436)
- mandatory docs / wiki / code lookup before flagging concurrency, security,
  or perf — the three categories that dominated false positives
- "do not flag" list covering compiler/linter-catchable nits, speculative
  claims without a concrete attacker/interleaving/scale, style preferences
  the codebase doesn't share, and test-quality nits on non-test diffs
- smart file-selection guidance for large PRs (deprioritize generated /
  vendored / pure-rename hunks)

Eval config switched to openai:gpt-5.5 + high reasoning effort for the next
benchmark run.

* trim prompt

* subagent prompting

* confidence ratings

* added medium

* enforce confidence threshold

* .

* reviewer: precision-tuned prompt + drop confidence gate

Rewrites the reviewer system prompt around a defensibility bar (anchor +
failure mode + maintainer wouldn't say "not a bug"), an explicit do-not-file
list (style nits, speculation, scope-policing, same-bug fan-out), and a
checklist of 10 bug archetypes drawn from a per-PR audit of the eval golden
set. The audit showed 145 FPs in the last eval split ~28% speculative, ~26%
style-nit, ~31% real-but-unscored (mostly same-archetype fan-out); the new
prompt targets each class directly.

Confidence is still recorded on every finding for post-hoc calibration but
no longer gates publication — the audit showed the gate was a no-op (agent
self-rated 65% of findings "high" regardless), and the prompt's defensibility
bar is the actual discipline. Drops CONFIDENCE_ORDER, CONFIDENCE_THRESHOLD,
the confidence_threshold kwarg on filter_findings_for_publish, the
confidence_filtered score_mode, and the min_confidence kwarg on the eval
target's _extract_comments — all dead once the gate is gone.

Also removes the "informational" severity tier from the Severity enum,
SEVERITY_ORDER, and all validators / tests / docstrings. It was reserved for
FYI observations the dataset never rewards.

* benchmax

* adding google provider

* slight steering

* tuning

* more tuning

* fix

* cleanup

* reducing overfitting

* Add per-repo review style profiles and inject them into the reviewer.

Dashboard users can analyze historical PR review feedback per repository,
edit the resulting style guide, and have it loaded from LangGraph Store at
reviewer runtime (including Martian eval runs) keyed by owner/name.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix review style job errors leaking exception details to clients.

Return generic dashboard messages while logging full stack traces server-side.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 18:35:00 +00:00
.github ci: nightly promote of main to prod for LangGraph deploys (#1285) 2026-05-09 01:15:26 +00:00
.vscode Brace/07 16/fixes (#431) 2025-07-16 13:17:35 -07:00
agent feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312) 2026-05-20 18:35:00 +00:00
evals/reviewer feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312) 2026-05-20 18:35:00 +00:00
scripts feat: move github workflows to gh cli (#1238) 2026-05-04 18:03:53 -07:00
static fix: Add back logo to readme (#1068) 2026-03-17 10:41:41 -07:00
tests feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312) 2026-05-20 18:35:00 +00:00
ui feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312) 2026-05-20 18:35:00 +00:00
.codespellignore init commit 2025-05-21 14:47:56 -07:00
.dockerignore feat: move github workflows to gh cli (#1238) 2026-05-04 18:03:53 -07:00
.gitignore feat(open-swe): Default to GPT-5.5 medium reasoning (#1224) 2026-04-28 15:03:21 -07:00
AGENTS.md feat: move github workflows to gh cli (#1238) 2026-05-04 18:03:53 -07:00
CLAUDE.md feat: move github workflows to gh cli (#1238) 2026-05-04 18:03:53 -07:00
CUSTOMIZATION.md feat: add idle TTL and delete-after-stop sandbox lifecycle controls (#1265) 2026-05-08 00:30:14 -04:00
default_prompt.md feat: add configurable default prompt file for org-level agent instructions [close OPE-36] (#1187) 2026-04-15 15:18:14 -07:00
Dockerfile feat: move github workflows to gh cli (#1238) 2026-05-04 18:03:53 -07:00
INSTALLATION.md feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312) 2026-05-20 18:35:00 +00:00
langgraph.json feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312) 2026-05-20 18:35:00 +00:00
LICENSE feat: Monorepo (#22) 2025-05-26 13:02:51 -07:00
Makefile feat: add reviewer graph + eval target wiring (#1241) 2026-05-06 10:15:58 -07:00
pyproject.toml feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312) 2026-05-20 18:35:00 +00:00
README.md feat: move github workflows to gh cli (#1238) 2026-05-04 18:03:53 -07:00
REVIEWER_DESIGN.md feat: implement reviewer findings, publish_review, and watch mode (#1253) 2026-05-07 14:48:43 -07:00
SECURITY.md fix: Security stuff (#441) 2025-07-17 12:19:20 -07:00
uv.lock feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312) 2026-05-20 18:35:00 +00:00

Open-source framework for building your org's internal coding agent.

License GitHub Stars Built on LangGraph Built on Deep Agents Twitter / X

Elite engineering orgs like Stripe, Ramp, and Coinbase are building their own internal coding agents — Slackbots, CLIs, and web apps that meet engineers where they already work. These agents are connected to internal systems with the right context, permissioning, and safety boundaries to operate with minimal human oversight.

Open SWE is the open-source version of this pattern. Built on LangGraph and Deep Agents, it gives you the same architecture those companies built internally: cloud sandboxes, Slack and Linear invocation, subagent orchestration, and automatic PR creation — ready to customize for your own codebase and workflows.

Note

💬 Read the announcement blog post here


Architecture

Open SWE makes the same core architectural decisions as the best internal coding agents. Here's how it maps to the patterns described in this overview of Stripe's Minions, Ramp's Inspect, and Coinbase's Cloudbot:

1. Agent Harness — Composed on Deep Agents

Rather than forking an existing agent or building from scratch, Open SWE composes on the Deep Agents framework — similar to how Ramp built on top of OpenCode. This gives you an upgrade path (pull in upstream improvements) while letting you customize the orchestration, tools, and middleware for your org.

create_deep_agent(
    model="openai:gpt-5.5",
    system_prompt=construct_system_prompt(...),
    tools=[http_request, fetch_url, linear_comment, slack_thread_reply],
    backend=sandbox_backend,
    middleware=[ToolErrorMiddleware(), check_message_queue_before_model, ...],
)

2. Sandbox — Isolated Cloud Environments

Every task runs in its own isolated cloud sandbox — a remote Linux environment with full shell access. The repo is cloned in, the agent gets full permissions, and the blast radius of any mistake is fully contained. No production access, no confirmation prompts.

Open SWE supports multiple sandbox providers out of the box — Modal, Daytona, Runloop, and LangSmith — and you can plug in your own. See the Customization Guide for details.

This follows the principle all three companies converge on: isolate first, then give full permissions inside the boundary.

  • Each thread gets a persistent sandbox (reused across follow-up messages)
  • Sandboxes auto-recreate if they become unreachable
  • Multiple tasks run in parallel — each in its own sandbox, no queuing

3. Tools — Curated, Not Accumulated

Stripe's key insight: tool curation matters more than tool quantity. Open SWE follows this principle with a small, focused toolset:

Tool Purpose
execute Shell commands in the sandbox
fetch_url Fetch web pages as markdown
http_request API calls (GET, POST, etc.)
linear_comment Post updates to Linear tickets
slack_thread_reply Reply in Slack threads

GitHub operations are performed with GH_TOKEN=dummy gh inside the sandbox, backed by the LangSmith proxy. Plus the built-in Deep Agents tools: read_file, write_file, edit_file, ls, glob, grep, write_todos, and task (subagent spawning).

4. Context Engineering — AGENTS.md + Source Context

Open SWE gathers context from two sources:

  • AGENTS.md — If the repo contains an AGENTS.md file at the root, it's read from the sandbox and injected into the system prompt. This is your repo-level equivalent of Stripe's rule files: encoding conventions, testing requirements, and architectural decisions that every agent run should follow.
  • Source context — The full Linear issue (title, description, comments) or Slack thread history is assembled and passed to the agent, so it starts with rich context rather than discovering everything through tool calls.

5. Orchestration — Subagents + Middleware

Open SWE's orchestration has two layers:

Subagents: The Deep Agents framework natively supports spawning child agents via the task tool. The main agent can fan out independent subtasks to isolated subagents — each with its own middleware stack, todo list, and file operations. This is similar to Ramp's child sessions for parallel work.

Middleware: Deterministic middleware hooks run around the agent loop:

  • check_message_queue_before_model — Injects follow-up messages (Linear comments or Slack messages that arrive mid-run) before the next model call. You can message the agent while it's working and it'll pick up your input at its next step.
  • notify_step_limit_reached — After-agent hook that posts a Slack reply when the agent hits the model-call limit, so users get a clear signal instead of silence.
  • ToolErrorMiddleware — Catches and handles tool errors gracefully.

6. Invocation — Slack, Linear, and GitHub

All three companies in the article converge on Slack as the primary invocation surface. Open SWE does the same:

  • Slack — Mention the bot in any thread. Supports repo:owner/name syntax to specify which repo to work on. The agent replies in-thread with status updates and PR links.
  • Linear — Comment @openswe on any issue. The agent reads the full issue context, reacts with 👀 to acknowledge, and posts results back as comments.
  • GitHub — Tag @openswe in PR comments on agent-created PRs to have it address review feedback and push fixes to the same branch.

Each invocation creates a deterministic thread ID, so follow-up messages on the same issue or thread route to the same running agent.

7. Validation — Prompt-Driven

The agent is instructed to run linters, formatters, and tests before committing, and is responsible end-to-end for committing, pushing, opening/updating the draft PR, and replying in the source channel. This is an area where you can extend Open SWE for your org: add deterministic CI checks, visual verification, or review gates as additional middleware. See the Customization Guide for how.


Comparison

Decision Open SWE Stripe (Minions) Ramp (Inspect) Coinbase (Cloudbot)
Harness Composed (Deep Agents/LangGraph) Forked (Goose) Composed (OpenCode) Built from scratch
Sandbox Pluggable (Modal, Daytona, Runloop, etc.) AWS EC2 devboxes (pre-warmed) Modal containers (pre-warmed) In-house
Tools ~15, curated ~500, curated per-agent OpenCode SDK + extensions MCPs + custom Skills
Context AGENTS.md + issue/thread Rule files + pre-hydration OpenCode built-in Linear-first + MCPs
Orchestration Subagents + middleware Blueprints (deterministic + agentic) Sessions + child sessions Three modes
Invocation Slack, Linear, GitHub Slack + embedded buttons Slack + web + Chrome extension Slack-native
Validation Prompt-driven 3-layer (local + CI + 1 retry) Visual DOM verification Agent councils + auto-merge

Features

  • Trigger from Linear, Slack, or GitHub — mention @openswe in a comment to kick off a task
  • Instant acknowledgement — reacts with 👀 the moment it picks up your message
  • Message it while it's running — send follow-up messages mid-task and it'll pick them up before its next step
  • Run multiple tasks in parallel — each task runs in its own isolated cloud sandbox
  • GitHub OAuth built-in — authenticates with your GitHub account automatically
  • Opens PRs automatically — commits changes and opens a draft PR when done, linked back to your ticket
  • Subagent support — the agent can spawn child agents for parallel subtasks

Getting Started

  • Installation Guide — GitHub App creation, LangSmith, Linear/Slack/GitHub triggers, and production deployment
  • Customization Guide — swap the sandbox, model, tools, triggers, system prompt, and middleware for your org

License

MIT