open-swe/tests/test_reviewer_tools.py
Adam Moussa 1f060f2a1d
Some checks are pending
CI / Lint (push) Waiting to run
CI / Format check (push) Waiting to run
CI / Unit tests (push) Waiting to run
CI / Playwright E2E (push) Waiting to run
chore: sync upstream/main, defer #1621 modular webhooks (#81)
* chore: bake sfw binary into sandbox image (#1611)

sfw only ships a launcher that fetches its real binary at first run and does a
daily update check against api.github.com/repos/SocketDev/sfw-free. Both fail in
the sandbox (restricted egress; the proxy injects the GitHub App installation
token, which lacks access to that repo), so `sfw yarn install` errors with
"could not fetch its binary". Pin sfw 2.0.6, warm + verify the binary cache at
build, and set SFW_SKIP_UPDATE_CHECK=1 so runs use the baked binary offline.

* feat: editable plan mode + fix review-plan banner overlap (#1610)

* feat: editable plan mode + fix review-plan banner overlap

Lets the thread owner edit the plan markdown by hand from the plan-review
page (Edit -> textarea -> Save) via a new PUT /dashboard/api/plan/{id}
endpoint that re-publishes the plan and mirrors it into the sandbox
plan.md, so approve hands the edited plan to the agent as the source of
truth. Also fixes the collapsed git-panel's floating expand button
covering the "Review plan ->" banner by reserving space for it.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: abort plan approval when the published plan read fails

get_plan_content() swallowed store errors and returned None, so a
transient failure during approve would still mark the plan approved and
dispatch the generic fallback text — silently dropping an owner's edited
plan. Read the plan strictly (raise_on_error=True) so approval aborts
instead, matching the comment read.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: show message timestamps (#1609)

* feat: show message timestamps

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: suppress fallback message timestamps

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: stable message + tool-call hover timestamps

Stamp a stable client-side arrival time per message and tool call (keyed
by id, persisted to localStorage). Messages render the timestamp inline;
tool rows reveal a dim timestamp chip on hover. Real backend created_at
still takes precedence when present.

* fix: hide client-stamped message timestamps

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: add PR trace resolution (#1612)

* feat: add PR trace resolution

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: inject reviewer trace context as JSON

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: address review on PR trace resolution

Use the documented LangSmith metadata filter syntax
(and(eq(metadata_key,...), eq(metadata_value,...))) instead of
has(metadata, '{...}'), which does not match runs — _list_thread_runs
was silently returning nothing. Bound full-text searches to a 90-day
window so they don't hit LangSmith's large-window rate limit.

Also folds in the best-effort branch->head-sha resolver (dropping the
weighted scoring/threshold + repo/file evidence + GitHub hydration),
sandbox JSON injection, and the admin "Resolve trace" dry-run endpoint.

The IDOR findings are moot: resolve_pr_to_threads/summarize_agent_session
were removed; resolution now runs deterministically from the trusted run
config with no model-controlled pr_url or thread_id.

* fix: scope branch trace search to the repo

Branch names like fix-tests aren't unique across repos (or older PRs) in
a shared tracing project, so an unscoped branch hit could resolve to an
unrelated thread and write its runs into the reviewer sandbox. Require
the repo slug to co-occur with the branch in matched runs; the full head
SHA stays unscoped since it is globally unique. Addresses open-swe review
on PR #1612.

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: include plan links in PR descriptions (#1613)

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: gate workflow pushes with approval (#1614)

* feat: gate workflow pushes with approval

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: preserve proxy refresh test compatibility

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: bind workflow approvals to pushed ref

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: recover thread work as patch (#1615)

* feat: recover thread work as patch

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: search sandbox cwd for recovery patches

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: omit plan link in PR description when no plan exists (#1618)

Plan links in PR descriptions were always built from the thread id, so
runs that never produced a plan linked to an empty plan-review page.
Now the plan content store is consulted first; the link is only added
when a plan with non-empty markdown actually exists. A transient store
failure degrades gracefully (no link) rather than blocking PR creation.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: add filter & grouping menu to agents threads sidebar (#1617)

Add a Cursor-style control to the agents sidebar that groups (None/Date/
Status/Project), filters (ownership, status, source, pull request, model,
repo, include-resolved), and compacts the threads list. All client-side over
already-fetched sidebar threads; preferences persist in localStorage.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* chore: update langsmith sdk to 0.9.3 (#1616)

Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: clickable shared PR header in git panel and reviews (#1620)

* feat: clickable shared PR header in git panel and reviews

Replace the standalone "View PR" button in the agent git panel with a
clickable PR title, matching the reviews view. Extract a shared PrHeader
component reused by both the git panel and the review main body.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* refactor: drop PrHeader wrapper, use shared component directly

The review-side PrHeader was just a thin adapter mapping detail -> the
shared component's props. Inline it at the call site and use the shared
PrHeader directly so there's a single component.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* refactor: durable interrupt dispatch + completion webhook (#1621)

* wip(rebuild): core reliability spine

- remove PR-babysitting (ci_autofix + ci_monitor graph + webhook wiring)
- dispatch core: agent/dispatch.py with multitask_strategy=interrupt +
  durability=sync + completion webhook; reroute all webhook + plan triggers;
  drop the racy in-process lock + is_thread_active busy-check
- completion webhook: agent/completion.py + /webhooks/run-complete loopback
  route for failure/timeout replies (idempotent)

Co-authored-by: open-swe[bot]

* feat(rebuild): async tools, reconcile, shared http timeouts, assembly tuning

Parallel batch on top of the reliability spine:
- async-ify all 24 tools (drop asyncio.run; requests->httpx); re-implement the
  http_request/fetch_url SSRF + DNS-rebinding defense httpx-natively and harden
  the IP check to 'not is_global' (+ IPv4-mapped unwrap)
- reconcile.py: stale pending-run sweep (threads.search -> per-thread runs.list
  -> cancel_many), wired into the scheduler graph via task='reconcile'
- shared DEFAULT_HTTP_TIMEOUT (agent/utils/http.py) on every bare
  httpx.AsyncClient() across utils/dashboard/webapp/middleware
- run budget: MODEL_CALL_RECURSION_LIMIT 5000->250
- fix stale OpenAI->Anthropic fallback id (claude-opus-4-5 -> 4-8)
- drop redundant custom repair middleware (deepagents auto-adds PatchToolCalls)
- confirm tool-result eviction + summarization auto-wired via backend
- slim system prompt ~8% (full harness-profile rewrite deferred)

Co-authored-by: open-swe[bot]

* feat(rebuild): harness-profile prompt + split webhooks out of webapp

- prompt.py: own the system prompt via a registered harness profile
  (OPEN_SWE_SHARED_BASE, kept neutral so the read-only reviewer/analyzer that
  share it stay safe), registered across all 4 providers; per-thread values
  stay in construct_system_prompt. Assembled main-agent prompt ~6.8k -> ~3.1k
  tokens (~55% smaller); de-duped PR/commit/suite/force-push guidance; dropped
  ALL-CAPS markers.
- webapp.py 3325 -> 1890 LOC: moved 14 per-source handlers into
  agent/webhooks/{linear,slack,github}.py; webapp re-exports them for the
  routes + tests; moved handlers reach shared helpers via the webapp namespace
  to preserve the test suite's monkeypatch targets.

Full suite: 1168 passing, lint clean.

Co-authored-by: open-swe[bot]

* Restore MODEL_CALL_RECURSION_LIMIT to 5000 for long-running tasks

Reverts the 250 cap from the run-budget change — long-running tasks legitimately
need many model calls. The notify_step_limit_reached safety net still fires if a
run does hit the cap, so runs end with a signal either way.

Co-authored-by: open-swe[bot]

* fix: address PR review (auth, SSRF, interrupted status, redirect headers)

- completion.py: drop `interrupted` from failure statuses — with
  multitask_strategy=interrupt a follow-up ends the prior run as interrupted,
  which is healthy, not a failure to report. [open-swe]
- /webhooks/run-complete: shared-secret auth — dispatch appends ?token= when
  RUN_COMPLETE_WEBHOOK_SECRET is set; route verifies via hmac.compare_digest.
  [corridor-security]
- SSRF: extract the URL validator to agent/utils/url_safety.py and apply it
  before server-side image fetches in multimodal.fetch_image_block.
  [corridor-security]
- http_request: preserve caller headers/extensions across redirect hops instead
  of dropping them on the first hop. [open-swe]

Co-authored-by: open-swe[bot]

* chore: remove REBUILD_PLAN.md (planning doc, not needed in the repo)

Co-authored-by: open-swe[bot]

* fix: fail closed on run-complete webhook auth when secret unset

Corridor follow-up: verify_run_complete_token returns False (not True) when
RUN_COMPLETE_WEBHOOK_SECRET is unset, so the public route is never
unauthenticated. Logs a startup warning when the secret is absent, and dispatch
skips registering the webhook when there's no secret (no rejected callbacks).

Co-authored-by: open-swe[bot]

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: restore forced tool call to prevent premature run stops (#1622)

Restore the ensure_no_empty_msg middleware and the always-call-a-tool system-prompt instruction that #1535 removed. When the model emits a message with no tool call (and hasn't already messaged the user or confirmed completion), the middleware re-injects a no_op / confirming_completion tool call so the run continues instead of ending mid-task.

Shipping to test whether it fixes runs that stop halfway through.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* chore(deps): bump langgraph-checkpoint from 4.1.0 to 4.1.1 (#1619)

Bumps [langgraph-checkpoint](https://github.com/langchain-ai/langgraph) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/langchain-ai/langgraph/releases)
- [Commits](https://github.com/langchain-ai/langgraph/compare/checkpoint==4.1.0...checkpoint==4.1.1)

---
updated-dependencies:
- dependency-name: langgraph-checkpoint
  dependency-version: 4.1.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: post reviewer resolution notes verbatim (#1624)

* fix: post reviewer resolution notes verbatim

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: stabilize dashboard follow-up e2e

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: preserve dashboard attribution in e2e

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: make e2e attribution marker durable

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: only echo found e2e attribution

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: check live dashboard attribution in e2e

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* hotfix: stop prompting agent/reviewer to wrap installs in sfw (#1625)

Installs hung when prefixed with sfw inside the sandbox (trace 019f0608
stalled on a pending `sfw npm install` execute, never returned). Strip the
Socket Firewall guidance from the agent and reviewer prompts so installs run
through the project's package manager directly. sfw stays in the Docker image;
nothing invokes it now.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: make plan view mobile friendly (#1636)

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: fall back to vision model for image threads (#1626)

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: surface Slack thread errors (#1627)

* fix: surface Slack thread errors

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: don't set failure_reply_posted on Slack preprocessing errors

The preprocessing error handler was setting failure_reply_posted=True,
the same idempotency flag handle_run_completion checks to suppress
duplicate run-failure replies. Since preprocessing failures happen
before any run exists but the flag persists on the thread, a subsequent
run failure on the same thread would be silently ignored.

The preprocessing handler already posts its own Slack reply, so the
run-completion idempotency flag should not be set here.

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* chore: avoid recapping Slack replies (#1629)

* chore: avoid recapping Slack replies

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* chore: simplify Slack reply prompt wording

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

---------

Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: update Slack trace reply on web handoff (#1630)

* fix: update Slack trace reply on web handoff

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: trigger web handoff on dashboard starts

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: format web handoff as contextual fragment

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: preserve trace_message_ts when overwriting Slack run mapping

When store_slack_run_mapping is called without trace_message_ts (e.g. on
follow-up Slack mentions), it was unconditionally overwriting the
thread-level mapping and clobbering the timestamp captured from the
initial trace reply. After that, _notify_slack_web_handoff could not find
the original message, so a subsequent move to Web silently skipped the
Slack trace update.

Now, when trace_message_ts is not passed, the existing thread mapping is
read first and its trace_message_ts is preserved.

* style: ruff format

---------

Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>

* fix: pre-bundle shiki/@pierre deps to stop dev dynamic-import failures (#1643)

* fix(ui): pre-bundle shiki/@pierre deps to stop dev dynamic-import failures

shiki lazy-imports a grammar per language and these libs only live inside
lazy route components, so Vite's startup scanner never sees them. They get
discovered on first thread navigation, triggering a dep re-optimize +
force-reload that aborts the in-flight route-chunk import, surfacing as
"Failed to fetch dynamically imported module: .../$threadId.tsx".

Pre-bundle them (and the github themes + common code-block languages) via
optimizeDeps.include so the optimize happens once at startup. Dev-only;
production bundles are unaffected.

* fix: pre-bundle canonical shiki docker/make langs instead of aliases

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: show queued dashboard follow-ups (#1631)

* feat: show queued dashboard follow-ups

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: de-dupe queued follow-ups while streaming

---------

Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>

* feat: notify Slack on plan approval (#1632)

* feat: notify Slack on plan approval

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: post Slack approval notice after successful dispatch

Move the _maybe_post_plan_approved_to_slack call until after
_dispatch_followup succeeds so the Slack thread is not told
implementation is beginning before the LangGraph run is created.

Addresses PR review comment.

---------

Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>

* feat: include Slack channel context in prompts (#1633)

Add cached Slack channel metadata enrichment for Slack-triggered runs so prompts can include channel names and descriptions without duplicate conversations.info calls.\n\nCo-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>

* chore: keep plan guidance high-level (#1634)

Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: publish plans from sandbox files (#1635)

* feat: publish plans from sandbox files

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: avoid fixed plan filenames

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: virtualize local sandbox file paths

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: preserve plan_file_path across set_plan_status

set_plan_status was rewriting the content record with only markdown
and status, dropping plan_file_path. After a reject, the owner's
dashboard edit would mirror to a different file than the agent's
original, and the next save_plan could republish the stale file.
Preserve plan_file_path when updating status.

---------

Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: return to thread after plan approval (#1637)

Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: add Slack breakout thread tool (#1638)

* feat: add Slack breakout thread tool

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* chore: make fake LLM scripts declarative

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: exclude slack_start_new_thread from plan mode

The breakout tool can dispatch a fresh agent run that starts outside the
current plan-mode state, bypassing the approval flow. Add it to
PLAN_MODE_EXCLUDED_TOOLS so it's hidden alongside the other mutating
tools while planning.

---------

Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* chore: require bun for ui agent work (#1639)

Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: request actions read for sandbox logs (#1642)

* fix: request actions read for sandbox logs

Request optional Actions read permission for sandbox proxy tokens, with fallback for installations that have not approved it yet. Update setup docs and prompt guidance for safe GitHub Actions log usage.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: restore actions:read scope after workflow push

After an approved workflow push, the guard was restoring the proxy with
BASE_RUNTIME_PROXY_TOKEN_PERMISSIONS, which excludes the actions: read
scope this PR adds. Restore with RUNTIME_PROXY_TOKEN_PERMISSIONS (which
includes actions: read) and fall back to BASE if the install hasn't
granted Actions read — mirroring the pattern in _create_sandbox_with_proxy.

Addresses review comment on PR #1642.

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: widen split review diffs (#1647)

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* chore: install missing deps before verification (#1646)

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* chore: switch ui to pnpm (#1645)

* chore: require pnpm for ui agent work

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* chore: switch ui to pnpm

Replace Bun and Yarn lockfiles with pnpm lockfile and update UI/Vercel commands to use pnpm.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* ci: use corepack for ui pnpm e2e build

Run pnpm through Corepack in the E2E global setup so CI can use the pinned package manager without a separate pnpm install step.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: add Sonnet 5 to model picker (#1651)

* chore: update Sonnet examples to Sonnet 5

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* chore: add Sonnet 5 to model picker

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* Remove dead breakout-thread e2e scenario after dropping the tool

The merge resolution deferred upstream's Slack breakout-thread tool
(slack_start_new_thread, #1638) since it depends on the #1621 dispatch
module, but the e2e harness still scripted it. Removing the tool name
from fake_llm.py's _tool_step call left a malformed scenario, crashing
the langgraph-dev web server at import (TypeError: _tool_step() missing
'call_id') and failing Playwright E2E.

Drop the "breakout" script scenario, its _is_breakout_request helper +
ScriptRule, and the corresponding full_flow.spec.ts test.

* Revert upstream pnpm switch; keep bun for the UI build

The merge auto-adopted upstream's pnpm switch (#1645) in tests/e2e/
global-setup.ts and ui/package.json, but our fork builds the UI with
bun (vercel.json + the E2E workflow's setup-bun). That left the
Playwright globalSetup running `corepack pnpm install --frozen-lockfile`
with no pnpm-lock.yaml, failing E2E at UI build time.

Revert global-setup.ts and ui/package.json to the dev (bun) baseline,
drop the merge-added ui/pnpm-lock.yaml, and remove the re-added
ui/AGENTS.md (our fork had deleted it).

* Align plan-review e2e + UI with the HEAD (pre-#1635) backend

The merge left a split plan vertical: the backend save_plan/plan_api are
HEAD (we deferred the editable-plan/sandbox-publish features #1610/#1635/
#1637 per #80), but the plan UI and e2e harness were upstream's. The
fake_llm scenario called save_plan(plan_file_path=...) — upstream's
file-based #1635 contract — while HEAD save_plan takes plan_markdown,
so the plan never saved and PlanReview never rendered (E2E failure on
the plan-review locator).

Pass plan_markdown to save_plan, and revert PlanReview.tsx / plan.ts /
$threadId_.plan.tsx / plan_review.spec.ts to the dev baseline so the
whole plan flow (save -> render -> approve -> implement) is consistent
with the HEAD backend.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Ramon Nogueira <ramon.nogueira@langchain.dev>
Co-authored-by: Ramon Nogueira <270434257+ramon-langchain@users.noreply.github.com>
Co-authored-by: Caroline di Vittorio <43390382+carolinedivittorio@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
Co-authored-by: Mukil Loganathan <mukil@langchain.dev>
2026-06-30 16:45:19 -04:00

766 lines
28 KiB
Python

"""Unit tests for the add_finding / update_finding / list_findings tools."""
from __future__ import annotations
from collections.abc import Iterator
from typing import Any
from unittest.mock import AsyncMock, patch
import pytest
from agent.tools.add_finding import add_finding
from agent.tools.list_findings import list_findings
from agent.tools.resolve_finding_thread import resolve_finding_thread
from agent.tools.update_finding import update_finding
@pytest.fixture(autouse=True)
def _stub_resolve_review_head_sha() -> Iterator[None]:
"""Resolve the review head from the run config (no thread-metadata fetch).
Mirrors the production fallback when metadata carries no head, keeping these
unit tests offline. Tests that exercise the metadata-override path patch
``resolve_review_head_sha`` themselves.
"""
def _head(thread_id: str, configurable: dict[str, Any]) -> str:
head = configurable.get("head_sha")
return head if isinstance(head, str) else ""
with (
patch("agent.tools.add_finding.resolve_review_head_sha", AsyncMock(side_effect=_head)),
patch("agent.tools.update_finding.resolve_review_head_sha", AsyncMock(side_effect=_head)),
):
yield
def _config(**configurable_overrides: Any) -> dict[str, Any]:
base: dict[str, Any] = {
"configurable": {
"thread_id": "tid-1",
"head_sha": "sha-head",
"diff_text": "",
"diff_line_set": {
"foo.py": {"RIGHT": set(range(10, 41)), "LEFT": set()},
},
},
"metadata": {},
}
base["configurable"].update(configurable_overrides)
return base
def _existing_finding(**overrides: Any) -> dict[str, Any]:
finding: dict[str, Any] = {"id": "f_a", "status": "open"}
finding.update(overrides)
return finding
async def test_add_finding_rejects_invalid_severity() -> None:
with patch("agent.tools.add_finding.get_config", return_value=_config()):
result = await add_finding(
severity="trivial",
confidence="high",
category="x",
file="foo.py",
title="Generated title",
description="d",
start_line=11,
end_line=11,
)
assert result["success"] is False
assert "severity" in result["error"].lower()
async def test_add_finding_rejects_empty_title() -> None:
with patch("agent.tools.add_finding.get_config", return_value=_config()):
result = await add_finding(
severity="high",
confidence="high",
category="correctness",
file="foo.py",
title=" ",
description="d",
start_line=11,
end_line=11,
)
assert result["success"] is False
assert "title" in result["error"].lower()
async def test_add_finding_rejects_out_of_diff_lines() -> None:
captured: list[Any] = []
async def fake_append(_thread_id: str, finding: Any) -> None:
captured.append(finding)
with (
patch("agent.tools.add_finding.get_config", return_value=_config()),
patch("agent.tools.add_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch("agent.tools.add_finding.append_finding", side_effect=fake_append),
):
result = await add_finding(
severity="high",
confidence="high",
category="correctness",
file="foo.py",
title="Generated title",
description="d",
start_line=99,
end_line=99,
)
assert result["success"] is False
assert result["in_diff"] is False
assert "disabled" in result["error"].lower()
assert captured == []
async def test_add_finding_accepts_left_side_anchor_on_old_line() -> None:
"""A finding on a deleted (LEFT-side) line must validate against the
old-side line set, not the new-side. With only RIGHT lines in 10..40,
a LEFT anchor at the same number should still pass when the line is in
the old-side set."""
config = {
"configurable": {
"thread_id": "tid-1",
"head_sha": "sha-head",
"diff_text": "",
"diff_line_set": {
"foo.py": {"RIGHT": {10, 11, 12}, "LEFT": {50, 51}},
},
},
"metadata": {},
}
with (
patch("agent.tools.add_finding.get_config", return_value=config),
patch("agent.tools.add_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch("agent.tools.add_finding.append_finding", new_callable=AsyncMock),
):
result = await add_finding(
severity="high",
confidence="high",
category="correctness",
file="foo.py",
title="Release resources removed",
description="deleted call to releaseResources()",
start_line=51,
end_line=51,
side="LEFT",
)
assert result["success"] is True
async def test_add_finding_left_anchor_outside_old_side_set_rejected() -> None:
"""A LEFT anchor on a line that's not in the old-side hunk is rejected —
out-of-diff findings are disabled, validated on the correct side."""
config = {
"configurable": {
"thread_id": "tid-1",
"head_sha": "sha-head",
"diff_text": "",
"diff_line_set": {
"foo.py": {"RIGHT": {10, 11, 12}, "LEFT": {50, 51}},
},
},
"metadata": {},
}
with (
patch("agent.tools.add_finding.get_config", return_value=config),
patch("agent.tools.add_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch("agent.tools.add_finding.append_finding", new_callable=AsyncMock),
):
result = await add_finding(
severity="high",
confidence="high",
category="correctness",
file="foo.py",
title="Generated title",
description="d",
start_line=99,
end_line=99,
side="LEFT",
)
assert result["success"] is False
assert result["in_diff"] is False
async def test_add_finding_rejects_invalid_confidence() -> None:
with patch("agent.tools.add_finding.get_config", return_value=_config()):
result = await add_finding(
severity="high",
confidence="certain",
category="correctness",
file="foo.py",
title="Generated title",
description="d",
start_line=11,
end_line=11,
)
assert result["success"] is False
assert "confidence" in result["error"].lower()
async def test_add_finding_persists_to_thread_metadata() -> None:
captured: list[Any] = []
async def fake_append(thread_id: str, finding: Any) -> Any:
captured.append((thread_id, finding))
return finding
with (
patch("agent.tools.add_finding.get_config", return_value=_config()),
patch("agent.tools.add_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch("agent.tools.add_finding.append_finding", side_effect=fake_append),
):
result = await add_finding(
severity="medium",
confidence="high",
category="style",
file="foo.py",
title="Rename breaks reference",
description="rename",
start_line=11,
end_line=12,
suggestion="renamed = 1",
)
assert result["success"] is True
assert "finding_id" in result
persisted_thread, persisted = captured[0]
assert persisted_thread == "tid-1"
assert persisted["title"] == "Rename breaks reference"
assert persisted["file"] == "foo.py"
assert persisted["start_line"] == 11
assert persisted["end_line"] == 12
assert persisted["suggestion"] == "renamed = 1"
assert persisted["status"] == "open"
assert persisted["first_seen_sha"] == "sha-head"
assert persisted["confidence"] == "high"
async def test_add_finding_uses_resolved_head_sha_for_provenance() -> None:
"""A net-new finding filed during a mid-run re-review must record the live
head (from thread metadata), not the stale head frozen in the run config."""
captured: list[Any] = []
async def fake_append(thread_id: str, finding: Any) -> Any:
captured.append(finding)
return finding
with (
patch("agent.tools.add_finding.get_config", return_value=_config()),
patch("agent.tools.add_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch(
"agent.tools.add_finding.resolve_review_head_sha",
AsyncMock(return_value="freshhead"),
),
patch("agent.tools.add_finding.append_finding", side_effect=fake_append),
):
result = await add_finding(
severity="medium",
confidence="high",
category="style",
file="foo.py",
title="Rename breaks reference",
description="rename",
start_line=11,
end_line=12,
)
assert result["success"] is True
assert captured[0]["first_seen_sha"] == "freshhead"
assert captured[0]["last_confirmed_sha"] == "freshhead"
async def test_add_finding_allows_file_level_with_no_lines() -> None:
with (
patch("agent.tools.add_finding.get_config", return_value=_config()),
patch("agent.tools.add_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch(
"agent.tools.add_finding.append_finding",
new_callable=AsyncMock,
side_effect=lambda _t, f: f,
),
):
result = await add_finding(
severity="low",
confidence="medium",
category="style",
file="missing.py",
title="File-level issue",
description="file-level note",
)
assert result["success"] is True
async def test_update_finding_rejects_invalid_status() -> None:
with patch("agent.tools.update_finding.get_config", return_value=_config()):
result = await update_finding(finding_id="f_x", status="archived")
assert result["success"] is False
async def test_resolve_finding_thread_resolves_all_known_threads() -> None:
finding = {
"id": "f1",
"status": "open",
"github_review_thread_ids": ["THREAD_1", "THREAD_2"],
"github_review_comment_ids": [11, 12],
}
update = AsyncMock(return_value={**finding, "status": "resolved"})
resolve = AsyncMock(return_value=True)
reply = AsyncMock(return_value={"id": 999})
with (
patch(
"agent.tools.resolve_finding_thread.get_config",
return_value=_config(repo={"owner": "o", "name": "r"}, pr_number=7),
),
patch("agent.tools.resolve_finding_thread.get_github_token", return_value="token"),
patch("agent.tools.resolve_finding_thread.get_thread_id_from_runtime", return_value="tid"),
patch("agent.tools.resolve_finding_thread.get_finding", AsyncMock(return_value=finding)),
patch("agent.tools.resolve_finding_thread.resolve_review_thread", resolve),
patch("agent.tools.resolve_finding_thread.reply_to_review_comment", reply),
patch("agent.tools.resolve_finding_thread.update_finding_fields", update),
):
result = await resolve_finding_thread(
"f1", status="resolved", note="Fixed in the latest commit"
)
assert result["success"] is True
assert result["resolved_thread_count"] == 2
assert [call.kwargs["thread_node_id"] for call in resolve.await_args_list] == [
"THREAD_1",
"THREAD_2",
]
assert [call.kwargs["review_comment_id"] for call in reply.await_args_list] == [11, 12]
assert all(
call.kwargs["body"] == "Fixed in the latest commit" for call in reply.await_args_list
)
updates = update.await_args.args[2]
assert updates["github_thread_resolved"] is True
assert updates["github_resolved_thread_ids"] == ["THREAD_1", "THREAD_2"]
assert updates["github_posted_resolution_comment_ids"] == [11, 12]
assert updates["resolution_note"] == "Fixed in the latest commit"
async def test_resolve_finding_thread_requires_note() -> None:
with patch(
"agent.tools.resolve_finding_thread.get_config",
return_value=_config(repo={"owner": "o", "name": "r"}, pr_number=7),
):
result = await resolve_finding_thread("f1", note=" ", status="resolved")
assert result["success"] is False
assert "requires a note" in result["error"]
async def test_update_finding_rejects_empty_update() -> None:
with patch("agent.tools.update_finding.get_config", return_value=_config()):
result = await update_finding(finding_id="f_x")
assert result["success"] is False
assert "No fields" in result["error"]
async def test_update_finding_requires_note_for_resolution() -> None:
with patch("agent.tools.update_finding.get_config", return_value=_config()):
result = await update_finding(finding_id="f_x", status="resolved")
assert result["success"] is False
assert "requires a note" in result["error"]
async def test_update_finding_updates_title() -> None:
captured: list[Any] = []
async def fake_update(thread_id: str, finding_id: str, updates: Any) -> Any:
captured.append(updates)
return {"id": finding_id, **updates}
with (
patch("agent.tools.update_finding.get_config", return_value=_config()),
patch("agent.tools.update_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch(
"agent.tools.update_finding.list_findings",
AsyncMock(return_value=[_existing_finding()]),
),
patch("agent.tools.update_finding.update_finding_fields", side_effect=fake_update),
):
result = await update_finding(finding_id="f_a", title="new generated title")
assert result["success"] is True
assert captured[0]["title"] == "new generated title"
async def test_add_finding_drops_long_suggestion() -> None:
captured: list[Any] = []
async def fake_append(thread_id: str, finding: Any) -> Any:
captured.append(finding)
return finding
long_suggestion = "\n".join(f"line_{i}" for i in range(6))
with (
patch("agent.tools.add_finding.get_config", return_value=_config()),
patch("agent.tools.add_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch("agent.tools.add_finding.append_finding", side_effect=fake_append),
):
result = await add_finding(
severity="medium",
confidence="high",
category="style",
file="foo.py",
title="Rewrite changes behavior",
description="rewrite",
start_line=11,
end_line=12,
suggestion=long_suggestion,
)
assert result["success"] is True
assert result.get("suggestion_dropped") is True
assert "warning" in result
assert captured[0]["suggestion"] is None
async def test_add_finding_keeps_short_suggestion() -> None:
captured: list[Any] = []
async def fake_append(thread_id: str, finding: Any) -> Any:
captured.append(finding)
return finding
short_suggestion = "a\nb\nc\nd" # exactly 4 lines — at the cap
with (
patch("agent.tools.add_finding.get_config", return_value=_config()),
patch("agent.tools.add_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch("agent.tools.add_finding.append_finding", side_effect=fake_append),
):
result = await add_finding(
severity="medium",
confidence="medium",
category="style",
file="foo.py",
title="Rename breaks reference",
description="rename",
start_line=11,
end_line=12,
suggestion=short_suggestion,
)
assert result["success"] is True
assert "suggestion_dropped" not in result
assert captured[0]["suggestion"] == short_suggestion
async def test_add_finding_preserves_multi_line_range() -> None:
"""Multi-line ranges are preserved end-to-end (no collapse to start_line)."""
captured: list[Any] = []
async def fake_append(thread_id: str, finding: Any) -> Any:
captured.append(finding)
return finding
with (
patch("agent.tools.add_finding.get_config", return_value=_config()),
patch("agent.tools.add_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch("agent.tools.add_finding.append_finding", side_effect=fake_append),
):
result = await add_finding(
severity="low",
confidence="low",
category="style",
file="foo.py",
title="Range spans issue",
description="span the relevant range",
start_line=15,
end_line=19,
)
assert result["success"] is True
assert captured[0]["start_line"] == 15
assert captured[0]["end_line"] == 19
async def test_update_finding_rejects_long_suggestion_without_clobbering() -> None:
"""Over-cap suggestion alongside other fields: drop suggestion, keep the rest."""
captured: list[Any] = []
async def fake_update(thread_id: str, finding_id: str, updates: Any) -> Any:
captured.append(updates)
return {"id": finding_id, **updates}
long_suggestion = "\n".join(f"line_{i}" for i in range(6))
with (
patch("agent.tools.update_finding.get_config", return_value=_config()),
patch("agent.tools.update_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch(
"agent.tools.update_finding.list_findings",
AsyncMock(return_value=[_existing_finding()]),
),
patch("agent.tools.update_finding.update_finding_fields", side_effect=fake_update),
):
result = await update_finding(
finding_id="f_a",
description="updated description",
suggestion=long_suggestion,
)
assert result["success"] is True
assert result.get("suggestion_dropped") is True
assert "suggestion" not in captured[0]
assert captured[0]["description"] == "updated description"
async def test_update_finding_long_suggestion_only_returns_failure() -> None:
"""Over-cap suggestion as the only field: fail outright rather than no-op."""
long_suggestion = "\n".join(f"line_{i}" for i in range(6))
with (
patch("agent.tools.update_finding.get_config", return_value=_config()),
patch("agent.tools.update_finding.get_thread_id_from_runtime", return_value="tid-1"),
):
result = await update_finding(finding_id="f_a", suggestion=long_suggestion)
assert result["success"] is False
assert result.get("suggestion_dropped") is True
assert "cap" in result["error"]
async def test_update_finding_empty_string_clears_suggestion() -> None:
captured: list[Any] = []
async def fake_update(thread_id: str, finding_id: str, updates: Any) -> Any:
captured.append(updates)
return {"id": finding_id, **updates}
with (
patch("agent.tools.update_finding.get_config", return_value=_config()),
patch("agent.tools.update_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch(
"agent.tools.update_finding.list_findings",
AsyncMock(return_value=[_existing_finding()]),
),
patch("agent.tools.update_finding.update_finding_fields", side_effect=fake_update),
):
result = await update_finding(finding_id="f_a", suggestion="")
assert result["success"] is True
assert captured[0]["suggestion"] is None
async def test_update_finding_passes_through_fields() -> None:
captured: list[Any] = []
async def fake_update(thread_id: str, finding_id: str, updates: Any) -> Any:
captured.append((thread_id, finding_id, updates))
return {"id": finding_id, **updates}
with (
patch("agent.tools.update_finding.get_config", return_value=_config()),
patch("agent.tools.update_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch(
"agent.tools.update_finding.list_findings",
AsyncMock(return_value=[_existing_finding()]),
),
patch("agent.tools.update_finding.update_finding_fields", side_effect=fake_update),
):
result = await update_finding(
finding_id="f_a",
status="resolved",
note="addressed by new commit",
)
assert result["success"] is True
_t, fid, updates = captured[0]
assert fid == "f_a"
assert updates["status"] == "resolved"
assert updates["last_update_note"] == "addressed by new commit"
assert updates["resolution_note"] == "addressed by new commit"
async def test_update_finding_resolves_github_thread_when_pr_context_available() -> None:
cfg = _config(repo={"owner": "o", "name": "r"}, pr_number=7)
with (
patch("agent.tools.update_finding.get_config", return_value=cfg),
patch("agent.tools.update_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch(
"agent.tools.update_finding.list_findings",
AsyncMock(return_value=[_existing_finding(github_review_thread_id="THREAD_1")]),
),
patch("agent.tools.resolve_finding_thread.get_config", return_value=cfg),
patch("agent.tools.resolve_finding_thread.get_github_token", return_value="token"),
patch("agent.tools.update_finding.update_finding_fields", AsyncMock()) as update,
patch(
"agent.tools.resolve_finding_thread._resolve_finding_thread_async",
new_callable=AsyncMock,
return_value={
"success": True,
"finding": {"id": "f_a", "status": "resolved", "github_thread_resolved": True},
"resolved_thread_count": 1,
},
) as resolve_async,
):
result = await update_finding(
finding_id="f_a",
status="resolved",
note="The latest commit adds the missing guard.",
)
assert result["success"] is True
assert result["github_resolution"]["success"] is True
assert result["finding"]["github_thread_resolved"] is True
resolve_async.assert_awaited_once()
update.assert_not_awaited()
async def test_update_finding_leaves_open_when_github_resolution_fails() -> None:
cfg = _config(repo={"owner": "o", "name": "r"}, pr_number=7)
with (
patch("agent.tools.update_finding.get_config", return_value=cfg),
patch("agent.tools.update_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch(
"agent.tools.update_finding.list_findings",
AsyncMock(return_value=[_existing_finding(github_review_thread_id="THREAD_1")]),
),
patch("agent.tools.resolve_finding_thread.get_config", return_value=cfg),
patch("agent.tools.resolve_finding_thread.get_github_token", return_value="token"),
patch("agent.tools.update_finding.update_finding_fields", AsyncMock()) as update,
patch(
"agent.tools.resolve_finding_thread._resolve_finding_thread_async",
new_callable=AsyncMock,
return_value={
"success": False,
"error": "Could not resolve GitHub review thread id",
},
) as resolve_async,
):
result = await update_finding(
finding_id="f_a",
status="resolved",
note="The latest commit adds the missing guard.",
)
assert result["success"] is False
assert "left open" in result["error"]
assert result["github_resolution"]["error"] == "Could not resolve GitHub review thread id"
resolve_async.assert_awaited_once()
update.assert_not_awaited()
async def test_update_finding_resolves_hidden_finding_locally() -> None:
captured: list[Any] = []
async def fake_update(thread_id: str, finding_id: str, updates: Any) -> Any:
captured.append((thread_id, finding_id, updates))
return {"id": finding_id, **updates}
cfg = _config(repo={"owner": "o", "name": "r"}, pr_number=7)
with (
patch("agent.tools.update_finding.get_config", return_value=cfg),
patch("agent.tools.update_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch(
"agent.tools.update_finding.list_findings",
AsyncMock(return_value=[_existing_finding()]),
),
patch("agent.tools.update_finding.update_finding_fields", side_effect=fake_update),
patch(
"agent.tools.resolve_finding_thread._resolve_finding_thread_async",
new_callable=AsyncMock,
) as resolve_async,
):
result = await update_finding(
finding_id="f_a",
status="resolved",
note="The latest commit adds the missing guard.",
)
assert result["success"] is True
_thread_id, _finding_id, updates = captured[0]
assert updates["status"] == "resolved"
assert updates["resolution_note"] == "The latest commit adds the missing guard."
resolve_async.assert_not_awaited()
async def test_list_findings_filters_by_status() -> None:
findings = [
{"id": "f_a", "status": "open"},
{"id": "f_b", "status": "resolved"},
{"id": "f_c", "status": "open"},
]
async def fake_list(_thread_id: str) -> list[Any]:
return findings
cfg = _config()
with (
patch("agent.tools.list_findings.get_thread_id_from_runtime", return_value="tid-1"),
patch("agent.tools.list_findings.list_findings_async", side_effect=fake_list),
patch("agent.tools.add_finding.get_config", return_value=cfg),
):
result = await list_findings(status_filter="open")
assert result["count"] == 2
assert [f["id"] for f in result["findings"]] == ["f_a", "f_c"]
async def test_list_findings_returns_all_when_filter_omitted() -> None:
findings = [{"id": "f_a", "status": "open"}, {"id": "f_b", "status": "resolved"}]
async def fake_list(_thread_id: str) -> list[Any]:
return findings
with (
patch("agent.tools.list_findings.get_thread_id_from_runtime", return_value="tid-1"),
patch("agent.tools.list_findings.list_findings_async", side_effect=fake_list),
):
result = await list_findings()
assert result["count"] == 2
async def test_add_finding_returns_structured_error_when_thread_missing() -> None:
"""A missing reviewer thread must come back as a do-not-retry tool result,
not a raised exception the agent retries against 10-30 times."""
from agent.reviewer_findings import ReviewerThreadMissingError
async def fake_append(thread_id: str, finding: Any) -> Any:
raise ReviewerThreadMissingError(thread_id, RuntimeError("thread X not found"))
with (
patch("agent.tools.add_finding.get_config", return_value=_config()),
patch("agent.tools.add_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch("agent.tools.add_finding.append_finding", side_effect=fake_append),
):
result = await add_finding(
severity="medium",
confidence="high",
category="correctness",
file="foo.py",
title="Rename breaks reference",
description="rename",
start_line=11,
)
assert result["success"] is False
assert result["error"] == "thread_not_found"
assert result["thread_id"] == "tid-1"
assert "Do not retry" in result["note"]
async def test_update_finding_returns_structured_error_when_thread_missing() -> None:
from agent.reviewer_findings import ReviewerThreadMissingError
async def fake_update(thread_id: str, finding_id: str, updates: Any) -> Any:
raise ReviewerThreadMissingError(thread_id, RuntimeError("thread X not found"))
with (
patch("agent.tools.update_finding.get_config", return_value=_config()),
patch("agent.tools.update_finding.get_thread_id_from_runtime", return_value="tid-1"),
patch(
"agent.tools.update_finding.list_findings",
AsyncMock(return_value=[_existing_finding()]),
),
patch("agent.tools.update_finding.update_finding_fields", side_effect=fake_update),
):
result = await update_finding(finding_id="f_a", status="resolved", note="fixed")
assert result["success"] is False
assert result["error"] == "thread_not_found"
assert result["thread_id"] == "tid-1"