open-swe/agent/tools/read_finding_outcomes.py
Johannes du Plessis 4a55145bb1
feat: outcomes dataset + bootstrap/continual split via skills (#1365)
* fix: reset stale sandbox creation sentinel

Co-authored-by: Johannes du Plessis <51395795+johannes117@users.noreply.github.com>

* fix: treat SANDBOX_CREATING as a timestamped cross-process lock

Only reset the sentinel when proven stale (older than the creation
timeout); otherwise wait for the worker that holds the lock so a
concurrent run does not create a duplicate sandbox.

* feat(analyzer): outcomes dataset + bootstrap/continual split via skills

Rename the review_style_analyzer graph to `analyzer` and split it into two
modes, plus capture reviewer finding outcomes for continual learning.

- Outcomes dataset: upsert resolved-by-commit (positive), dismissed (false
  positive), and GitHub/Slack thumbs findings into a single LangSmith dataset
  (openswe-reviewer-outcomes), keyed deterministically per finding+source.
  Emit points wired into update_finding, resolve_finding_thread, and the
  GitHub/Slack reaction handlers.
- Two playbooks delivered as deepagents skills (bootstrap-repo-analysis,
  continual-learning), served as virtual files via a CompositeBackend /skills/
  route + StateBackend (seeded into the run files channel at invoke time, never
  written to the sandbox). Mode is set by the launcher; continual runs fall
  back to the GitHub App installation token.
- Split launcher into start_bootstrap_analysis + start_continual_run; register
  a per-repo nightly continual-learning cron when bootstrap completes.
- New read_finding_outcomes tool feeds confirmed/dismissed findings back to the
  continual playbook.

Tests for outcome label mapping, skills helper, and cron idempotency.

* fix(analyzer): anchor continual cron runs to a real thread_id

The nightly continual-learning cron is threadless, and get_analyzer
early-returns an empty agent when configurable.thread_id is missing — so
every cron-launched run no-op'd before reading outcomes or saving a refined
prompt. Include the repo's deterministic analyzer thread_id in the continual
run configurable so the run executes; the threadless run carries no message
history, so nightly runs don't accumulate context.

* refactor(analyzer): move cron lifecycle calls out of the review-styles store

Drop the inline `analyzer_cron` imports from review_styles.py (added only to
dodge a circular import) by relocating the cron-trigger calls to the layer
above the store: registration to the save_review_style tool (after a prompt is
saved) and removal to the dashboard delete route. review_styles.py is now a
pure store again with top-level imports only.

* refactor: hoist reviewer_outcomes imports to module level

Move the two inline emit_finding_status_outcome imports introduced in this PR
(update_finding, resolve_finding_thread) to top-level imports. reviewer_outcomes
only depends on langsmith, so there is no circular import to avoid.

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
2026-06-01 13:25:12 -07:00

39 lines
1.4 KiB
Python

"""Tool: read this reviewer's past finding outcomes for the repo under analysis.
Surfaces findings that were later confirmed (resolved by a commit / 👍) vs
dismissed (false positive / 👎), so the analyzer can promote the bug patterns
this team actually fixes and add the noisy ones to a skip-list.
"""
from __future__ import annotations
from typing import Any
from langgraph.config import get_config
from ..utils.reviewer_outcomes import read_outcomes_for_repo
def read_finding_outcomes(limit: int = 60) -> dict[str, Any]:
"""Return confirmed and dismissed past findings for the repo being analyzed.
Call this before synthesizing the style prompt. Use the ``confirmed``
findings to reinforce what to hunt for and the ``dismissed`` findings to
build the "do not flag" list.
"""
config = get_config()
configurable = config.get("configurable") or {}
full_name = configurable.get("review_style_full_name")
if not isinstance(full_name, str) or "/" not in full_name:
return {"ok": False, "error": "no repo under analysis", "confirmed": [], "dismissed": []}
outcomes = read_outcomes_for_repo(full_name, limit=limit)
confirmed = outcomes["confirmed"]
dismissed = outcomes["dismissed"]
return {
"ok": True,
"repo": full_name,
"counts": {"confirmed": len(confirmed), "dismissed": len(dismissed)},
"confirmed": confirmed,
"dismissed": dismissed,
}