open-swe/agent/dashboard
Johannes du Plessis eb18b07b20
feat: trigger reviewer evals from the admin page (#1524)
* feat: trigger reviewer evals from the admin page

Add an admin-only "Reviewer eval" section + endpoints that launch the
reviewer benchmark as an isolated subprocess against the running
deployment, with live status and the LangSmith experiment link. Route
eval traces to a dedicated open-swe-evals project so they stay out of
the production tracing project.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: reconcile reviewer eval status via heartbeat, not local process

The persisted record is shared across workers but _PROCS is process-local.
The owning worker now refreshes a heartbeat while the subprocess runs, and
status is only reconciled to failed once the heartbeat is stale, so a poll on
a worker without the local handle no longer kills a live run (and a duplicate
start is rejected across workers).

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
2026-06-15 09:48:12 -07:00
..
__init__.py feat: open-swe dashboard for per-user profile config (#1302) 2026-05-15 11:23:53 -07:00
admin.py feat: server-side Datadog/LangSmith observability tools + team creds [closes OPE-54] (#1476) 2026-06-10 11:07:42 -07:00
agent_instructions.py feat: per-repo custom instructions for the coding agent (#1460) 2026-06-09 15:46:29 -07:00
agent_overrides.py feat: Store-backed GitHub/Slack user mapping (self-service + admin) (#1369) 2026-06-01 14:37:19 -07:00
agent_usage.py Revert out-of-process usage-snapshot builder (#1473) 2026-06-09 13:19:11 -07:00
analyzer_cron.py feat: outcomes dataset + bootstrap/continual split via skills (#1365) 2026-06-01 13:25:12 -07:00
enabled_repos.py feat: restructure Open SWE Review tab + wire create_prs (#1319) 2026-05-21 09:17:07 -07:00
eval_jobs.py feat: trigger reviewer evals from the admin page (#1524) 2026-06-15 09:48:12 -07:00
oauth.py fix: auto-recover from expired GitHub refresh tokens (#1491) 2026-06-11 12:05:27 -07:00
options.py Remove Kimi K2.6 from supported model selector (#1522) 2026-06-15 09:39:08 -07:00
pr_diff.py feat: render Reviews page diffs with pierre MultiFileDiff (#1517) 2026-06-12 11:03:51 -07:00
profiles.py fix: auto-recover from expired GitHub refresh tokens (#1491) 2026-06-11 12:05:27 -07:00
repo_access.py feat: add scheduled web agents (#1422) 2026-06-05 02:20:24 +00:00
review_api.py perf: speed up Reviews list (My PRs + All PRs) (#1518) 2026-06-12 14:14:50 -07:00
review_style_jobs.py feat: outcomes dataset + bootstrap/continual split via skills (#1365) 2026-06-01 13:25:12 -07:00
review_styles.py feat: outcomes dataset + bootstrap/continual split via skills (#1365) 2026-06-01 13:25:12 -07:00
routes.py feat: trigger reviewer evals from the admin page (#1524) 2026-06-15 09:48:12 -07:00
schedules.py feat: add scheduled web agents (#1422) 2026-06-05 02:20:24 +00:00
slack_oauth.py feat: open Slack-triggered PRs as the triggering user (#1375) 2026-06-02 15:04:20 -07:00
team_credentials.py feat: server-side Datadog/LangSmith observability tools + team creds [closes OPE-54] (#1476) 2026-06-10 11:07:42 -07:00
team_settings.py feat: inject org-wide guidelines into reviewer prompt (#1431) 2026-06-05 13:49:30 -07:00
thread_api.py feat: mark threads as resolved to hide from sidebar (#1500) 2026-06-12 12:01:42 -07:00
user_mappings.py fix: use Slack OIDC mappings for Slack thread ownership (#1410) 2026-06-04 19:58:30 +00:00