open-swe/.github/workflows
Johannes du Plessis 0927f2dd9c
feat: Run reviewer eval in a GitHub Action; dashboard becomes read-only (#1556)
* Run reviewer eval in a GitHub Action; make dashboard a read-only progress view

The dashboard launched the eval as a subprocess inside the serving deployment
worker, so a container recycle killed long runs and discarded results that had
already completed server-side. Move the harness to a workflow_dispatch Action
(run on prod). run_eval now publishes status/progress/log-tail to the LangGraph
store record the dashboard reads, so /admin/evals stays a live view; a killed
Action surfaces as failed via the stale-heartbeat reconcile.

* reviewer_eval workflow: pass inputs via env, no shell interpolation

Addresses the reviewer finding: workflow_dispatch string inputs were
interpolated into the run: block (limit unquoted), allowing shell injection in
a job holding LANGSMITH/ANTHROPIC keys. Pass inputs through env and reference
quoted "$VARS"; validate limit is numeric and build its flag in bash.

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
2026-06-16 19:38:36 -07:00
..
ci.yml chore(deps): bump astral-sh/setup-uv in the minor-and-patch group (#1232) 2026-05-01 18:00:53 -07:00
pr_lint.yml chore(deps): bump python-dotenv from 1.2.1 to 1.2.2 (#1212) 2026-05-01 22:27:21 +00:00
promote_main_to_prod.yml ci: nightly promote of main to prod for LangGraph deploys (#1285) 2026-05-09 01:15:26 +00:00
reviewer_eval.yml feat: Run reviewer eval in a GitHub Action; dashboard becomes read-only (#1556) 2026-06-16 19:38:36 -07:00