open-swe/tests
Johannes du Plessis 8e39f62122
feat: activate PR babysitting UI toggles for autofix and trigger mode (#1561)
* feat: activate PR babysitting UI toggles for autofix and trigger mode

Remove the "coming soon" gating on the Autofix Mode, Autofix Severity
Threshold, and Trigger Mode controls in the review settings page so
admins can enable CI auto-fix and review-comment resolution on PRs
that Open SWE opens. The backend (ci_autofix.py, webapp.py webhook
routing) was already fully wired — only the UI was disabled.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: simplify autofix to on/off toggle, remove severity threshold

Replace the four-level AutofixMode (off/low/medium/high) and the
autofix_severity_threshold setting with a single boolean
autofix_enabled toggle. The severity threshold was leftover from the
reviewer finding-severity model and does not apply to CI autofix;
the agent should fix any failing CI and resolve any comments on PRs
it opens.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: move autofix toggle to per-user profile, remove team-level setting

The autofix toggle is now per-user (auto_fix_ci in the user profile)
instead of team-level (admin-only). This uses the existing auto_fix_ci
field that was already in ProfileUpdate but never wired up.

Changes:
- ci_autofix.py: check per-user auto_fix_ci profile flag after
  resolving the agent thread's github_login, instead of checking
  team-level autofix_enabled before knowing the PR
- webapp.py: removed early is_autofix_enabled() webhook gates; the
  per-user check now happens in ci_autofix.py once the thread is found
- team_settings.py: removed autofix_enabled field, is_autofix_enabled()
- cloud-agents.tsx: enabled the auto_fix_ci toggle (was comingSoon)
- review.tsx: removed the admin-level autofix switch
- Updated tests and AGENTS.md

The agent graph (not the reviewer) is what gets dispatched - this was
already correct in ci_autofix.py line 223: client.runs.create(
thread_id, "agent", ...).

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* feat: batch PR babysitting events

Remove the leftover trigger-mode gate from PR babysitting and batch new CI/review events while an agent run is already active so the running agent can handle the latest PR state before finishing. Also moves review-feedback permission checks behind the per-user opt-out and applies the auto-fix profile gate to merge-conflict babysitting.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: consume batched babysitting events

Teach the agent queue middleware to turn pending PR babysitting metadata into an injected instruction for the active run, so batched CI/review events are not dropped while still avoiding duplicate run creation.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: address review findings in PR babysitting batching

- Route batched events through the LangGraph store (read in-process by the
  message-queue middleware) instead of a per-model-call threads.get on every
  agent thread.
- Only record an attempt / mark the head SHA handled on a real dispatch, not
  on a batch, so an event isn't permanently dropped if the in-flight run ends
  before consuming it.
- Carry the reviewer's comment through batched review feedback instead of
  replacing it with a generic re-check nudge.

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
2026-06-17 14:12:04 -07:00
..
middleware fix: keep sandbox backend stable across recovery (#1294)w 2026-05-11 16:03:38 -07:00
conftest.py Remove reviewer env allowlist (#1353) 2026-05-28 14:29:33 -07:00
test_account_link.py fix: use Slack OIDC mappings for Slack thread ownership (#1410) 2026-06-04 19:58:30 +00:00
test_agent_instructions.py feat: per-repo custom instructions for the coding agent (#1460) 2026-06-09 15:46:29 -07:00
test_agent_schedules.py feat: add scheduled web agents (#1422) 2026-06-05 02:20:24 +00:00
test_agent_subagent_models.py feat(open-swe): stream agent chat via @langchain/react v2 protocol (#1475) 2026-06-11 09:54:35 -07:00
test_agent_thread_pr_state.py feat: track PR lifecycle state per thread for sidebar (#1492) 2026-06-11 12:21:59 -07:00
test_agent_usage.py Revert out-of-process usage-snapshot builder (#1473) 2026-06-09 13:19:11 -07:00
test_analyzer_cron.py feat: outcomes dataset + bootstrap/continual split via skills (#1365) 2026-06-01 13:25:12 -07:00
test_analyzer_skills.py feat: outcomes dataset + bootstrap/continual split via skills (#1365) 2026-06-01 13:25:12 -07:00
test_anthropic_effort.py feat(open-swe): stream agent chat via @langchain/react v2 protocol (#1475) 2026-06-11 09:54:35 -07:00
test_api_standards_skill.py feat: apply API standards skill in PR reviews for API changes (#1452) 2026-06-08 14:37:04 -07:00
test_auth_sources.py fix: stop storing GitHub tokens in metadata (#1405) 2026-06-04 09:33:51 -07:00
test_authorship.py feat: open Slack-triggered PRs as the triggering user (#1375) 2026-06-02 15:04:20 -07:00
test_autofix_state.py feat: activate PR babysitting UI toggles for autofix and trigger mode (#1561) 2026-06-17 14:12:04 -07:00
test_autofix_webhook.py feat: activate PR babysitting UI toggles for autofix and trigger mode (#1561) 2026-06-17 14:12:04 -07:00
test_check_message_queue.py feat: activate PR babysitting UI toggles for autofix and trigger mode (#1561) 2026-06-17 14:12:04 -07:00
test_ci_autofix.py feat: activate PR babysitting UI toggles for autofix and trigger mode (#1561) 2026-06-17 14:12:04 -07:00
test_currents_tools.py feat: add user-scoped Currents.dev API key for e2e test investigation (#1566) 2026-06-17 13:59:55 -07:00
test_dashboard_csrf.py fix: harden origin parsing and home prompt submit failure (#1499) 2026-06-11 11:54:25 -07:00
test_dashboard_links.py fix: point reviewer "Open in Web" link to the review page (#1519) 2026-06-12 14:15:58 -07:00
test_dashboard_org_login_gate.py fix: Lock dashboard login to GitHub org members (#1367) 2026-06-01 20:56:30 +00:00
test_dashboard_repo_optional.py feat(open-swe): stream agent chat via @langchain/react v2 protocol (#1475) 2026-06-11 09:54:35 -07:00
test_dashboard_repos.py fix: Reduce graph load and dashboard repo failures (#1412) 2026-06-04 13:54:06 -07:00
test_dashboard_reviews.py perf: speed up Reviews list (My PRs + All PRs) (#1518) 2026-06-12 14:14:50 -07:00
test_dashboard_run_email.py fix: use Slack OIDC mappings for Slack thread ownership (#1410) 2026-06-04 19:58:30 +00:00
test_dashboard_thread_api.py feat: handle images sent to non-vision models in Slack, Linear, and web UI (#1560) 2026-06-17 09:12:52 -07:00
test_dashboard_thread_api_activity.py fix: restore org-wide read access to agent threads (#1474) 2026-06-09 16:41:07 -07:00
test_dashboard_web_handoff.py feat(open-swe): stream agent chat via @langchain/react v2 protocol (#1475) 2026-06-11 09:54:35 -07:00
test_daytona_integration.py fix(daytona): make sandbox snapshot configurable (#1220) 2026-05-01 22:51:34 +00:00
test_encryption.py feat: support TOKEN_ENCRYPTION_KEY rotation via MultiFernet [closes AB-2323] (#1275) 2026-05-08 14:29:23 -07:00
test_eval_jobs.py feat: Run reviewer eval in a GitHub Action; dashboard becomes read-only (#1556) 2026-06-16 19:38:36 -07:00
test_eval_store_reporter.py feat: Run reviewer eval in a GitHub Action; dashboard becomes read-only (#1556) 2026-06-16 19:38:36 -07:00
test_fireworks_model.py Remove Kimi K2.6 from supported model selector (#1522) 2026-06-15 09:39:08 -07:00
test_github_app.py fix: scope public reviewer tokens (#1389) 2026-06-03 09:25:17 -07:00
test_github_checks.py feat: CI auto-fix and PR babysitting for agent PRs (#1530) 2026-06-15 13:53:50 -07:00
test_github_ci.py feat: CI auto-fix and PR babysitting for agent PRs (#1530) 2026-06-15 13:53:50 -07:00
test_github_comment_prompts.py feat: link PR attribution footer to the originating thread (#1539) 2026-06-16 09:50:18 -07:00
test_github_feedback.py feat: reconcile reviewer comment lifecycle (#1332) 2026-05-26 16:24:34 -07:00
test_github_issue_webhook.py feat: surface dashboard UI link on PR reviews [INF-0000] (#1440) 2026-06-07 05:17:09 +00:00
test_github_oauth_refresh.py fix: auto-recover from expired GitHub refresh tokens (#1491) 2026-06-11 12:05:27 -07:00
test_github_proxy_refresh.py fix: refresh sandbox GitHub proxy token before mid-run expiry (#1496) 2026-06-11 10:59:21 -07:00
test_github_token_ttl.py fix: stop storing GitHub tokens in metadata (#1405) 2026-06-04 09:33:51 -07:00
test_google_model.py feat: add Gemini 3.5 Flash provider (#1420) 2026-06-04 17:38:54 -07:00
test_http_security.py fix: harden http_request SSRF guard against DNS rebinding [closes AB-2321] (#1277) 2026-05-08 22:06:25 +00:00
test_langsmith_sandbox_config.py feat: add idle TTL and delete-after-stop sandbox lifecycle controls (#1265) 2026-05-08 00:30:14 -04:00
test_langsmith_sandbox_timeout.py fix: enforce client-side deadline on sandbox execute (#1451) 2026-06-08 11:16:08 -07:00
test_langsmith_trace_url.py fix: point View trace link at the correct tracing project (#1521) 2026-06-13 09:37:15 -07:00
test_local_integration.py fix: auto-create local sandbox root dir (#1402) 2026-06-03 23:47:08 +00:00
test_model_fallback_middleware.py feat(open-swe): stream agent chat via @langchain/react v2 protocol (#1475) 2026-06-11 09:54:35 -07:00
test_model_fallback_resolution.py feat: upgrade default agent + reviewer model to Opus 4.8 (#1350) 2026-05-28 10:59:29 -07:00
test_multimodal.py feat: handle images sent to non-vision models in Slack, Linear, and web UI (#1560) 2026-06-17 09:12:52 -07:00
test_normalize_repo.py feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312) 2026-05-20 18:35:00 +00:00
test_notify_step_limit_middleware.py fix: notify users via Slack when agent hits model call step limit (#1204) 2026-05-01 14:24:25 -07:00
test_observability_tools.py feat: server-side Datadog/LangSmith observability tools + team creds [closes OPE-54] (#1476) 2026-06-10 11:07:42 -07:00
test_open_pull_request.py feat: link Slack thread/Linear ticket in PRs and append ticket to title (#1504) 2026-06-11 15:52:40 -07:00
test_pr_ready_auto_review.py fix: reviewer resolves GitHub App token at run start, not via cross-process cache (#1409) 2026-06-04 12:11:23 -07:00
test_prompt_default_repo.py fix: make default repository configurable (#1429) 2026-06-05 13:48:47 -07:00
test_proxy_auth.py fix: refresh sandbox GitHub proxy token before mid-run expiry (#1496) 2026-06-11 10:59:21 -07:00
test_public_repo_org_gate.py fix: remove manual review trigger surfaces (#1396) 2026-06-03 12:33:22 -07:00
test_recent_comments.py chore: Drop monorepo (#1029) 2026-03-06 16:10:34 -08:00
test_refresh_slack_status_middleware.py feat: add optional Slack Assistants API typing status indicator (#1269) 2026-05-08 10:21:55 -07:00
test_repo_extraction.py feat: resolve Slack repo from channel topic/purpose (#1453) 2026-06-08 13:53:16 -07:00
test_repo_prep.py fix: reviewer can silently review a stale checkout on reused sandboxes (#1503) 2026-06-11 13:42:10 -07:00
test_review_api.py feat: AI-sorted PR review view with diff grouping (#1544) 2026-06-16 13:59:37 -07:00
test_review_chat.py feat: chat with your PR on the review page (#1534) 2026-06-15 17:17:30 -07:00
test_review_style_collector.py feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312) 2026-05-20 18:35:00 +00:00
test_review_style_sync.py fix: review style prompts UX, stale runs, and OAuth refresh (#1321) 2026-05-21 17:44:04 +00:00
test_review_styles_store.py feat: tune reviewer for precision — web/wiki tools + recalibrated prompt (#1312) 2026-05-20 18:35:00 +00:00
test_reviewer.py fix: reviewer can silently review a stale checkout on reused sandboxes (#1503) 2026-06-11 13:42:10 -07:00
test_reviewer_diff.py fix: fetch PR diff via GitHub API to re-enable add_finding validation (#1339) 2026-05-27 17:03:33 +00:00
test_reviewer_eval_run.py Reviewer eval admin: configurable runs + stacked form layout (#1540) 2026-06-16 10:12:35 -07:00
test_reviewer_eval_target.py feat: Run reviewer eval in a GitHub Action; dashboard becomes read-only (#1556) 2026-06-16 19:38:36 -07:00
test_reviewer_findings.py fix: honest publish_review reporting + structured thread-not-found errors (#1481) 2026-06-10 10:44:13 -07:00
test_reviewer_groups.py fix: Simplify review explanation: full-width, plain prose, no diff links (#1547) 2026-06-16 15:32:24 -07:00
test_reviewer_outcomes.py feat: outcomes dataset + bootstrap/continual split via skills (#1365) 2026-06-01 13:25:12 -07:00
test_reviewer_publish.py feat: disable out-of-diff reviewer findings (#1516) 2026-06-12 10:19:30 -07:00
test_reviewer_reconcile.py feat: let reviewer set comment titles (#1356) 2026-05-28 16:04:38 -07:00
test_reviewer_tools.py feat: disable out-of-diff reviewer findings (#1516) 2026-06-12 10:19:30 -07:00
test_reviewer_watch.py fix: keep review check on follow-up commits, drop neutral conclusion (#1486) 2026-06-10 13:33:34 -07:00
test_sandbox_paths.py fix: better custom backend support (#1071) 2026-03-17 11:55:36 -07:00
test_sanitize_thinking_blocks.py fix: sanitize malformed Anthropic thinking blocks (#1357) 2026-05-28 16:15:13 -07:00
test_sanitize_tool_inputs.py fix: coerce malformed integer strings in read_file offset/limit params (#1216) 2026-05-01 14:29:48 -07:00
test_slack_assistants_status.py feat: add Slack Block Kit reply options (#1407) 2026-06-04 10:26:09 -07:00
test_slack_context.py feat: handle images sent to non-vision models in Slack, Linear, and web UI (#1560) 2026-06-17 09:12:52 -07:00
test_slack_feedback.py feat: add Slack reaction feedback to LangSmith (#1231) 2026-05-08 14:24:12 -07:00
test_slack_oauth.py feat: open Slack-triggered PRs as the triggering user (#1375) 2026-06-02 15:04:20 -07:00
test_slack_thread_reply_tool.py feat: add Slack Block Kit reply options (#1407) 2026-06-04 10:26:09 -07:00
test_stale_sandbox_creating.py fix: refresh sandbox GitHub proxy token before mid-run expiry (#1496) 2026-06-11 10:59:21 -07:00
test_team_credentials.py feat: server-side Datadog/LangSmith observability tools + team creds [closes OPE-54] (#1476) 2026-06-10 11:07:42 -07:00
test_team_settings_grouping.py feat: AI-sorted PR review view with diff grouping (#1544) 2026-06-16 13:59:37 -07:00
test_team_settings_org_guidelines.py feat: chat with your PR on the review page (#1534) 2026-06-15 17:17:30 -07:00
test_tool_artifact_middleware.py feat(open-swe): stream agent chat via @langchain/react v2 protocol (#1475) 2026-06-11 09:54:35 -07:00
test_user_credentials.py feat: add user-scoped Currents.dev API key for e2e test investigation (#1566) 2026-06-17 13:59:55 -07:00
test_user_mappings.py fix: use Slack OIDC mappings for Slack thread ownership (#1410) 2026-06-04 19:58:30 +00:00