open-swe/tests/e2e/e2e_env.py

84 lines
3.1 KiB
Python
Raw Normal View History

test(open-swe): add Playwright E2E for the Slack → PR → web handoff (#1583) * test(open-swe): add Playwright E2E for the Slack → PR → web handoff Local, secrets-free end-to-end suite that drives the full happy path through mock Slack/GitHub control panels and the real dashboard UI. Only the LLM and external SaaS HTTP boundaries (GitHub/Slack APIs, OAuth token mint) are faked — the real process_slack_mention, get_agent, deepagents loop, tools, middleware, and dashboard authorization all run under `langgraph dev` with a scripted fake chat model and a local temp-dir sandbox. - full_flow: a Slack mention runs the agent, which implements a change in the sandbox, opens a PR against a fake GitHub remote, and replies with the PR link in the same thread. - dashboard: clicking the bot's real "Open in Web" link loads the built ui/ app (served same-origin); the thread owner can continue the conversation, while a different user sees the same thread read-only (no composer). Wired into Agent CI as a `Playwright E2E` job that runs on pull requests. * fix(open-swe): serve E2E UI assets via explicit route; pin Playwright The dashboard E2E served the built ui/ SPA's /assets via app.mount(StaticFiles), but LangGraph's custom-app loader serves APIRoutes and drops sub-app Mounts, so /assets 404'd under `langgraph dev` in CI — the React app never booted and the composer/transcript never rendered. Serve assets via an explicit route instead. Also pin @playwright/test to the latest (1.61.0) for reproducible runs, and make the owner composer assertion tolerant of either hydration state. * test(open-swe): record Playwright trace + video on every E2E run Capture a replayable trace (DOM snapshots, network, console, source) and a screen recording for every test, not just retries, plus a screenshot on failure. The CI job already uploads playwright-report/ and test-results/, so each run now has a downloadable replay; documented how to open it.
2026-06-22 15:54:46 -04:00
"""Shared environment + constants for the full-flow E2E.
Imported FIRST by both the agent graph entrypoint and the HTTP harness, before
any ``agent.*`` module — several webapp/auth/slack constants are read into module
globals at import time, so the env must be set beforehand.
Everything here only configures *boundaries* (which sandbox, which fake API
URLs, where git writes its global config). The agent code itself is unchanged.
"""
from __future__ import annotations
import os
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[2]
TMP = Path(os.environ.setdefault("E2E_TMP", str(Path(__file__).parent / ".e2e-tmp")))
# Demo repo the fake GitHub serves and the agent operates on.
OWNER = "fakeorg"
REPO = "demo"
BASE_BRANCH = "main"
FEATURE_BRANCH = "add-greet"
PR_TITLE = "Add greet() helper"
FEATURE_FILE = "greet.py"
# Fixed Slack identifiers so the mock UI and assertions are deterministic.
BOT_USER_ID = "U0BOT"
BOT_USERNAME = "open-swe"
DEMO_CHANNEL = "C_DEMO"
HUMAN_USER = "U_HUMAN"
PORT = os.environ.setdefault("E2E_PORT", "2024")
BASE_URL = os.environ.setdefault("E2E_BASE", f"http://127.0.0.1:{PORT}")
_GH_DIR = TMP / "github"
_WORK_DIR = TMP / "work"
BARE_REMOTE = _GH_DIR / f"{OWNER}__{REPO}.git"
_DEFAULTS = {
# Sandbox: real local provider, rooted in a throwaway temp dir.
"SANDBOX_TYPE": "local",
"LOCAL_SANDBOX_ROOT_DIR": str(_WORK_DIR),
# Keep git's --global writes (bot identity) out of the user's ~/.gitconfig.
"GIT_CONFIG_GLOBAL": str(TMP / "gitconfig-global"),
"GIT_CONFIG_SYSTEM": "/dev/null",
# Path the scripted agent clones from (a local bare repo = "fake GitHub").
"E2E_REMOTE": str(BARE_REMOTE),
# Webhook signing + bot identity.
"GITHUB_WEBHOOK_SECRET": "test-github-secret",
"SLACK_SIGNING_SECRET": "test-slack-secret",
"SLACK_BOT_TOKEN": "xoxb-test-token",
"SLACK_BOT_USER_ID": BOT_USER_ID,
"SLACK_BOT_USERNAME": BOT_USERNAME,
# Slack runs resolve the repo from this when the channel/thread carry none.
"DEFAULT_REPO_OWNER": OWNER,
"DEFAULT_REPO_NAME": REPO,
# Bot-token-only mode: lets Slack runs proceed without a per-user OAuth token.
"LANGSMITH_API_KEY_PROD": "test-bot-mode",
# SDK client target (same dev server).
"LANGGRAPH_URL": BASE_URL,
# Dashboard: the "Open in Web" link target + session-cookie signing. Use
# 127.0.0.1 (not localhost) so the local-dev LLM-key check stays skipped.
"DASHBOARD_BASE_URL": BASE_URL,
"DASHBOARD_API_BASE_URL": BASE_URL,
"DASHBOARD_ALLOWED_ORIGINS": BASE_URL,
"DASHBOARD_JWT_SECRET": "test-dashboard-jwt-secret",
}
# The Slack run's triggering user (fake users.info returns this email) — the
# thread owner. A session with this email may continue the thread on the web.
SAME_USER = {"login": "dev-user", "email": "dev@example.com"}
# Any other logged-in user: can view the thread read-only, cannot post.
OTHER_USER = {"login": "someone-else", "email": "someone@example.com"}
for _k, _v in _DEFAULTS.items():
os.environ.setdefault(_k, _v)
for _d in (TMP, _GH_DIR, _WORK_DIR):
_d.mkdir(parents=True, exist_ok=True)
FAKE_GITHUB_API = f"{BASE_URL}/fake-gh"
FAKE_SLACK_API = f"{BASE_URL}/fake-slack"