This repository has been archived on 2026-08-04. You can view files and clone it, but cannot push or open issues or pull requests.
orchestrator/agent-team/agent_team/status_page.py
Adam Moussa a632a7df7d fix(agent-team): scrub exception text from /api/state + /api/topology errors
/sh-security-review confirmed SEC-DASH-001 (low): build_snapshot embedded str(exc)
of a sqlite/OS error into the /api/state payload, leaking the absolute DB path /
table names to the unauthenticated LAN surface. Return only type(exc).__name__
(matching the task_detail hardening); apply the same to /api/topology's error
branch (SEC-DASH-003). Full exception detail stays in server-side logs.
2026-06-23 17:28:22 -04:00

582 lines
22 KiB
Python

"""LAN-only, READ-ONLY web status dashboard for the agent-team coordinator.
The agent-team is a single durable LangGraph pipeline; "agents" here are the
per-stage model roles, and what an operator actually wants to see is *which task
is in which phase* and *which tasks are blocked on the human gate*. This module
serves that view as a tiny self-refreshing HTML page so Adam can glance at the
queue without SSHing to the R720 and running ``run-team.py list``.
Posture (read this before changing anything):
* **READ-ONLY.** The page exposes no mutating endpoints. The SQLite ledger is
opened read-only (``mode=ro`` / ``immutable``-safe URI) and never written. It
reads the same two durable sources the coordinator owns:
- ``pending_questions`` (the human-interaction lifecycle ledger) for "who is
waiting on what" — reused via :mod:`agent_team.ledger`.
- the LangGraph SQLite checkpoint (``channel_values`` holds ``task``,
``current_phase``, ``status``) for each task's phase/description — read
through the same :class:`SqliteSaver` the coordinator uses in
:func:`agent_team.graph.build_sqlite_checkpointer`.
* **Sensitive content.** Task descriptions are operator input and may contain
sensitive detail (repo names, ticket content, remediation context). The page
renders them in plain text with HTML-escaping but applies NO authentication.
It must therefore stay **LAN/VPN-only and never public**. The sh-secrev R720
VM (10.10.60.120, VLAN 60) has no public NIC and sits behind the UniFi
firewall, so binding ``0.0.0.0`` exposes it to the LAN/VPN only. No secrets,
tokens, channel refs, or answer bodies are rendered.
* **Fail-safe.** A missing, locked, or unreadable DB renders a friendly
"no data" page; the server never crashes on a bad read.
The page is a live visual pipeline map: server-rendered inline SVG of the
agent-team DAG (``INTAKE -> CLARIFY (human gate) -> PLAN <-> REVIEW ->
[BUILD -> VERIFY -> DISPATCH] -> DONE``), with each stage coloured by live
state and badged with a task count. Vanilla inline JS polls a JSON sidecar
endpoint (``GET /api/state``) every few seconds with ``fetch()`` and updates
the node states, counts, tooltips, and "last updated" clock *in place* — no
full-page reload, so hover and scroll survive the refresh. A ``<noscript>``
meta-refresh is the JS-disabled fallback. Everything (SVG + CSS + JS) is inline
in the served document; nothing is fetched from a CDN, because the R720 VM is
offline/LAN-only.
Why stdlib ``http.server`` and not FastAPI (which is already a dep): the page is
two read-only GETs (the HTML map and its JSON sidecar). FastAPI/uvicorn buys
nothing here (no auth, no async I/O) and would couple this LAN dashboard to the
optional WS1 HTTP-API dependency set. stdlib ``http.server`` keeps it
zero-new-deps and importable anywhere the ``agent_team`` package is, which also
keeps :func:`render_html` and :func:`snapshot_to_dict` trivially unit-testable
without binding a socket.
Config (all env, with defaults):
* ``AGENT_TEAM_DB`` — path to the SQLite ledger
(default ``agent-team/state/agent_team.sqlite`` relative to this package).
* ``AGENT_TEAM_STATUS_HOST`` — bind host (default ``0.0.0.0``; LAN/VPN-only by
network posture, see above).
* ``AGENT_TEAM_STATUS_PORT`` — bind port (default ``8770``).
"""
from __future__ import annotations
import os
import sqlite3
from collections import Counter
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
# The package root is .../agent-team/agent_team; the default ledger lives at
# .../agent-team/state/agent_team.sqlite (mirrors run-team.py's _DEFAULT_DB).
_PACKAGE_DIR = Path(__file__).resolve().parent
_DEFAULT_DB = _PACKAGE_DIR.parent / "state" / "agent_team.sqlite"
# Phases we treat as "still progressing" for the queue summary. PARKED/DONE/
# FAILED are terminal-ish; everything else is in flight.
_ACTIVE_STATUSES = {"active", "waiting_human"}
_PARKED_STATUSES = {"parked"}
# --- Pipeline map model -------------------------------------------------------
#
# The agent-team is one durable LangGraph DAG. These STAGES are the canonical
# flow we draw, in order, each carrying the model "agent"/role that owns it (per
# the design: CLARIFY + PLAN = Claude on subscription; REVIEW = GPT-4.1
# cross_reviewer; SCAN = Gemini scanner; BUILD = DeepSeek fast_coder; the HUMAN
# GATE = the Slack owner). ``phases`` lists the Phase enum value(s) that map a
# live task onto that stage; tasks are grouped onto stages by ``current_phase``.
#
# VERIFY/DISPATCH are gated/inert in the current deploy (P3 not live) but are
# drawn so the operator sees the full intended pipeline. SCAN is a role node
# (Gemini), not a distinct LangGraph phase, so no task ever lands on it — it
# renders as a labelled-but-idle capability the BUILD/VERIFY stages can call.
@dataclass(frozen=True)
class Stage:
"""One node in the rendered pipeline map.
``key`` is a stable id used by the SVG + JS. ``phases`` are the Phase enum
*values* whose tasks belong to this stage (empty for pure role nodes like
the human gate / scan that no ``current_phase`` ever names).
"""
key: str
label: str
agent: str # model / role that owns the stage
phases: tuple[str, ...] = ()
gated: bool = False # drawn but inert in the current deploy
role_node: bool = False # capability, not a Phase tasks land on
# Left-to-right pipeline order. Keep in sync with task_model.Phase.
STAGES: tuple[Stage, ...] = (
Stage("intake", "Intake", "coordinator", phases=("intake",)),
Stage("clarify", "Clarify", "Claude (sub)", phases=("clarify",)),
Stage("gate", "Human Gate", "Slack owner", role_node=True),
Stage("plan", "Plan", "Claude (sub)", phases=("plan",)),
Stage("review", "Review", "GPT-4.1 (cross)", phases=("review",)),
Stage("scan", "Scan", "Gemini", role_node=True),
Stage("build", "Build", "DeepSeek (fast)", phases=("build",), gated=True),
Stage("verify", "Verify", "Claude (sub)", phases=("verify",), gated=True),
Stage("dispatch", "Dispatch", "GitHub PR", gated=True, role_node=True),
Stage("done", "Done", "—", phases=("done",)),
)
# Phase value -> stage key, derived once from STAGES.
_PHASE_TO_STAGE: dict[str, str] = {
phase: stage.key for stage in STAGES for phase in stage.phases
}
@dataclass(frozen=True)
class TaskView:
"""A read-only, render-ready view of one pipeline task.
Built from the LangGraph checkpoint's ``channel_values`` joined to any open
pending question for that thread. ``short_id`` is the first 8 chars of the
thread_id for compact display; ``thread_id`` is retained for completeness.
"""
thread_id: str
short_id: str
task: str
current_phase: str
status: str
waiting: bool = False
waiting_since: str | None = None
@dataclass(frozen=True)
class Snapshot:
"""Everything :func:`render_html` needs, as plain data (no DB handle).
Keeping this pure and serializable is what makes the renderer unit-testable
without a socket or a live database.
"""
generated_at: str
db_path: str
ok: bool = True
error: str | None = None
tasks: list[TaskView] = field(default_factory=list)
question_counts: dict[str, int] = field(default_factory=dict)
recent_spend: list[dict[str, Any]] = field(default_factory=list)
spend_total_usd: float | None = None
budget_available: bool = False
@property
def active_count(self) -> int:
return sum(1 for t in self.tasks if t.status in _ACTIVE_STATUSES)
@property
def parked_count(self) -> int:
return sum(1 for t in self.tasks if t.status in _PARKED_STATUSES)
@property
def waiting_tasks(self) -> list[TaskView]:
return [t for t in self.tasks if t.waiting]
@property
def progressing_tasks(self) -> list[TaskView]:
return [t for t in self.tasks if not t.waiting]
def tasks_for_stage(self, stage_key: str) -> list[TaskView]:
"""Tasks whose ``current_phase`` maps onto ``stage_key``.
Role nodes (human gate / scan / dispatch) own no phase, so they get no
tasks here even when work is "logically" at the gate — a CLARIFY task
awaiting an answer stays on the CLARIFY node and is flagged
awaiting-human there, which is where the operator looks.
"""
return [
t for t in self.tasks if _PHASE_TO_STAGE.get(t.current_phase) == stage_key
]
def stage_state(self, stage: Stage) -> str:
"""Live state for a stage node: awaiting_human > active > parked > idle.
``awaiting_human`` wins so a blocked stage reads as blocked at a glance.
Role/gated nodes with no tasks fall through to ``idle``.
"""
members = self.tasks_for_stage(stage.key)
if not members:
return "idle"
if any(t.waiting for t in members):
return "awaiting_human"
if any(t.status in _PARKED_STATUSES for t in members):
return "parked"
if any(t.status in _ACTIVE_STATUSES for t in members):
return "active"
return "idle"
def _utc_now_iso() -> str:
"""Current UTC time as a stable, human-readable string."""
return datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
def _short(thread_id: str) -> str:
"""First 8 chars of a thread_id, for compact display."""
return thread_id[:8] if thread_id else "(none)"
def _ro_connect(db_path: Path) -> sqlite3.Connection:
"""Open ``db_path`` strictly READ-ONLY.
Uses the SQLite ``file:...?mode=ro`` URI so the connection can never create
or write the database. ``sqlite3.Row`` row factory matches the reuse readers
(:mod:`agent_team.ledger`). Raises if the file does not exist (``mode=ro``
will not create it) — the caller turns that into the friendly no-data page.
"""
uri = f"file:{db_path}?mode=ro"
conn = sqlite3.connect(uri, uri=True, timeout=2.0)
conn.row_factory = sqlite3.Row
return conn
def _enumerate_threads(conn: sqlite3.Connection) -> list[str]:
"""Return the distinct thread_ids the checkpointer has persisted.
The LangGraph ``SqliteSaver`` owns the ``checkpoints`` table; we read it
READ-ONLY. If the table does not exist yet (no task ever ran), returns ``[]``
rather than raising.
"""
try:
rows = conn.execute(
"SELECT DISTINCT thread_id FROM checkpoints ORDER BY thread_id"
).fetchall()
except sqlite3.OperationalError:
# No checkpoints table yet (fresh DB) — not an error for a status view.
return []
return [r["thread_id"] for r in rows if r["thread_id"]]
def _channel_values_for(saver: Any, thread_id: str) -> dict[str, Any]:
"""Read the latest checkpoint's ``channel_values`` for one thread.
Mirrors the access pattern the coordinator/graph rely on:
``saver.get({"configurable": {"thread_id": tid}})`` returns the latest
checkpoint, whose ``channel_values`` carries ``task`` / ``current_phase`` /
``status`` / ``qa_history``. ``.get`` may return a checkpoint mapping
directly or ``None`` if the thread has no committed checkpoint; we tolerate
both and never raise into the page.
"""
try:
checkpoint = saver.get({"configurable": {"thread_id": thread_id}})
except Exception:
return {}
if not checkpoint:
return {}
values = checkpoint.get("channel_values") if isinstance(checkpoint, dict) else None
return values if isinstance(values, dict) else {}
def build_snapshot(db_path: Path | str | None = None) -> Snapshot:
"""Read the ledger READ-ONLY and assemble a render-ready :class:`Snapshot`.
Fail-safe by contract: any DB problem (missing file, locked, missing tables,
optional langgraph package absent) yields ``ok=False`` with a short message
rather than raising. The renderer turns that into the friendly no-data page.
"""
resolved = Path(db_path) if db_path is not None else _resolve_db_path()
generated_at = _utc_now_iso()
if not resolved.exists():
return Snapshot(
generated_at=generated_at,
db_path=str(resolved),
ok=False,
error=f"ledger not found at {resolved} (no tasks have run yet?)",
)
conn: sqlite3.Connection | None = None
try:
conn = _ro_connect(resolved)
question_counts = _read_question_counts(conn)
open_by_thread = _read_open_questions(conn)
thread_ids = _enumerate_threads(conn)
tasks = _read_tasks(resolved, thread_ids, open_by_thread)
spend, spend_total, budget_available = _read_recent_spend(conn)
except sqlite3.OperationalError as exc:
# Locked / busy / unreadable — never crash, just say so. Return only the
# exception TYPE, never str(exc): a SQLite message can carry the DB path /
# table names, and this payload is served unauthenticated on the LAN
# (matches the task_detail hardening). [sh-security-review SEC-DASH-001]
return Snapshot(
generated_at=generated_at,
db_path=str(resolved),
ok=False,
error=f"could not read ledger (busy or unreadable): {type(exc).__name__}",
)
except Exception as exc: # defensive: any unexpected read failure
# Type only, never str(exc) — see SEC-DASH-001 note above.
return Snapshot(
generated_at=generated_at,
db_path=str(resolved),
ok=False,
error=f"unexpected error reading ledger: {type(exc).__name__}",
)
finally:
if conn is not None:
conn.close()
return Snapshot(
generated_at=generated_at,
db_path=str(resolved),
ok=True,
tasks=tasks,
question_counts=question_counts,
recent_spend=spend,
spend_total_usd=spend_total,
budget_available=budget_available,
)
def snapshot_to_dict(snapshot: Snapshot) -> dict[str, Any]:
"""Serialize a :class:`Snapshot` into the ``/api/state`` JSON payload.
Pure (no I/O) and self-contained so it is unit-testable without a socket.
The shape is the contract the inline JS poller depends on:
* ``ok`` / ``error`` / ``generated_at`` — header + clock + no-data handling.
* ``stages`` — one entry per pipeline node in draw order, each with its live
``state``, a task ``count``, and the member ``tasks`` (for the tooltip).
* ``tasks`` / ``waiting`` / ``question_counts`` / ``summary`` / ``budget`` —
the detail table + count cards the page also renders.
Task descriptions are operator input; they are emitted as raw strings here
and the *consumer* (the JS, via ``textContent``/JSON parsing) is responsible
for not interpreting them as markup. ``json.dumps`` itself escapes the
JSON-string context, and the renderer additionally hardens the inlined seed
against ``</script>`` / HTML-comment breakouts (see ``_json_for_script``).
"""
if not snapshot.ok:
return {
"ok": False,
"error": snapshot.error or "ledger unavailable",
"generated_at": snapshot.generated_at,
"db_path": snapshot.db_path,
"stages": [_stage_payload(s, []) for s in STAGES],
"tasks": [],
"waiting": [],
"question_counts": {},
"summary": {
"active": 0,
"parked": 0,
"waiting": 0,
"open_questions": 0,
"total": 0,
},
"budget": {"available": False},
}
stages = [
_stage_payload(s, snapshot.tasks_for_stage(s.key), snapshot.stage_state(s))
for s in STAGES
]
qc = snapshot.question_counts
return {
"ok": True,
"error": None,
"generated_at": snapshot.generated_at,
"db_path": snapshot.db_path,
"stages": stages,
"tasks": [_task_payload(t) for t in snapshot.tasks],
"waiting": [_task_payload(t) for t in snapshot.waiting_tasks],
"question_counts": qc,
"summary": {
"active": snapshot.active_count,
"parked": snapshot.parked_count,
"waiting": len(snapshot.waiting_tasks),
"open_questions": int(qc.get("open", 0)),
"total": len(snapshot.tasks),
},
"budget": {
"available": snapshot.budget_available,
"total_usd": snapshot.spend_total_usd,
"recent": snapshot.recent_spend,
},
}
def _task_payload(t: TaskView) -> dict[str, Any]:
"""JSON-safe view of one task (the tooltip + detail-table source)."""
return {
"thread_id": t.thread_id,
"short_id": t.short_id,
"task": t.task,
"current_phase": t.current_phase,
"status": t.status,
"waiting": t.waiting,
"waiting_since": t.waiting_since,
}
def _stage_payload(
stage: Stage,
members: list[TaskView],
state: str | None = None,
) -> dict[str, Any]:
"""JSON-safe view of one pipeline stage node."""
payload = asdict(stage)
payload["state"] = state if state is not None else "idle"
payload["count"] = len(members)
payload["tasks"] = [_task_payload(t) for t in members]
return payload
def _resolve_db_path() -> Path:
"""Resolve the ledger path from ``AGENT_TEAM_DB`` or the package default."""
env = os.environ.get("AGENT_TEAM_DB")
return Path(env) if env else _DEFAULT_DB
def _read_question_counts(conn: sqlite3.Connection) -> dict[str, int]:
"""Count pending_questions rows by status (open/answered/expired/...)."""
counts: Counter[str] = Counter()
try:
rows = conn.execute(
"SELECT status, COUNT(*) AS n FROM pending_questions GROUP BY status"
).fetchall()
except sqlite3.OperationalError:
return {}
for r in rows:
counts[r["status"]] = int(r["n"])
return dict(counts)
def _read_open_questions(conn: sqlite3.Connection) -> dict[str, str]:
"""Map thread_id -> earliest ``posted_at`` of an OPEN question on it.
An open question means that thread is blocked on the human gate. We keep the
oldest posted_at so the page can show how long it has been waiting.
"""
waiting: dict[str, str] = {}
try:
rows = conn.execute(
"SELECT thread_id, posted_at FROM pending_questions "
"WHERE status='open' ORDER BY posted_at ASC"
).fetchall()
except sqlite3.OperationalError:
return {}
for r in rows:
tid = r["thread_id"]
if tid and tid not in waiting:
waiting[tid] = r["posted_at"] or ""
return waiting
def _read_tasks(
db_path: Path,
thread_ids: list[str],
open_by_thread: dict[str, str],
) -> list[TaskView]:
"""Build a :class:`TaskView` per thread from the LangGraph checkpoint.
Opens its own READ-ONLY :class:`SqliteSaver` over the same DB file (the saver
needs its own connection / serde). If the optional langgraph checkpoint
package is absent, returns ``[]`` so the page degrades to ledger-only data
instead of crashing.
"""
if not thread_ids:
return []
saver_cm = _readonly_saver(db_path)
if saver_cm is None:
return []
tasks: list[TaskView] = []
try:
with saver_cm as saver:
for tid in thread_ids:
values = _channel_values_for(saver, tid)
status = str(values.get("status") or "unknown")
tasks.append(
TaskView(
thread_id=tid,
short_id=_short(tid),
task=str(values.get("task") or "(no description)"),
current_phase=str(values.get("current_phase") or "unknown"),
status=status,
waiting=tid in open_by_thread,
waiting_since=open_by_thread.get(tid) or None,
)
)
except Exception:
# Saver construction/read failed entirely — degrade to ledger-only.
return tasks
# Waiting tasks first, then by phase, for a sensible default ordering.
tasks.sort(key=lambda t: (not t.waiting, t.current_phase, t.short_id))
return tasks
def _readonly_saver(db_path: Path) -> Any:
"""Return an *un-entered* SqliteSaver context manager over a RO connection.
Reuses the same :class:`SqliteSaver` + serde the coordinator builds in
:func:`agent_team.graph.build_checkpoint_serde`, but over a ``mode=ro``
connection so this dashboard can never write the checkpoint tables. Returns
``None`` if the optional package is not installed (pre-deploy scaffolding /
test envs without the checkpoint dep), letting the caller degrade to
ledger-only data.
"""
try:
from contextlib import closing, contextmanager
from langgraph.checkpoint.sqlite import SqliteSaver
from agent_team.graph import build_checkpoint_serde
except Exception:
return None
serde = build_checkpoint_serde()
@contextmanager
def _cm() -> Any:
uri = f"file:{db_path}?mode=ro"
with closing(
sqlite3.connect(uri, uri=True, check_same_thread=False, timeout=2.0)
) as conn:
yield SqliteSaver(conn, serde=serde)
return _cm()
def _read_recent_spend(
conn: sqlite3.Connection,
) -> tuple[list[dict[str, Any]], float | None, bool]:
"""Read recent budget_ledger spend, if the table is present/readable.
Returns ``(recent_rows, total_usd, available)``. ``available`` is ``False``
when the table is absent (older DB) — the page then simply omits the budget
panel rather than showing a misleading zero.
"""
try:
rows = conn.execute(
"SELECT day_bucket, stage, model, usd_cost, recorded_at "
"FROM budget_ledger ORDER BY recorded_at DESC LIMIT 10"
).fetchall()
total_row = conn.execute(
"SELECT COALESCE(SUM(usd_cost), 0.0) AS total FROM budget_ledger"
).fetchone()
except sqlite3.OperationalError:
return [], None, False
recent = [
{
"day_bucket": r["day_bucket"],
"stage": r["stage"],
"model": r["model"],
"usd_cost": float(r["usd_cost"] or 0.0),
"recorded_at": r["recorded_at"],
}
for r in rows
]
total = float(total_row["total"]) if total_row is not None else 0.0
return recent, total, True
# NOTE: HTML/SVG rendering and the stdlib http.server were retired in the WebUI
# makeover. The dashboard is now a React/Vite SPA served by agent_team.dashboard
# (FastAPI). This module is kept as the read-only DATA LAYER: build_snapshot()
# and snapshot_to_dict() feed the dashboard's /api/state endpoint.