Router misroutes generic small coding tasks to fast_coder instead of implementer #7

New issue

Closed

opened 2026-06-16 19:15:46 +00:00 by amoussa1229 · 1 comment

amoussa1229 commented

2026-06-16 19:15:46 +00:00

(Migrated from github.com)

Summary

graph.router_node routes "Write a Python function that validates email addresses using a regex" to fast_coder (DeepSeek), but the routing golden expects implementer (Claude). Deterministic (fails 3/3).

Intended policy (from the golden set in `tests/test_routing_golden.py`)

Generic coding tasks with a clear spec → implementer.
fast_coder only on an explicit speed/cost cue — the three fast_coder goldens all say "Quick small task using DeepSeek…", "Fast bounded coding job…", "Use the cheap fast coder…".

The failing task has no speed cue, so by that policy it should be implementer. The router over-weighted "small/bounded one-function task" and picked fast_coder. It's a borderline case that straddles the implementer/fast_coder line.

Repro

.venv/bin/python -m pytest -q "tests/test_routing_golden.py::test_router_picks_expected_agent[Write a Python function that validates email addresses using a regex-implementer]"

(Live test — needs ANTHROPIC_API_KEY etc. Not CI-gated: CI runs lint + test-collect only.)

Impact — low / quality-only

The orchestrator still routes and executes; this one borderline class of task lands on DeepSeek instead of Claude (cheaper/faster model on a small well-specified job). Nothing hangs or errors.
Does not affect the security-review / R720 Path B system — run_headless.py, review.sh, and nightly_sweep.sh don't call graph.py; the optional (off-by-default) xmodel hook sends a different task. So the nightly sweep is unaffected.

Fix options (pick one)

Preferred: tighten the router so generic coding without an explicit speed/cost cue → implementer (matches the documented delegation policy).
Relax the golden if DeepSeek is acceptable for tiny well-specified helpers.

Context

Pre-existing on main; surfaced (not introduced) during the security-review Phase 3 finalize work (PR #6). Pick up in a later session after #6 merges.

## Summary `graph.router_node` routes **"Write a Python function that validates email addresses using a regex"** to `fast_coder` (DeepSeek), but the routing golden expects `implementer` (Claude). Deterministic (fails 3/3). ## Intended policy (from the golden set in `tests/test_routing_golden.py`) - Generic coding tasks with a clear spec → **`implementer`**. - **`fast_coder` only on an explicit speed/cost cue** — the three `fast_coder` goldens all say *"Quick small task using DeepSeek…"*, *"Fast bounded coding job…"*, *"Use the cheap fast coder…"*. The failing task has no speed cue, so by that policy it should be `implementer`. The router over-weighted "small/bounded one-function task" and picked `fast_coder`. It's a borderline case that straddles the implementer/fast_coder line. ## Repro ``` .venv/bin/python -m pytest -q "tests/test_routing_golden.py::test_router_picks_expected_agent[Write a Python function that validates email addresses using a regex-implementer]" ``` (Live test — needs `ANTHROPIC_API_KEY` etc. Not CI-gated: CI runs `lint` + `test-collect` only.) ## Impact — low / quality-only - The orchestrator still routes and executes; this one borderline class of task lands on DeepSeek instead of Claude (cheaper/faster model on a small well-specified job). Nothing hangs or errors. - **Does not affect the security-review / R720 Path B system** — `run_headless.py`, `review.sh`, and `nightly_sweep.sh` don't call `graph.py`; the optional (off-by-default) `xmodel` hook sends a different task. So the nightly sweep is unaffected. ## Fix options (pick one) 1. **Preferred:** tighten the router so generic coding *without* an explicit speed/cost cue → `implementer` (matches the documented delegation policy). 2. Relax the golden if DeepSeek is acceptable for tiny well-specified helpers. ## Context Pre-existing on `main`; surfaced (not introduced) during the security-review Phase 3 finalize work (PR #6). Pick up in a later session **after #6 merges**.

amoussa1229 commented

2026-07-14 23:03:22 +00:00

(Migrated from github.com)

The LangGraph router is retired with this repo; cross-family review re-homed as a direct CLI (cross_review.py) in Sea-Haven-Industries/security-review, which has no routing layer — misroute is moot. Closing.

This repo is archived. You cannot comment on issues.