open-swe/agent/middleware
Johannes du Plessis b19804536c
feat: handle images sent to non-vision models in Slack, Linear, and web UI (#1560)
* feat: handle images sent to non-vision models in Slack, Linear, and web UI

Add vision capability checks across all image input paths. When a user
sends images to a text-only model (e.g. GLM 5.2, DeepSeek V4 Pro), the
images are now skipped and a warning is injected into the prompt instead
of sending unsupported content to the model.

- Slack: resolve model at webhook time, skip image fetch + add warning
- Linear: same pattern as Slack
- Queued message middleware: read resolved model from thread metadata,
  strip images from queued payloads for text-only models
- Web UI: disable submit + show inline warning when images are attached
  to a non-vision model selection
- Shared: resolve_agent_model_id helper + vision_not_supported_warning

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* test: mock resolve_agent_model_id in Slack mention test

The test_process_slack_mention_queues_active_thread_message test was
missing a mock for the new resolve_agent_model_id call added to the
Slack webhook handler, causing a TypeError when image URLs triggered
the model resolution path.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: include vision warning in queued payload for text-only models

Update the prompt variable (not just content_blocks) before clearing
image_urls so the queued payload also carries the warning text when a
Slack/Linear follow-up arrives while the thread is busy.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
2026-06-17 09:12:52 -07:00
..
__init__.py feat: let the agent stop naturally without forced tool calls (#1535) 2026-06-15 14:54:01 -07:00
check_message_queue.py feat: handle images sent to non-vision models in Slack, Linear, and web UI (#1560) 2026-06-17 09:12:52 -07:00
exclude_tools.py feat: add reviewer graph + eval target wiring (#1241) 2026-05-06 10:15:58 -07:00
model_fallback.py feat(open-swe): stream agent chat via @langchain/react v2 protocol (#1475) 2026-06-11 09:54:35 -07:00
notify_step_limit.py fix: notify users via Slack when agent hits model call step limit (#1204) 2026-05-01 14:24:25 -07:00
refresh_github_proxy.py fix: refresh sandbox GitHub proxy token before mid-run expiry (#1496) 2026-06-11 10:59:21 -07:00
refresh_slack_status.py fix Slack assistant status endpoint (#1272) 2026-05-08 13:07:32 -07:00
sandbox_circuit_breaker.py fix: recover from mid-run sandbox death (#1274) 2026-05-08 12:55:36 -07:00
sanitize_thinking_blocks.py fix: sanitize malformed Anthropic thinking blocks (#1357) 2026-05-28 16:15:13 -07:00
sanitize_tool_inputs.py fix: coerce malformed integer strings in read_file offset/limit params (#1216) 2026-05-01 14:29:48 -07:00
settle_review_check.py fix: settle incomplete review check as neutral, not failure (#1501) 2026-06-11 13:03:34 -07:00
tool_artifact.py feat(open-swe): stream agent chat via @langchain/react v2 protocol (#1475) 2026-06-11 09:54:35 -07:00
tool_error_handler.py fix: keep sandbox backend stable across recovery (#1294)w 2026-05-11 16:03:38 -07:00