open-swe/agent/utils
Johannes du Plessis b19804536c
feat: handle images sent to non-vision models in Slack, Linear, and web UI (#1560)
* feat: handle images sent to non-vision models in Slack, Linear, and web UI

Add vision capability checks across all image input paths. When a user
sends images to a text-only model (e.g. GLM 5.2, DeepSeek V4 Pro), the
images are now skipped and a warning is injected into the prompt instead
of sending unsupported content to the model.

- Slack: resolve model at webhook time, skip image fetch + add warning
- Linear: same pattern as Slack
- Queued message middleware: read resolved model from thread metadata,
  strip images from queued payloads for text-only models
- Web UI: disable submit + show inline warning when images are attached
  to a non-vision model selection
- Shared: resolve_agent_model_id helper + vision_not_supported_warning

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* test: mock resolve_agent_model_id in Slack mention test

The test_process_slack_mention_queues_active_thread_message test was
missing a mock for the new resolve_agent_model_id call added to the
Slack webhook handler, causing a TypeError when image URLs triggered
the model resolution path.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

* fix: include vision warning in queued payload for text-only models

Update the prompt variable (not just content_blocks) before clearing
image_urls so the queued payload also carries the warning text when a
Slack/Linear follow-up arrives while the thread is busy.

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
2026-06-17 09:12:52 -07:00
..
agents_md.py feat: inline AGENTS.md into reviewer system prompt (#1328) 2026-05-22 14:58:27 -07:00
analyzer_skills.py feat: outcomes dataset + bootstrap/continual split via skills (#1365) 2026-06-01 13:25:12 -07:00
api_standards_skill.py feat: apply API standards skill in PR reviews for API changes (#1452) 2026-06-08 14:37:04 -07:00
auth.py feat: add scheduled web agents (#1422) 2026-06-05 02:20:24 +00:00
authorship.py feat: link PR attribution footer to the originating thread (#1539) 2026-06-16 09:50:18 -07:00
comments.py chore: Drop monorepo (#1029) 2026-03-06 16:10:34 -08:00
dashboard_links.py fix: point reviewer "Open in Web" link to the review page (#1519) 2026-06-12 14:15:58 -07:00
github_app.py fix: scope public reviewer tokens (#1389) 2026-06-03 09:25:17 -07:00
github_checks.py feat: CI auto-fix and PR babysitting for agent PRs (#1530) 2026-06-15 13:53:50 -07:00
github_ci.py feat: CI auto-fix and PR babysitting for agent PRs (#1530) 2026-06-15 13:53:50 -07:00
github_comments.py feat: track PR lifecycle state per thread for sidebar (#1492) 2026-06-11 12:21:59 -07:00
github_feedback.py feat: outcomes dataset + bootstrap/continual split via skills (#1365) 2026-06-01 13:25:12 -07:00
github_org_membership.py fix: Lock dashboard login to GitHub org members (#1367) 2026-06-01 20:56:30 +00:00
github_proxy.py fix: refresh sandbox GitHub proxy token before mid-run expiry (#1496) 2026-06-11 10:59:21 -07:00
github_token.py fix: stop storing GitHub tokens in metadata (#1405) 2026-06-04 09:33:51 -07:00
langsmith.py fix: point View trace link at the correct tracing project (#1521) 2026-06-13 09:37:15 -07:00
linear.py fix: use thread-level LangSmith URLs instead of run-level URLs to fix broken trace links (#1217) 2026-04-23 13:46:28 -07:00
linear_team_repo_map.py chore: Add langsmith deployments data plane linear project (#1044) 2026-03-09 18:36:00 -07:00
model.py feat(open-swe): stream agent chat via @langchain/react v2 protocol (#1475) 2026-06-11 09:54:35 -07:00
multimodal.py feat: handle images sent to non-vision models in Slack, Linear, and web UI (#1560) 2026-06-17 09:12:52 -07:00
repo.py feat: extract repo parsing into shared util, add linear comment repo override (#1103) 2026-03-20 13:34:00 -07:00
repo_prep.py fix: reviewer can silently review a stale checkout on reused sandboxes (#1503) 2026-06-11 13:42:10 -07:00
reviewer_outcomes.py feat: outcomes dataset + bootstrap/continual split via skills (#1365) 2026-06-01 13:25:12 -07:00
sandbox.py fix: Reduce graph load and dashboard repo failures (#1412) 2026-06-04 13:54:06 -07:00
sandbox_paths.py fix: better custom backend support (#1071) 2026-03-17 11:55:36 -07:00
sandbox_state.py fix: keep sandbox backend stable across recovery (#1294)w 2026-05-11 16:03:38 -07:00
slack.py feat: link Slack thread/Linear ticket in PRs and append ticket to title (#1504) 2026-06-11 15:52:40 -07:00
slack_feedback.py feat: outcomes dataset + bootstrap/continual split via skills (#1365) 2026-06-01 13:25:12 -07:00
thread_ops.py feat(open-swe): stream agent chat via @langchain/react v2 protocol (#1475) 2026-06-11 09:54:35 -07:00
tracing.py feat: route graphs to separate LangSmith tracing projects (#1508) 2026-06-11 17:57:16 -07:00