Skip to content

perf: skip VLM thinking on image inspection - #158

Merged
suguanYang merged 1 commit into
stagingfrom
perf/wangbinqi/inspect-disable-vlm-thinking
Aug 25, 2026
Merged

perf: skip VLM thinking on image inspection#158
suguanYang merged 1 commit into
stagingfrom
perf/wangbinqi/inspect-disable-vlm-thinking

Conversation

@suguanYang

Copy link
Copy Markdown
Contributor

Summary

  • Disable thinking on inspect VLM calls so page OCR does not wait on reasoning tokens.
  • Qwen 3.5 Flash: enableThinking: false. Gemini 3 Flash: thinkingLevel: "minimal" (lowest it allows).
  • Same Tesla page inspect dropped from ~20–40s (Qwen default thinking) to ~8–11s; structured output still succeeded.

Test plan

  • Ask a page/table question on staging that cites a page image (inspect must run before finalize).
  • Confirm chat: image inspection requestresponse is closer to ~8–15s than ~30s.
  • Confirm highlight boxes and cited numbers still look right.
  • After staging merge, promote to prod (main) via the usual staging → main PR.

Made with Cursor

Inspect is OCR and provenance boxes, not multi-step reasoning. Qwen 3.5 Flash thinks by default and that was 20–40s; turning thinking off dropped it to ~8–11s without a clear quality loss.

Co-authored-by: Cursor <cursoragent@cursor.com>
@vercel

vercel Bot commented Aug 25, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
knowhere-notebook-staging Ready Ready Preview Aug 25, 2026 11:02am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
knowhere-notebook Ignored Ignored Aug 25, 2026 11:02am

Request Review

@suguanYang
suguanYang merged commit 79a5753 into staging Aug 25, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant