Skip to content

Support Qwen3-VL-4B with hidden states extraction and compression script - #2

Open
WhXcjm wants to merge 1 commit into
MercuryB1:hessianfrom
WhXcjm:hessian
Open

Support Qwen3-VL-4B with hidden states extraction and compression script#2
WhXcjm wants to merge 1 commit into
MercuryB1:hessianfrom
WhXcjm:hessian

Conversation

@WhXcjm

@WhXcjm WhXcjm commented Mar 31, 2026

Copy link
Copy Markdown

Introduce functionality to extract hidden states from decoder layer outputs and add a new compression script for Qwen3-VL-4B.

Copilot AI review requested due to automatic review settings March 31, 2026 03:27

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for extracting decoder hidden states across different decoder-layer output formats (needed for Qwen3-VL-4B), and introduces a new convenience script for compressing/evaluating Qwen3-VL-4B.

Changes:

  • Add a shared helper to consistently extract hidden states from decoder layer outputs during calibration/block search.
  • Update block forward paths to use the new hidden-state extraction helper instead of assuming tuple outputs.
  • Add a Qwen3-VL-4B compression shell script and ignore a new submissions/ output folder.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.

File Description
scripts/compress_qwen3vl4b_jsq_hessian.sh Adds a runnable script for compressing Qwen3-VL-4B with JSQ settings and running lmms-eval tasks.
jsq/compression/collector.py Adds _extract_hidden_states() and uses it when running decoder blocks to get correct hidden states across output formats.
jsq/compression/block_search.py Uses _extract_hidden_states() in the lightweight forward path during MA-JSQ block search.
.gitignore Ignores submissions/ directory.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread scripts/compress_qwen3vl4b_jsq_hessian.sh Outdated
@WhXcjm

WhXcjm commented Mar 31, 2026

Copy link
Copy Markdown
Author

@copilot review current codes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants