Problem / Background
HostPreprocessorError::InvalidConfig in src/multimodal/host_preprocessor.rs:1074 is declared as:
#[error("invalid LLaVA host-preprocessor config: {0}")]
InvalidConfig(String),
Several of the messages constructed with that variant have nothing to do with a LLaVA checkpoint (line numbers against main at c3e134ba5):
host_preprocessor.rs:95, in XlaVisionBackendPolicy::from_value: MLXCEL_XLA_VISION_BACKEND must be auto, host, or iree; got {other:?}. This is family independent, and on the Qwen2-VL path it fires from from_env() after the family has been identified as Qwen2-VL, never LLaVA.
host_preprocessor.rs:158: Qwen2-VL XLA vision has no MLX fallback; MLXCEL_XLA_VISION_BACKEND=host is unsupported.
host_preprocessor.rs:178: Qwen2-VL XLA image execution requires the xla-iree feature (the #[cfg(not(feature = "xla-iree"))] arm).
Two related cases found while checking the above:
host_preprocessor.rs:150: failed to identify model family from {path}: {error}. This one fires before any family is known, so claiming LLaVA is guaranteed wrong.
host_preprocessor.rs:237: MLXCEL_XLA_VISION_BACKEND=iree requires the xla-iree feature. This is reachable only under load_llava_image_preprocessor, so the LLaVA prefix is not false, but the message is about an env var and a build feature rather than a config field, and the combined string reads oddly.
Impact is diagnostic only, but it is user facing. Both surfaces that load the preprocessor stringify the error:
src/backend/xla.rs:111 wraps it inside create_session as OpenXLA image preprocessor load failed: {error}.
src/server/batch/xla_preprocess.rs:74 maps the loader error with error.to_string() and passes it through the worker ready channel, which fails server startup with that text.
So for a Qwen2-VL checkpoint an operator sees:
OpenXLA image preprocessor load failed: invalid LLaVA host-preprocessor config: Qwen2-VL XLA image execution requires the xla-iree feature
They are told a LLaVA config is invalid while loading a Qwen2-VL checkpoint, which sends them looking in the wrong place.
Proposed Solution
Either of:
- Make the variant text family neutral, for example
invalid OpenXLA host-preprocessor config: {0}, or
- Split a family-specific variant off so LLaVA weight and config failures keep the LLaVA wording while the policy, family-detection, and Qwen2-VL arms get their own variant.
Prefer whichever keeps the LLaVA weight-loading and config messages as informative as they are today. WeightLoad (failed to load LLaVA host-preprocessor weights: {0}) is only produced from the LLaVA loader paths in src/loading/vlm_llava.rs, so it can stay as is unless the split makes a matching rename natural.
Acceptance Criteria
Technical Considerations
This was split out of #966 to keep that PR surgical. #966 deliberately left the wrapper text alone and only touched the Qwen2-VL loader contract; it does extend the message at host_preprocessor.rs:178 with the rebuild instruction (; rebuild mlxcel with --features xla-iree (this family has no MLX vision fallback)), so whoever picks this up should rebase on the merged #966 and preserve whatever final text landed there.
No behavior change is intended: the same conditions must still fail, with the same variant discriminant unless a new variant is introduced, and only the rendered prefix changes.
Problem / Background
HostPreprocessorError::InvalidConfiginsrc/multimodal/host_preprocessor.rs:1074is declared as:Several of the messages constructed with that variant have nothing to do with a LLaVA checkpoint (line numbers against
mainatc3e134ba5):host_preprocessor.rs:95, inXlaVisionBackendPolicy::from_value:MLXCEL_XLA_VISION_BACKEND must be auto, host, or iree; got {other:?}. This is family independent, and on the Qwen2-VL path it fires fromfrom_env()after the family has been identified as Qwen2-VL, never LLaVA.host_preprocessor.rs:158:Qwen2-VL XLA vision has no MLX fallback; MLXCEL_XLA_VISION_BACKEND=host is unsupported.host_preprocessor.rs:178:Qwen2-VL XLA image execution requires the xla-iree feature(the#[cfg(not(feature = "xla-iree"))]arm).Two related cases found while checking the above:
host_preprocessor.rs:150:failed to identify model family from {path}: {error}. This one fires before any family is known, so claiming LLaVA is guaranteed wrong.host_preprocessor.rs:237:MLXCEL_XLA_VISION_BACKEND=iree requires the xla-iree feature. This is reachable only underload_llava_image_preprocessor, so the LLaVA prefix is not false, but the message is about an env var and a build feature rather than a config field, and the combined string reads oddly.Impact is diagnostic only, but it is user facing. Both surfaces that load the preprocessor stringify the error:
src/backend/xla.rs:111wraps it insidecreate_sessionasOpenXLA image preprocessor load failed: {error}.src/server/batch/xla_preprocess.rs:74maps the loader error witherror.to_string()and passes it through the worker ready channel, which fails server startup with that text.So for a Qwen2-VL checkpoint an operator sees:
They are told a LLaVA config is invalid while loading a Qwen2-VL checkpoint, which sends them looking in the wrong place.
Proposed Solution
Either of:
invalid OpenXLA host-preprocessor config: {0}, orPrefer whichever keeps the LLaVA weight-loading and config messages as informative as they are today.
WeightLoad(failed to load LLaVA host-preprocessor weights: {0}) is only produced from the LLaVA loader paths insrc/loading/vlm_llava.rs, so it can stay as is unless the split makes a matching rename natural.Acceptance Criteria
MLXCEL_XLA_VISION_BACKENDpolicy parsing, model-family detection failure) or for Qwen2-VL messages.MLXCEL_XLA_VISION_BACKENDand its accepted values, and the--features xla-ireerebuild instruction.cargo test --release --lib --features metal,accelerate multimodal::host_preprocessorstays green.src/multimodal/host_preprocessor_tests.rsor thesrc/server/batch/xla_preprocess.rstests is updated. Current state: the host-preprocessor tests match on the variant withmatches!(host_preprocessor_tests.rs:300and:314) rather than on the display string, and the batch tests assert on other substrings, so no text assertion is expected to break. Re-check after the change.Technical Considerations
This was split out of #966 to keep that PR surgical. #966 deliberately left the wrapper text alone and only touched the Qwen2-VL loader contract; it does extend the message at
host_preprocessor.rs:178with the rebuild instruction (; rebuild mlxcel with --features xla-iree (this family has no MLX vision fallback)), so whoever picks this up should rebase on the merged #966 and preserve whatever final text landed there.No behavior change is intended: the same conditions must still fail, with the same variant discriminant unless a new variant is introduced, and only the rendered prefix changes.