diff --git a/README.zh-CN.md b/README.zh-CN.md index 420449983..2b40c8380 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -118,7 +118,7 @@ libreyolo predict --model yolo9-t --source screen # 屏幕捕获 | **视频嵌入** | V-JEPA 2（片段级嵌入，另有可训练注意力探针的视频分类） | | **人体网格** | SAM 3D Body | | **图像复原** | DDColor、HVI-CIDNet、LaMa、NAFNet、QuickSRNet、Real-ESRGAN、SwinIR | -| **背景移除** | BiRefNet、FeyNobg、ViTMatte | +| **背景移除** | BiRefNet、FeyNobg、BEN2、ViTMatte | | **OCR** | PP-OCR | | **点检测** | FOMO、LocateAnything | | **视线估计** | L2CS | diff --git a/THIRD_PARTY_NOTICES.txt b/THIRD_PARTY_NOTICES.txt index 86a9e4c01..e5fe8a1d0 100644 --- a/THIRD_PARTY_NOTICES.txt +++ b/THIRD_PARTY_NOTICES.txt @@ -351,6 +351,7 @@ learned parameters unchanged). See libreyolo/models/pe/NOTICE.md. YOLOX (Megvii-BaseDetection) -------------------------------------------------------------------- Source: https://github.com/Megvii-BaseDetection/YOLOX +Commit: 6d30efe8f9f277e8788fb2d9b17d08238a5ea292 License: Apache License 2.0 Copyright (c) 2021-2022 Megvii Inc. All rights reserved. Used for: YOLOX model family (libreyolo/models/yolox/), EMA helper @@ -359,7 +360,12 @@ Used for: YOLOX model family (libreyolo/models/yolox/), EMA helper adapted for the YOLOv7 family (libreyolo/models/yolo7/loss.py: imports bboxes_iou/IoULoss from the yolox modules and adapts the get_assignments/get_geometry_constraint/simota_matching/get_losses - assignment logic to the v7 anchor head). + assignment logic to the v7 anchor head). The shared COCO bounding-box + clipping helper (libreyolo/utils/coco_geometry.py) is used by the COCO + dataset loader (libreyolo/data/dataset.py), native COCO VLM training + dataset (libreyolo/models/vlm/training/data.py), and VLM confidence + gate writer/report reconstruction (libreyolo/validation/ + vlm_confidence_validator.py and vlm_confidence_report.py). -------------------------------------------------------------------- YOLO (MultimediaTechLab/YOLO) @@ -1921,6 +1927,31 @@ Used for: the Transformers-backed LibreEdgeTAM adapter in The full Apache License 2.0 text is included at licenses/Apache-2.0.txt. +-------------------------------------------------------------------- +Qwen3-VL fine-tuning recipe (QwenLM) +-------------------------------------------------------------------- +Source: https://github.com/QwenLM/Qwen3-VL +Commit: 96588727e44c78b25ba03ea03b8e12f7e64fd0da +Source paths: + qwen-vl-finetune/qwenvl/train/argument.py + qwen-vl-finetune/qwenvl/train/train_qwen.py + qwen-vl-finetune/README.md +License: Apache License 2.0 (see licenses/Apache-2.0.txt) +Copyright 2023 Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, + Yann Dubois, Xuechen Li (retained by train_qwen.py, which + identifies FastChat and Stanford Alpaca as earlier sources) +Used for: the Qwen3-VL recipe reference behind + libreyolo/models/vlm/training/recipes.py, + libreyolo/models/vlm/training/trainer.py, and the trainability + declaration in libreyolo/models/vlm/qwen3vl.py: PEFT LoRA on + language-model projections while the vision tower remains frozen, + plus the 0.03 warmup-ratio and cosine-schedule reference. LibreYOLO + adds its own MLP projection scope and detection-specific pipeline, + and uses different rank, alpha, dropout, learning-rate, and + weight-decay values. No upstream source file is bundled. + +The full Apache License 2.0 text is included at licenses/Apache-2.0.txt. + -------------------------------------------------------------------- SenseNova-Vision / Bagel (SenseTime; ByteDance; Hugging Face; BFL) -------------------------------------------------------------------- diff --git a/docs/adr/0002-librevlm-contract.md b/docs/adr/0002-librevlm-contract.md index 541a32f0c..968995caf 100644 --- a/docs/adr/0002-librevlm-contract.md +++ b/docs/adr/0002-librevlm-contract.md @@ -1,7 +1,7 @@ # ADR 0002: LibreVLM Contract For Vision-Language Detectors - Status: Accepted -- Date: 2026-06-05 (updated 2026-06-06) +- Date: 2026-06-05 (updated 2026-08-16) - Scope: New model tier (vision-language models used as open-vocab detectors) ## Context @@ -75,6 +75,49 @@ text = model.chat("image.jpg", "How many cars are pink?") # raw escape hatch from a fixed task / grounding token plus the class list, so `prompt=` is ignored for those two. +The supported VLM CLI surface is intentionally smaller than the detector CLI: + +```bash +libreyolo predict --model qwen3-vl-4b --source image.jpg \ + --names '["pink car", "wheel"]' +libreyolo train --model qwen3-vl-2b --data dataset.yaml +``` + +- `predict` resolves VLM aliases, schema-valid checkpoint directories, and + immutable `hf+vlm://owner/repo@` artifacts. `--names` is a VLM-only + JSON vocabulary; `--classes` remains a numeric output filter. Explicit + `imgsz=` is rejected because the family processor owns image resizing. + Directory chunks still generate one image at a time. +- `train` is verified only for the Qwen3-VL 2B and 4B base aliases. It uses + VLM-native defaults rather than detector defaults, and selects `best` by + validation loss when a validation split exists, otherwise training loss. + Continue an adapter with the base alias plus `resume=`; + training an inference-loaded checkpoint wrapper is rejected. +- Detector-only training options, unsupported families/sizes, and standalone + `val`, `export`, or `quantize` requests fail before loading VLM weights. + +### Publication artifacts + +Local training checkpoints and published artifacts are different contracts. +The v1 publication builder accepts only Qwen3-VL 2B/4B detection LoRA output +written with `peft==0.19.1` and `transformers==5.12.1`. It requires external, +human-approved evidence that binds the adapter, checkpoint contract, processor, +complete immutable base snapshot, training-data manifest, evaluation report, +an exact-zero two-run repeatability receipt, clean code revision, and fixed +recipe. Publication evidence v2 binds the primary and repeat run identities, +the raw receipt, and its canonical comparison. The library can generate an +unapproved template with derived hashes, but it cannot manufacture an +approval. + +The artifact includes the adapter and exact Qwen processor, tokenizer, and +chat-template assets under Apache-2.0. Base weights remain reference-only. +`push_vlm_artifact()` refuses existing repositories, starts private, creates +one commit, verifies that immutable tree through a fresh download, and returns +`hf+vlm://owner/repo@<40-character-commit>`. The detector `hf://` transport and +generic Hub logger are not used. Hashes are integrity bindings, not signatures +or proof that human claims are true. See +[`../vlm_hub_artifact.md`](../vlm_hub_artifact.md). + ## Internal Contract `LibreVLMModel(BaseModel)` is the shared base. It does NOT define `can_load`, so @@ -103,8 +146,10 @@ shared `InferenceRunner` drives: - `_preprocess(image, ...)` builds the chat-template inputs from the image plus the detection prompt; returns `(inputs, pil_image, (W, H), ratio=1.0)`. Boxes come back normalized to the image, so there is no letterbox/unpad math. -- `_forward(inputs)` runs `model.generate(...)` greedily and returns only the - newly generated tokens. +- `_forward(inputs)` runs `model.generate(...)` greedily and returns the newly + generated tokens. A family-gated scoring path can also attach one + selected-token log-probability per step without retaining vocabulary-sized + score tensors. - `_postprocess(output, conf, ...)` decodes, tolerantly parses the JSON, scales the coordinates per `BBOX_KEY`/`COORD_DIVISOR`, and returns the standard detection dict `{boxes, scores, classes, num_detections}` that @@ -126,25 +171,56 @@ in [`../librevlm_design.md`](../librevlm_design.md). ## Confidence -Generated detections carry no calibrated per-box score. The tier assigns a -constant placeholder (`DEFAULT_SCORE = 1.0`), so `predict`/draw/`track` behave -normally and `conf=` filtering still functions mechanically. Consequences: - -- `conf=` thresholds and ranking are soft, not calibrated. -- `track()` runs, but because every box is scored 1.0, ByteTrack's two-stage, - score-stratified association is inert (no separate low-confidence recovery - stage and `new_track_thresh` never bites) until a real score lands. -- `val()` (mAP) is intentionally unsupported; it would be misleading. - -`_score_detections(items)` is the documented override point for a real signal -(decoder token log-probs or self-consistency) in a later iteration. +Generated detections carry no calibrated per-box score. The generic VLM families +currently assign a constant placeholder (`DEFAULT_SCORE = 1.0`). A bounded-memory +candidate for Qwen3-VL can derive a ranking signal from the geometric mean of +generated label-token and coordinate-token probabilities. It records one +selected-token log-probability after the configured generation processors per +step, rather than retaining a vocabulary-sized score tensor for every token. +The candidate remains disabled until its real-data gate passes, so ordinary +`predict()` keeps the established constant-score behavior. Candidate scores are +available only to the internal validator, not public prediction, the CLI, or +integrations. LibreMODUS separately uses the minimum constrained-token +probability for each detection. + +`model.confidence_method` reports the configured source (`constant` today for +Qwen3-VL, and `constrained_token_min` for LibreMODUS). + +Consequences: + +- On constant-score families, `conf=` filtering is mechanical and ByteTrack's + score-stratified association remains inert (no separate low-confidence + recovery stage). +- `val()` (mAP) remains unsupported until the candidate score orders correct + detections better than the constant baseline, behaves safely with the public + confidence threshold, and is reproducible. Unit tests establish plumbing, not + score quality. +- Candidate-only Brier error, fixed-bin ECE, and reliability bins are diagnostic + outputs of the internal gate, not a calibration claim. Missing scores never + inherit the constant fallback in those statistics; fallback safety and score + coverage are reported separately. +- Persisted gate reports use schema v2 and can be compared across processes only + after strict reconstruction validates their hashes and duplicated metrics. + Timing and plot artifacts do not define reproducibility. The hashes provide + consistency checks, not report authentication. +- A canonical repeatability receipt uses two distinct run and process + identifiers and binds both report/envelope byte identities, zero tolerances, + and the complete comparison. Publication treats it as structural integrity + evidence, not publisher or reviewer authentication or proof that reports are + truthful. + +`_score_detections(items)` remains the scalar fallback for custom generation +paths. Scored greedy generations use the additive per-item scoring path. ## Licensing -LibreYOLO ships only its own VLM adapter code: families either load through the -Apache-2.0 `transformers` API or, when a model genuinely requires Hugging Face -remote code, download that upstream model-repository code at runtime under the -upstream model repo's terms. LibreYOLO does not redistribute VLM weights. +Family aliases either load through the Apache-2.0 `transformers` API or, when a +model genuinely requires Hugging Face remote code, download that upstream +repository at runtime under its terms. LibreYOLO does not mirror those base +weights or remote-code repositories. The strict Qwen LoRA artifact is the +narrow exception for non-weight assets: it redistributes the exact processor, +tokenizer, and chat-template files under Apache-2.0 while retaining the base +weights as an immutable external reference. The default model (Qwen3-VL-4B) is Apache-2.0, so it needs no notice. When a model's weights or required model-repository code are under a non-permissive @@ -160,15 +236,15 @@ executing mutable upstream model-repository code under the same alias. ## Out Of Scope (v1) -- Training / fine-tuning (`train()` raises; fine-tune upstream). Superseded - for Qwen3-VL: LoRA detection fine-tuning shipped later behind the same - `train()` surface, with untrainable families keeping documented refusal - messages. See `docs/vlm_training.md`. -- Dataset validation / mAP (`val()` raises; see "Confidence"). -- Export to ONNX/TensorRT/etc. (`export()` raises; generative decode). -- CLI: the `libreyolo` command does not resolve VLM aliases in v1. The tier is a - Python-API surface (`LibreVLM(...)`); `predict`/`track` parity is at the API - level, not the CLI. +- Training outside the verified Qwen3-VL 2B/4B detection cohort. Supported + fine-tuning is documented in [`../vlm_training.md`](../vlm_training.md). +- Dataset validation / mAP (`val()` and the CLI `val` command reject VLMs; see + "Confidence"). +- Export to ONNX/TensorRT/etc. and post-training quantization. The CLI `export` + and `quantize` commands reject VLM references before loading weights. +- Broad CLI parity. The documented VLM CLI contract is prediction plus the + verified Qwen training cohort; tier-specific methods such as `chat()` remain + Python API surfaces. ## Consequences @@ -180,7 +256,8 @@ executing mutable upstream model-repository code under the same alias. ### Negative -- Confidence is synthetic until the log-prob path lands. +- Generic-family confidence remains constant until each score path passes its + real-data quality gate; LibreMODUS's constrained-token score is uncalibrated. - Generation is slower and less deterministic than a detector forward. - Adds `transformers` (already an optional extra) to the `vlm` extra. @@ -191,4 +268,9 @@ executing mutable upstream model-repository code under the same alias. SmolVLM2, Gemma 4). Florence-2 and Kosmos-2 use task / grounding tokens. Moondream uses native detect/point skills. See the Available-models table in [`../librevlm_design.md`](../librevlm_design.md). +- CLI alias/checkpoint/immutable-artifact routing for VLM prediction and + verified Qwen3-VL 2B/4B training, with pre-load guards for remote training + and unsupported validation, export, and quantization. +- Strict, reviewed Qwen3-VL LoRA artifact construction and immutable Hub + transport, separate from detector checkpoint publication. - Offline parser unit tests plus a `vlm`-marked end-to-end smoke test. diff --git a/docs/checkpoint_schema.md b/docs/checkpoint_schema.md index 5ecee4f64..ec915f71d 100644 --- a/docs/checkpoint_schema.md +++ b/docs/checkpoint_schema.md @@ -328,6 +328,36 @@ distributed as training checkpoints. For release compatibility, readers accept legacy best-metric aliases such as `best_mAP50_95`, `best_mAP50`, `best_metric`, and `best_metric_name`. +## VLM directory checkpoints and Hub artifacts + +LibreVLM training outputs directories rather than native detector `.pt` files. +Each local directory carries `libreyolo_vlm.json` schema 1, which records the +family, size, task, ordered vocabulary, prompt and box conventions, metrics, +LibreYOLO version, and exact base repository revision. A LoRA checkpoint also +contains `adapter_model.safetensors`, `adapter_config.json`, and its processor +files. A full fine-tune instead contains a self-contained model directory. +Both are local training/reload formats, not publication approval. + +`libreyolo.vlm-artifact.v1` is a separate, flat publication format restricted +to the verified Qwen3-VL 2B/4B detection LoRA cohort. Its +`libreyolo_vlm_artifact.json` manifest binds the exact payload inventory and +the adapter, contract, processor, evidence, and immutable base-snapshot +identities. Base weights are referenced and verified but are not included; +the exact Qwen processor, tokenizer, and chat-template assets are redistributed +under Apache-2.0 with generated license and notice files. + +The artifact carries reviewed `libreyolo.vlm-publication-evidence.v2` evidence. +Its evaluation record binds one primary confidence report and envelope plus a +canonical `libreyolo.vlm-confidence-repeatability-receipt.v1` comparison of +that primary run with a second fresh-process run. Both the raw receipt and its +canonical comparison object are SHA-256 bound. These are structural integrity +records, not publisher or reviewer authentication. + +Published artifacts are addressed as +`hf+vlm://owner/repo@<40-character-commit>`. This is distinct from the +single-file detector `hf://` transport. The evidence and Hub workflow are +defined in [`vlm_hub_artifact.md`](vlm_hub_artifact.md). + ## External Snapshot Exception The schema above governs LibreYOLO-authored `.pt` checkpoints. It does not diff --git a/docs/fiftyone.md b/docs/fiftyone.md index a9019d0b5..1c8b5e4f4 100644 --- a/docs/fiftyone.md +++ b/docs/fiftyone.md @@ -2,8 +2,8 @@ [FiftyOne](https://github.com/voxel51/fiftyone) (Apache-2.0) is a dataset curation and prediction-analysis tool. `libreyolo.integrations.fiftyone` sends -LibreYOLO predictions into a FiftyOne dataset and moves datasets in both -directions between a LibreYOLO dataset yaml and a FiftyOne dataset. +LibreYOLO and LibreVLM predictions into a FiftyOne dataset and moves datasets +in both directions between a LibreYOLO dataset yaml and a FiftyOne dataset. Nothing is vendored and nothing is imported at `import libreyolo` time. The module imports `fiftyone` lazily and raises an install hint when it is absent. @@ -41,11 +41,35 @@ apply_model(dataset, model, label_field="predictions", conf=0.25, batch_size=8) session = fo.launch_app(dataset) ``` -`apply_model` accepts a loaded model or a checkpoint name, forwards `conf`, -`iou`, `imgsz`, `device`, `classes`, and `max_det` to `model.predict`, and runs -through FiftyOne's own `apply_model`, so the progress bar, `skip_failures`, and -batching behave the way FiftyOne users expect. `batch_size` maps onto -LibreYOLO's batched list inference: one stacked forward per chunk. +For open-vocabulary VLM prediction, configure the vocabulary on the loaded +model first: + +```python +from libreyolo import LibreVLM + +model = LibreVLM("qwen3-vl-4b") +model.set_classes(["forklift", "worker"]) +apply_model(dataset, model, label_field="vlm_predictions", batch_size=8) +``` + +`apply_model` accepts a loaded model, a detector checkpoint/name, a VLM alias, +or a VLM checkpoint directory. It forwards `conf`, `iou`, `device`, `classes`, +and `max_det` to `model.predict`. Detector callers may also override `imgsz`; +LibreVLM callers must omit it because each VLM's processor owns image resizing. +The integration runs through FiftyOne's own `apply_model`, so the progress bar +and `skip_failures` behave the way FiftyOne users expect. `batch_size` maps onto +LibreYOLO's batched list inference when the model supports it. Generative +`LibreVLM` adapters currently opt out of batched prediction, so FiftyOne may +still form chunks but the VLM generates one image at a time; increasing +`batch_size` does not improve VLM throughput. + +`LibreVLM` detections use the same standard `Results` conversion shown below. +Generic chat VLMs currently expose an uncalibrated constant confidence of +`1.0`, so confidence ranking, mistakenness, and threshold-based curation are not +meaningful for those families until their documented real-data score gate +passes. The candidate Qwen3-VL token score is internal and is not exposed to +FiftyOne. Box geometry, labels, numeric class filtering, and visualization +remain valid. To use the model with any other FiftyOne API that takes a model, wrap it directly: diff --git a/docs/hf_hub.md b/docs/hf_hub.md index c94d279a4..7abe35e37 100644 --- a/docs/hf_hub.md +++ b/docs/hf_hub.md @@ -1,19 +1,32 @@ # Hugging Face Hub integration -LibreYOLO can load checkpoints from any Hugging Face Hub repository and push -your own fine-tuned checkpoints back, in both cases using the standard -LibreYOLO metadata schema (v1.0, see `docs/checkpoint_schema.md`). +LibreYOLO has two separate Hub transports: -The integration is optional: +- `hf://` and bare `owner/repo` references select one detector checkpoint file. +- `hf+vlm://owner/repo@` selects one immutable, multi-file VLM + publication artifact. + +Detector checkpoints use the standard LibreYOLO metadata schema described in +[`checkpoint_schema.md`](checkpoint_schema.md). VLM artifacts use the stricter +directory contract in [`vlm_hub_artifact.md`](vlm_hub_artifact.md). The two URI +forms are not interchangeable. + +Detector Hub transport is optional: ```bash pip install libreyolo[hf] ``` +Loading an immutable VLM artifact also needs the VLM runtime: + +```bash +pip install "libreyolo[vlm,hf]" +``` + `import libreyolo` never imports `huggingface_hub`; it is only loaded when a Hub reference or push is actually used. -## Loading models from the Hub +## Loading detector checkpoints from the Hub Any repository that contains a LibreYOLO checkpoint can be loaded directly: @@ -51,7 +64,7 @@ pass a local path, which re-enables legacy architecture detection. Private and gated repos work once you are authenticated (see below). -## Pushing models to the Hub +## Pushing detector checkpoints to the Hub Any loaded model can be published, together with an auto-generated model card derived from its checkpoint metadata (family, size, task, classes, metrics): @@ -97,6 +110,61 @@ uploads unattended and a model trained on proprietary data must not become public by surprise. Pass `private=False` to publish from training. Repos that already exist keep their current visibility either way. +This detector push path, including `HuggingFaceHubLogger`, does not accept +LibreVLM directory checkpoints. VLM publication is an explicit, reviewed +operation described below. + +## Loading and publishing VLM artifacts + +A published VLM adapter is addressed only by a canonical URI with a lowercase, +40-character commit SHA: + +```python +from libreyolo import LibreVLM +from libreyolo.models.vlm import inspect_vlm_hub_artifact + +uri = "hf+vlm://someuser/strawberry-vlm@0123456789abcdef0123456789abcdef01234567" +manifest = inspect_vlm_hub_artifact(uri) # manifest only; no tensor payload +model = LibreVLM(uri) +``` + +Branches, tags, abbreviated hashes, bare repository IDs, query strings, and +file suffixes are rejected. `LibreVLM(uri)` validates the artifact, acquires +and validates the exact Qwen base snapshot recorded by it, and revalidates both +before use. The base weights are not stored in the adapter repository. + +To materialize an artifact without loading a model: + +```python +from libreyolo.models.vlm import download_vlm_artifact, validate_vlm_artifact + +info = download_vlm_artifact(uri, "artifacts/strawberry-vlm") +validate_vlm_artifact(info.root) +``` + +The destination must not already exist. Online inspection and download also +require the repository tree at that commit to equal the manifest inventory. +`local_files_only=True` validates the cached allowlisted files, but cannot +prove that the remote commit has no additional files. + +Upload is separate from training and from the detector Hub logger: + +```python +from libreyolo.models.vlm import push_vlm_artifact + +uri = push_vlm_artifact( + "artifacts/strawberry-vlm", + "someuser/strawberry-vlm", +) # private=True by default; returns an immutable hf+vlm:// URI +``` + +`push_vlm_artifact` accepts only a fully validated artifact and refuses any +pre-existing repository, including an empty one. It creates a private repo, +uploads the exact artifact in one commit, and verifies that commit through a +fresh download. With `private=False`, visibility changes only after that +verification succeeds. See [`vlm_hub_artifact.md`](vlm_hub_artifact.md) for the +required human evidence gates and build workflow. + ## Authentication Reading public repos needs no login. Private repos, gated repos, and all @@ -104,7 +172,8 @@ pushes need a Hugging Face token, resolved the standard way: 1. Run `hf auth login` once (stores a token on the machine), or 2. set the `HF_TOKEN` environment variable, or -3. pass `token="hf_..."` to `push_to_hub` / `HuggingFaceHubLogger`. +3. pass `token="hf_..."` to `push_to_hub`, `HuggingFaceHubLogger`, or the VLM + Hub functions. Pushing requires a token with write scope; create one at . Error messages repeat these steps diff --git a/docs/librevlm_design.md b/docs/librevlm_design.md index e064b7ef3..29350d82c 100644 --- a/docs/librevlm_design.md +++ b/docs/librevlm_design.md @@ -120,19 +120,89 @@ Rationale: Per-image, free-form queries are served by `chat()`, which is the right place for genuinely per-call prompts. +### CLI surface + +The CLI recognizes every VLM alias, a checkpoint directory containing a valid +`libreyolo_vlm.json` contract, and an immutable +`hf+vlm://owner/repo@` publication URI. Prediction uses the same sticky +open-vocabulary surface through `--names`: + +```bash +libreyolo predict --model qwen3-vl-4b --source image.jpg \ + --names '["pink car", "wheel"]' +``` + +`--names` must be a non-empty JSON list of unique, non-empty strings and is +accepted only for VLM references. `--classes` remains the numeric output +filter. Explicit `--imgsz` is rejected because each family processor owns its +resize contract. Directory chunks are accepted, but generative VLM inference +remains serial within each chunk, so increasing `--batch` does not improve +throughput. + +The train command routes verified Qwen3-VL 2B and 4B base aliases to the VLM +trainer: + +```bash +libreyolo train --model qwen3-vl-2b --data strawberries.yaml +libreyolo train --model qwen3-vl-2b --data strawberries.yaml \ + --resume runs/vlm/train/weights/last +``` + +It uses the VLM-native defaults, including ten epochs, batch one, LoRA, +recipe-owned learning rate, gradient accumulation, and `runs/vlm/train` +output. Best-checkpoint selection is based on validation loss when a validation +split exists, otherwise training loss. Detector-only training knobs and +unverified family/size combinations are rejected before model loading. +Training an inference-loaded checkpoint wrapper is also rejected because its +adapter is already merged; continuation uses a pristine base alias plus +`resume=` so the saved adapter remains relative to the +recorded base. +Standalone `val`, `export`, and `quantize` also reject VLM aliases and +checkpoints before loading weights. + +### VLM Hub artifacts + +VLM publication is separate from the detector `hf://` single-file transport. +The verified Qwen3-VL 2B/4B LoRA cohort can be packaged only after a human +approves manifest-bound data, license, privacy, evaluation, and code-provenance +evidence. The builder includes the adapter and exact Qwen processor assets but +references the immutable base-weight snapshot instead of copying it. + +`push_vlm_artifact()` accepts only the resulting strict artifact, refuses an +existing repository, uploads one commit private by default, verifies it through +a fresh download, and returns its immutable `hf+vlm://` URI. The generic Hub +logger and `LibreVLMModel.push_to_hub()` remain disabled. Remote artifacts are +prediction inputs; training still begins from a verified base alias with an +optional local `resume=` checkpoint. See +[`vlm_hub_artifact.md`](vlm_hub_artifact.md). + ## Decision 3: the output is `Results`, and confidence is honestly soft The tier returns the standard `Results` (`boxes.xyxy`, `boxes.cls`, `boxes.conf`, `.plot()`, `.save()`), so folders, video, tracking, and drawing all work unchanged. No new output type is invented. -But these models emit no calibrated per-box score, so `conf` is a placeholder. -We do not pretend otherwise: - -- `conf=` filtering and ranking are soft, not calibrated. -- `val()` (mAP) is intentionally unsupported, because it would be misleading. -- `_score_detections()` is the documented hook for a real signal later (decoder - token log-probabilities or self-consistency). +These models emit no calibrated detector score. The generic VLM families retain +`1.0`. A bounded-memory Qwen3-VL candidate can derive a ranking signal from +generated label-token and coordinate-token probabilities, but it remains off +until the real-data quality gate passes and is available only to the internal +validator. Qwen3-VL public prediction, the CLI, and integrations continue to +receive the constant score. Inspect `model.confidence_method` for the +configured source. We do not pretend any token-derived score is calibrated: + +- On constant-score families, `conf=` filtering is mechanical. +- `val()` (mAP) remains unsupported until a real Qwen benchmark demonstrates + useful ordering, safe threshold behavior, and reproducibility. +- The internal quality gate reports candidate-only Brier error, fixed-bin ECE, + and a reliability diagram. Unscored boxes are excluded from calibration and + remain visible through separate response, parsed-object, and prediction + coverage plus fallback-retention diagnostics. Plot-enabled gate runs write + `vlm_confidence_reliability.svg`; the JSON bin counts remain authoritative. +- Gate reports use the strict `libreyolo.vlm-confidence-report.v2` schema. The + internal persisted-report comparator reconstructs both runs and ignores only + validated timing and plot-artifact differences. Report hashes detect stale or + inconsistent fields; they are not signatures and do not authenticate a file. +- `_score_detections()` remains the scalar fallback for custom generation paths. This is the honest boundary of the tier: it gives you boxes and labels, not a calibrated detector contract. For calibrated scores and tight boxes, the @@ -198,12 +268,11 @@ output but grounds single-class queries extremely well, so its family runs one "Locate every