Repository navigation
Conversation
main_export stored revision, cache_dir, subfolder, trust_remote_code and force_download in kwargs and dropped token and local_files_only, but the fill-mask, text2text-generation, automatic-speech-recognition and text-generation loaders called from_pretrained(model_name_or_path) without them. An export with revision="v2" therefore loaded and exported the weights from main, from the default cache, without the given token. Collect these options with a get_hub_kwargs helper and pass them to from_pretrained (and to AutoConfig in the causal LM loader). Default cache_dir to HF_HUB_CACHE instead of the legacy HUGGINGFACE_HUB_CACHE constant, so forwarding it does not override a user's HF_HUB_CACHE.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Exporting with a pinned revision, a custom cache, or an explicit token through the Python API exports the wrong model.
main_export(..., revision="v2")andExecuTorchModelForCausalLM.from_pretrained(model_id, export=True, revision="v2")both produce a.ptebuilt from themainweights. They also look in the default cache and send no token, so a private repo fails and a pinned export is not reproducible. The text-generation, fill-mask, text2text-generation and automatic-speech-recognition tasks are affected.What happens today
main_exportstoresrevision,cache_dir,subfolder,trust_remote_codeandforce_downloadinkwargs. It dropstokenandlocal_files_only. The task loaders then callfrom_pretrained(model_name_or_path)without any of them.image-classificationandobject-detectionalready forward**kwargs, so they were not affected.flowchart LR subgraph Before A1["main_export(revision='v2', cache_dir, token)"] --> B1["kwargs: revision, cache_dir, ...<br/>(token, local_files_only dropped)"] B1 --> C1["tasks/masked_lm.py etc.<br/>from_pretrained(model_name_or_path)"] C1 --> D1["weights from main, default cache, no token"] end subgraph After A2["main_export(revision='v2', cache_dir, token)"] --> B2["kwargs incl. token, local_files_only"] B2 --> C2["from_pretrained(model_name_or_path, **get_hub_kwargs(kwargs))"] C2 --> D2["weights from v2, given cache and token"] endReproducer
The script below needs no network access. It builds a Hub-style cache for
demo/tiny-bertwith two revisions (main: 1 layer,v2: 2 layers), then exports each revision withmain_export(..., task="fill-mask", recipe="portable", revision=rev). It prints the depth of the model the task loader returned.repro_revision.py
Before:
After:
Changes
exporters/executorch/utils.py: newget_hub_kwargs(kwargs). It returns the Hub loading options present inkwargs(cache_dir,force_download,local_files_only,revision,subfolder,token,trust_remote_code).exporters/executorch/__main__.py:main_exportnow also storestokenandlocal_files_onlyinkwargs.tasks/masked_lm.py,tasks/seq2seq_lm.py,tasks/asr.pyandtasks/causal_lm.pypass these options tofrom_pretrained. Incausal_lm.pythey also go toAutoConfig.from_pretrained, so the config and the weights come from the same revision.__main__.pyandoptimum/executorch/modeling.py: the defaultcache_diris nowhuggingface_hub.constants.HF_HUB_CACHEinstead of the legacyHUGGINGFACE_HUB_CACHE. The legacy constant ignores theHF_HUB_CACHEenvironment variable. Before this change the default was never forwarded, so it did not matter. Once it is forwarded, it would override a user'sHF_HUB_CACHE.HF_HUB_CACHEresolves to the same path transformers uses whencache_dirisNone.All forwarded defaults (
revision="main",subfolder="",token=None,trust_remote_code=False,force_download=False,local_files_only=False) match the transformers defaults, so calls without these arguments behave as before.Not in this PR:
tasks/multimodal_text_to_text.pyand the WhisperAutoProcessor.from_pretrainedcall inSeq2SeqLMExportableModulehave the same gap. I left them out because several open PRs edit those files. I can follow up once those land.Tests
MainExportHubKwargsTestintests/models/test_modeling_common.pycoversfill-mask(BERT),text-generation(Llama) andtext2text-generation(T5). Each case builds a tiny two-revision cache in a temp dir, callsmain_export(..., cache_dir=..., revision="v2", local_files_only=True)withexport_to_executorchmocked, and checks that the loaded model has 2 layers. The test needs no network and takes about 40 s on CPU.Results on Windows 11, Python 3.11, torch 2.12.0+cpu, executorch 1.3.1, transformers 5.0.0rc1:
pytest tests/models/test_modeling_common.py -k "MainExportHubKwargs or cli_helps"gives 4 passed.main: 3 failed. The loaders ignorecache_dirand raiseLocalEntryNotFoundError.black --check .andruff check .both pass.The XNNPACK recipe could not be run on Windows because the executorch wheel has no
flatc. The reproducer uses the portable recipe, and the Linux CI covers XNNPACK.