Skip to content

Use dtype instead of deprecated torch_dtype for transformers >= 4.56 #99

Description

@xyf5432

In train.py lines 331 and 344, ShowUIForConditionalGeneration.from_pretrained and Qwen2VLForConditionalGeneration.from_pretrained are called with torch_dtype=torch_dtype (e.g. torch_dtype=torch_dtype, low_cpu_mem_usage=True, _attn_implementation=..., quantization_config=bnb_config, device_map=...).

The torch_dtype keyword argument was deprecated in transformers 4.56 (PR #39782) and replaced by dtype. On transformers 4.56+ these calls emit a DeprecationWarning, and the argument will be removed in a future release, breaking training.

Suggested fix: choose the keyword based on the installed transformers version with packaging.version:

import transformers
from packaging.version import Version

def _dtype_kwargs(dtype):
    """`dtype` keyword of `from_pretrained` exists since transformers 4.56 (PR #39782);
    older versions use `torch_dtype`."""
    if Version(transformers.__version__) >= Version("4.56"):
        return {"dtype": dtype}
    return {"torch_dtype": dtype}

model = ShowUIForConditionalGeneration.from_pretrained(
    model_url,
    **_dtype_kwargs(torch_dtype),
    low_cpu_mem_usage=True,
    _attn_implementation=args.attn_imple,
    quantization_config=bnb_config,
    device_map=f"cuda:{args.local_rank}",
    lm_skip_layer=lm_skip_layer,
    lm_skip_ratio=args.lm_skip_ratio,
)

(Same pattern for the Qwen2VLForConditionalGeneration call on line 344.) This keeps compatibility with transformers < 4.56 and stops the deprecation warning on 4.56+.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions