Skip to content

stable_diffusion_v1_4使用torchtrt推理时报错 #754

Description

@MaltoseFlower

ERROR: [Torch-TensorRT] - Unsupported operator: aten::to.dtype_layout(Tensor(a) self, *, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=None, bool non_blocking=False, bool copy=False, MemoryFormat? memory_format=None) -> Tensor(a)
/workspace/luor/FlagPerf/inference/benchmarks/stable_diffusion_v1_4/pytorch/model_utils/unet2d.py(854): forward
/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch/nn/modules/module.py(1501): _slow_forward
/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch/nn/modules/module.py(1520): _call_impl
/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch/nn/modules/module.py(1511): _wrapped_call_impl
/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch/jit/_trace.py(1074): trace_module
/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch/jit/_trace.py(806): trace
/workspace/luor/FlagPerf/inference/inference_engine/nvidia/torchtrt.py(23): call
/workspace/luor/FlagPerf/inference/benchmarks/stable_diffusion_v1_4/pytorch/forward.py(209): engine_forward
/workspace/luor/FlagPerf/inference/run_inference.py(80): main
/workspace/luor/FlagPerf/inference/run_inference.py(137):

Traceback (most recent call last):
File "/workspace/luor/FlagPerf/inference/run_inference.py", line 137, in
config, p_forward, p_infer, p_forward_core, p_infer_core, val_acc, infer_acc = main(
File "/workspace/luor/FlagPerf/inference/run_inference.py", line 80, in main
p_infer, p_infer_core, infer_acc = benchmark_module.engine_forward(
File "/workspace/luor/FlagPerf/inference/benchmarks/stable_diffusion_v1_4/pytorch/forward.py", line 209, in engine_forward
outputs = model(inputs)
File "/workspace/luor/FlagPerf/inference/inference_engine/nvidia/torchtrt.py", line 25, in call
self.trt_model = torchtrt.compile(
File "/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch_tensorrt/_compile.py", line 185, in compile
compiled_ts_module: torch.jit.ScriptModule = torchscript_compile(
File "/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch_tensorrt/ts/_compiler.py", line 151, in compile
compiled_cpp_mod = _C.compile_graph(module._c, _parse_compile_spec(spec))
RuntimeError: [Error thrown at core/compiler.cpp:413] Expected conversion::VerifyConverterSupportForBlock(g->block()) to be true but got false
Not all operations in graph are supported by the compiler

请问测试时,官方使用的软件版本是多少?
以下是我测试时的版本:
libtorch 2.4.0
torch 2.2.2
torch-tensorrt 2.2.0
torchmetrics 1.4.2
tensorrt 8.6.1.post1
tensorrt-bindings 8.6.1
tensorrt-libs 8.6.1

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions