Error when running Torch Backend

[shkr]

Following the instructions in README.md

downladed the .bin file set the environment variable

env:TORCH_CUDA_VERSION='cu116'

➜  stable-diffusion-burn git:(main) ✗ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_Mar__8_18:18:20_PST_2022
Cuda compilation tools, release 11.6, V11.6.124
Build cuda_11.6.r11.6/compiler.31057947_0
➜  stable-diffusion-burn git:(main) ✗ 

Ran the sample command
cargo run --release --bin sample burn SDv1-4 7.5 20 "An ancient mossy stone." img
Encountered the error message below.

Loading tokenizer...
Loading model...
thread 'main' panicked at /home/shkr/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tch-0.14.0/src/wrappers/tensor_generated.rs:17378:27:
called `Result::unwrap()` on an `Err` value: Torch("Could not run 'aten::empty_strided' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::empty_strided' is only available for these backends: [CPU, Meta, QuantizedCPU, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradHIP, AutogradXLA, AutogradMPS, AutogradIPU, AutogradXPU, AutogradHPU, AutogradVE, AutogradLazy, AutogradMTIA, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, AutogradMeta, AutogradNestedTensor, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PreDispatch, PythonDispatcher].\n\nCPU: registered at aten/src/ATen/RegisterCPU.cpp:31188 [kernel]\nMeta: registered at aten/src/ATen/RegisterMeta.cpp:26829 [kernel]\nQuantizedCPU: registered at aten/src/ATen/RegisterQuantizedCPU.cpp:944 [kernel]\nBackendSelect: registered at aten/src/ATen/RegisterBackendSelect.cpp:742 [kernel]\nPython: registered at ../aten/src/ATen/core/PythonFallbackKernel.cpp:153 [backend fallback]\nFuncTorchDynamicLayerBackMode: registered at ../aten/src/ATen/functorch/DynamicLayer.cpp:498 [backend fallback]\nFunctionalize: registered at ../aten/src/ATen/FunctionalizeFallbackKernel.cpp:290 [backend fallback]\nNamed: registered at ../aten/src/ATen/core/NamedRegistrations.cpp:7 [backend fallback]\nConjugate: fallthrough registered at ../aten/src/ATen/ConjugateFallback.cpp:21 [kernel]\nNegative: fallthrough registered at ../aten/src/ATen/native/NegateFallback.cpp:23 [kernel]\nZeroTensor: fallthrough registered at ../aten/src/ATen/ZeroTensorFallback.cpp:90 [kernel]\nADInplaceOrView: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:86 [backend fallback]\nAutogradOther: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradCPU: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradCUDA: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradHIP: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradXLA: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradMPS: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradIPU: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradXPU: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradHPU: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradVE: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradLazy: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradMTIA: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradPrivateUse1: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradPrivateUse2: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradPrivateUse3: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradMeta: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nAutogradNestedTensor: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:18610 [autograd kernel]\nTracer: registered at ../torch/csrc/autograd/generated/TraceType_2.cpp:17079 [kernel]\nAutocastCPU: fallthrough registered at ../aten/src/ATen/autocast_mode.cpp:382 [backend fallback]\nAutocastCUDA: fallthrough registered at ../aten/src/ATen/autocast_mode.cpp:249 [backend fallback]\nFuncTorchBatched: registered at ../aten/src/ATen/functorch/LegacyBatchingRegistrations.cpp:710 [backend fallback]\nFuncTorchVmapMode: fallthrough registered at ../aten/src/ATen/functorch/VmapModeRegistrations.cpp:28 [backend fallback]\nBatched: registered at ../aten/src/ATen/LegacyBatchingRegistrations.cpp:1075 [backend fallback]\nVmapMode: fallthrough registered at ../aten/src/ATen/VmapModeRegistrations.cpp:33 [backend fallback]\nFuncTorchGradWrapper: registered at ../aten/src/ATen/functorch/TensorWrapper.cpp:203 [backend fallback]\nPythonTLSSnapshot: registered at ../aten/src/ATen/core/PythonFallbackKernel.cpp:161 [backend fallback]\nFuncTorchDynamicLayerFrontMode: registered at ../aten/src/ATen/functorch/DynamicLayer.cpp:494 [backend fallback]\nPreDispatch: registered at ../aten/src/ATen/core/PythonFallbackKernel.cpp:165 [backend fallback]\nPythonDispatcher: registered at ../aten/src/ATen/core/PythonFallbackKernel.cpp:157 [backend fallback]\n\nException raised from reportError at ../aten/src/ATen/core/dispatch/OperatorEntry.cpp:554 (most recent call first):\nframe #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x6b (0x7f50ceaeb1fb in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libc10.so)\nframe #1: c10::impl::OperatorEntry::reportError(c10::DispatchKey) const + 0x449 (0x7f50cff0b4d9 in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #2: <unknown function> + 0x19ca46b (0x7f50d050946b in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #3: at::_ops::empty_strided::redispatch(c10::DispatchKeySet, c10::ArrayRef<c10::SymInt>, c10::ArrayRef<c10::SymInt>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>) + 0xac (0x7f50d0bfe6dc in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #4: <unknown function> + 0x242f335 (0x7f50d0f6e335 in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #5: at::_ops::empty_strided::call(c10::ArrayRef<c10::SymInt>, c10::ArrayRef<c10::SymInt>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>) + 0x168 (0x7f50d0c47ea8 in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #6: <unknown function> + 0x14eaa0f (0x7f50d0029a0f in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #7: at::native::_to_copy(at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) + 0x1380 (0x7f50d03b3ac0 in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #8: <unknown function> + 0x261ddf3 (0x7f50d115cdf3 in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #9: at::_ops::_to_copy::redispatch(c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) + 0x103 (0x7f50d08caca3 in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #10: <unknown function> + 0x2432898 (0x7f50d0f71898 in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #11: at::_ops::_to_copy::redispatch(c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) + 0x103 (0x7f50d08caca3 in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #12: <unknown function> + 0x39ef341 (0x7f50d252e341 in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #13: <unknown function> + 0x39ef8eb (0x7f50d252e8eb in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #14: at::_ops::_to_copy::call(at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) + 0x201 (0x7f50d094e1e1 in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #15: at::native::to(at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, bool, c10::optional<c10::MemoryFormat>) + 0x13e (0x7f50d03aac4e in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #16: <unknown function> + 0x2802839 (0x7f50d1341839 in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #17: at::_ops::to_dtype_layout::call(at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, bool, c10::optional<c10::MemoryFormat>) + 0x215 (0x7f50d0ad5175 in /home/shkr/Documents/repos/rohtas/stable-diffusion-burn/target/release/build/torch-sys-b04b51a767e8ffae/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #18: <unknown function> + 0x25041d (0x5588859f341d in target/release/sample)\nframe #19: <unknown function> + 0x2540a5 (0x5588859f70a5 in target/release/sample)\nframe #20: <unknown function> + 0x23f955 (0x5588859e2955 in target/release/sample)\nframe #21: <unknown function> + 0xd20f1 (0x5588858750f1 in target/release/sample)\nframe #22: <unknown function> + 0xd9a21 (0x55888587ca21 in target/release/sample)\nframe #23: <unknown function> + 0xad408 (0x558885850408 in target/release/sample)\nframe #24: <unknown function> + 0xb3493 (0x558885856493 in target/release/sample)\nframe #25: <unknown function> + 0xa1ba9 (0x558885844ba9 in target/release/sample)\nframe #26: <unknown function> + 0x281cdb (0x558885a24cdb in target/release/sample)\nframe #27: <unknown function> + 0xade45 (0x558885850e45 in target/release/sample)\nframe #28: __libc_start_main + 0xf3 (0x7f50ce6d70b3 in /lib/x86_64-linux-gnu/libc.so.6)\nframe #29: <unknown function> + 0x61e5e (0x558885804e5e in target/release/sample)\n")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I might be missing some dependency, but I would need guidance to understand what it is.
OS: Ubuntu 22
CPU: Intel(R) Core(TM) i9-9820X CPU @ 3.30GHz
GPU: Nvidia GeForce RTX 3090

[shkr]

Might be related to this issue pytorch/pytorch#71402

[EnverOsmanov]

For me it was wrong "TORCH_CUDA_VERSION".
Also, I had to do "cargo clean" before running again with the correct version.

PS. There is a similar issue described here.