Skip to content

Fix tensor handling and comments for VIEW and PERMUTE ops#79

Merged
wine99 merged 4 commits intoravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/falback-handle-input
Mar 23, 2026
Merged

Fix tensor handling and comments for VIEW and PERMUTE ops#79
wine99 merged 4 commits intoravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/falback-handle-input

Conversation

@zhaixuejun1993
Copy link
Collaborator

This pull request improves support for split models and handling of VIEW and PERMUTE tensor operations in the OpenVINO backend of ggml. The changes ensure correct data reconstruction for non-contiguous tensors in split models, enhance error checking, and refine cache logic for model parameters.

Split model support and tensor reconstruction:

  • Added explicit strided-copy reconstruction for PERMUTE and VIEW tensors in split models within convert_ggml_input_to_ov, ensuring non-contiguous data is correctly copied into contiguous OpenVINO tensors. This is done by iterating over all dimensions and applying strides and offsets. (ggml/src/ggml-openvino/utils.cpp, ggml/src/ggml-openvino/utils.cppL614-R652)
  • Modified logic to only use the extra field for input tensors if the model is not split, preventing incorrect tensor handling in split models. (ggml/src/ggml-openvino/utils.cpp, ggml/src/ggml-openvino/utils.cppL601-R603)
  • Adjusted input shape determination for VIEW operations to only use view_src shape if the model is not split, maintaining correct behavior for both split and non-split models. (ggml/src/ggml-openvino/utils.cpp, ggml/src/ggml-openvino/utils.cppL614-R652)

VIEW operation handling and error checking:

  • Added a check to flag VIEW operations as unsupported if the number of elements differs from their source tensor, with a clear error message for unsupported cases. (ggml/src/ggml-openvino/ggml-openvino.cpp, ggml/src/ggml-openvino/ggml-openvino.cppR912-R920)
  • Updated compute_op_case to handle split models by setting the operation case to 0 for relevant input tensors, improving compatibility with split model input handling. (ggml/src/ggml-openvino/ggml-decoder.cpp, ggml/src/ggml-openvino/ggml-decoder.cppR224-R226)

Cache logic refinement:

@wine99
Copy link
Collaborator

wine99 commented Mar 20, 2026

since it does not effect normal execution. you can merge after this comment is addressed.

@wine99 wine99 merged commit dc7ff7f into ravi9:dev_backend_openvino Mar 23, 2026
34 of 47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants