Fix const casting in non-writable numpy-based tensors #3086
Conversation
sgonorov
requested review from
AlexanderKalistratov,
Wovchena,
sbalandi and
yatarkan
github-actions
bot
added
category: image generation
There was a problem hiding this comment.
Pull request overview
This PR fixes issues related to const casting in non-writable numpy-based tensors by creating writable copies of tensors before processing. The changes ensure that read-only numpy arrays (where writeable = False) can be safely used as inputs to various pipeline operations.
Key changes:
- Added defensive tensor copying in C++ code to handle read-only numpy tensors
- Added test coverage to verify read-only tensors work correctly across different pipeline types
- Modified tensor handling to use
copy_to()instead of direct assignment or pointer sharing
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/python_tests/test_vlm_pipeline.py | Added test case for VLM pipeline with read-only image tensors |
| tests/python_tests/test_llm_pipeline_static.py | Added test case for static LLM pipeline with read-only input tensors |
| tests/python_tests/test_llm_pipeline.py | Added test case for LLM pipeline with read-only input tensors |
| src/cpp/src/visual_language/phi4mm/classes.cpp | Creates writable copy of image tensor before processing |
| src/cpp/src/visual_language/inputs_embedder.cpp | Uses memcpy to create independent tensor copies instead of sharing pointers |
| src/cpp/src/speculative_decoding/speculative_decoding_stateful.cpp | Creates writable copies of input_ids and attention_mask tensors |
| src/cpp/src/llm/pipeline_static.cpp | Creates writable copies of input_ids and attention_mask tensors |
| src/cpp/src/image_generation/image_processor.cpp | Creates writable copy of image tensor before processing |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…33120) ### Details: - The tensor can be created as read-only view when input pointer is constant. This kind of tensor can be used as model inputs as model input should be read-only. There is issue to use `data()` if input tensor is not const object the acquired pointer is not constant which throw exception to indicate that there is intention to modify of read-only tensor. - Update core, and plugins to convert tensor object to const when get data pointer to call correct overload - Update tests to use read-only tensor for inference on different devices. ### Blocked PR: - openvinotoolkit/openvino.genai#3086 ### Tickets: - CVS-176653 --------- Signed-off-by: Pawel Raasz <[email protected]> Signed-off-by: Raasz, Pawel <[email protected]>
sgonorov
force-pushed
the
fix_const_casting
branch
from
a2554a2 to
b2df610
Compare
github-actions
bot
removed
category: image generation
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
|
|
||
| @parametrize_one_model_sdpa | ||
| def test_vlm_readonly_image_tensor(ov_pipe_model: VlmModelInfo, cat_image_32x32): |
There was a problem hiding this comment.
The test calls ov_pipe.generate() but does not verify the output or assert that generation succeeded. Consider adding an assertion to check that the result is not empty or meets expected criteria to ensure the read-only tensor is properly handled.
|
|
||
| @pytest.mark.parametrize("llm_model", MODELS_LIST, indirect=True) | ||
| @pytest.mark.parametrize("npu_config", PIPELINE_CONFIGS, indirect=True) | ||
| def test_readonly_input_tensor(npu_model: LLMPipeline): |
There was a problem hiding this comment.
The test calls npu_model.generate() twice but does not capture or validate the outputs. Add assertions to verify that generation completes successfully and produces expected results when using read-only tensors.
|
|
||
|
|
||
| @pytest.mark.parametrize("llm_model", ["katuni4ka/tiny-random-phi3"], indirect=True) | ||
| def test_readonly_input_tensor(ov_pipe: ov_genai.LLMPipeline) -> None: |
There was a problem hiding this comment.
The test calls ov_pipe.generate() twice but does not validate the outputs. Add assertions to confirm that generation succeeds and produces non-empty results when using read-only tensors.
87f9d9e
4 participants
Fix const casting in non-writable numpy-based tensors.