Kcz/master/support for video in benchmark

[krzyczar]

Description

CVS-173846

Checklist:

• Tests have been updated or added to cover the new code.
• This patch fully addresses the ticket.
• I have made corresponding changes to the documentation.

[Copilot]

Pull Request Overview

This PR adds support for video input processing in the benchmark tool for visual language generation tasks. The main enhancement allows the tool to process video files in addition to existing image support, with configurable frame decimation/reduction capabilities.

Key changes:

• Added video processing functionality with OpenCV integration
• Implemented frame decimation logic to control the number of frames processed
• Extended prompt parsing to handle both image and video inputs

Reviewed Changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
tools/llm_bench/task/visual_language_generation.py	Updated to support video inputs by integrating new prompt extraction utilities and video tensor processing
tools/llm_bench/requirements.txt	Added opencv-python dependency for video processing
tools/llm_bench/llm_bench_utils/prompt_utils.py	Implemented video loading, frame decimation, and unified prompt extraction for images and videos
tools/llm_bench/llm_bench_utils/parse_json_data.py	Refactored JSON parsing with a shared base function and added video field support
tools/llm_bench/llm_bench_utils/model_utils.py	Extended parameter handling to support video inputs alongside media/images
tools/llm_bench/benchmark.py	Added command-line arguments for video input and frame control
tests/python_tests/samples/test_tools_llm_benchmark.py	Added tests for video prompt processing with various frame decimation scenarios
tests/python_tests/samples/conftest.py	Added fixture to download test video from GitHub repository

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sbalandi]

build_jenkins

[sbalandi]

looks like you don't need these imports now, please, remove

[sbalandi]

LGTM, let's remove commented text and PR will be ready to merge for me
@as-suvorov please, take a look

but, I face problem on generate() with qwen2-vl-7b-instruct, with master of GenAI and openvino dev build 11.24 . It's not a problem of that PR, I got it with another code too(also llava-next video works well), I just want to ask if it is known problem ?

generation_result = model.generate(prompts[0], generation_config=gen_config, **kwargs)
RuntimeError: Exception from src/inference/src/cpp/infer_request.cpp:75:
Exception from src/inference/src/cpp/infer_request.cpp:67:
Exception from src/inference/src/dev/isync_infer_request.cpp:228:
Failed to set tensor. Exception from src/inference/src/dev/make_tensor.cpp:161:
Can not access non-const pointer use e.g. 'static_cast<const ov::Tensor&>.data()'

[as-suvorov]

LGTM, let's remove commented text and PR will be ready to merge for me @as-suvorov please, take a look

but, I face problem on generate() with qwen2-vl-7b-instruct, with master of GenAI and openvino dev build 11.24 . It's not a problem of that PR, I got it with another code too(also llava-next video works well), I just want to ask if it is known problem ?

generation_result = model.generate(prompts[0], generation_config=gen_config, **kwargs)
RuntimeError: Exception from src/inference/src/cpp/infer_request.cpp:75:
Exception from src/inference/src/cpp/infer_request.cpp:67:
Exception from src/inference/src/dev/isync_infer_request.cpp:228:
Failed to set tensor. Exception from src/inference/src/dev/make_tensor.cpp:161:
Can not access non-const pointer use e.g. 'static_cast<const ov::Tensor&>.data()'

Probable fix: #3026. @krzyczar I'm not sure if your branch have this fix, could you please update to latest master

[as-suvorov]

LGTM, let's remove commented text and PR will be ready to merge for me @as-suvorov please, take a look
but, I face problem on generate() with qwen2-vl-7b-instruct, with master of GenAI and openvino dev build 11.24 . It's not a problem of that PR, I got it with another code too(also llava-next video works well), I just want to ask if it is known problem ?

generation_result = model.generate(prompts[0], generation_config=gen_config, **kwargs)
RuntimeError: Exception from src/inference/src/cpp/infer_request.cpp:75:
Exception from src/inference/src/cpp/infer_request.cpp:67:
Exception from src/inference/src/dev/isync_infer_request.cpp:228:
Failed to set tensor. Exception from src/inference/src/dev/make_tensor.cpp:161:
Can not access non-const pointer use e.g. 'static_cast<const ov::Tensor&>.data()'

Probable fix: #3026. @krzyczar I'm not sure if your branch have this fix, could you please update to latest master

Double checked @sbalandi message. It's not reproduced in this PR, so master update is not strongly required.
@sbalandi please ask @sgonorov he recenlty worked on a similar issue

[Copilot]

Pull request overview

Copilot reviewed 8 out of 9 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sbalandi]

build_jenkins