Added GIL release for time consuming methods. #1673
Conversation
github-actions
bot
added
category: text to image
| const std::string& neg_prompt, | ||
| bool do_classifier_free_guidance | ||
| ) { | ||
| py::gil_scoped_release rel; |
There was a problem hiding this comment.
can we use py::call_guard<py::gil_scoped_release>() for such cases to avoid explicit method implementation?
See https://pybind11.readthedocs.io/en/stable/advanced/misc.html#global-interpreter-lock-gil
There was a problem hiding this comment.
We do not make any python cast inside the function, so look like we can use mentioned by @ilya-lavrenov
| @@ -96,7 +96,7 @@ py::object call_vlm_generate( | |||
| ) { | |||
| auto updated_config = *pyutils::update_config_from_kwargs(generation_config, kwargs); | |||
| ov::genai::StreamerVariant streamer = pyutils::pystreamer_to_streamer(py_streamer); | |||
|
|
|||
| py::gil_scoped_release rel; | |||
There was a problem hiding this comment.
I'm not sure whether only pipe.generate(prompt, images, updated_config, streamer) should release GIL? what's about py::cast ?
CC @as-suvorov do you know?
There was a problem hiding this comment.
I'd guess this is fine. Even though py::cast() returns an object which can be interpreted as a python object, it only belongs to this function until it's returned and not visible to anyone.
Ticket: CVS-161082