Any ideas for how to use multiprocessing with dpg.load_image?

[keck-in-space]

I have an application I am working on that deals with a high number of large images. I am trying to load them based on a specified timestamp by the user. It is currently working, but it is rather slow. Upon profiling the code, I saw that dpg.load_image was taking up the majority of the time. (about 0.1 seconds on average)

I already have this code running in a separate thread, but likely due to the GIL, I am having issues with slowness, affecting the framerate of DPG significantly. I have tried to use multiprocessing to parallelize this more efficiently, but the issue I ran into is that the mvBuffer generated by dpg.load_image is not pickleable. I was trying to find a way to share memory between processes, but I wasn't able to tell if that is possible in Windows. I see you can have shared ctypes, but I'm not sure what that means for an mvBuffer type.

Anyone out there have more experience and have some ideas? Thank you!

[bandit-masked]

I have not seen any prior discussions with good analysis. Could you do a benchmark with PIL and OpenCV for example?

There is this analysis for example.
https://learnopencv.com/efficient-image-loading/

Maybe manually managing the mutex might help a bit.

[keck-in-space]

Are there any examples on how to use PIL/OpenCV with the dpg.add_dynamic_texture function? I've tried a few permutations but I can't get anything to render.

[Pcothren]

so your loading images to use in the texture system or to do sort of image processing?

[keck-in-space]

Great question.

So, what I am doing is loading images based on a timestamp provided by the user.

What this looks like in my code is I create a dynamic or raw texture when the first timestamp is requested and then update that texture for subsequent requests via dpg.set_value("texture_tag", image_buffer). When I time the code, I can see that my call to dpg.load_image("image.png") is taking the majority of the time, while the dpg.set_value call is quite fast.

I have noticed that some images take much longer than the sample set I have been working with. A smaller in disk size, but larger in resolution JPEG takes longer than a lower resolution, higher disk size PNG for instance.

Here's the jpg image I used. I renamed it to nasa.jpg.

[bandit-masked]

Could DPG do this? Pygame can...

https://www.youtube.com/watch?v=VAWHfo2Zn_Q

[bandit-masked]

I created a feature request here.
#1793

[bandit-masked]

https://github.com/Pcothren/DearPyGui-Examples/blob/main/camera_capture_with_opencv.py

https://www.youtube.com/watch?v=-JZK8h-3bNk

This example uses Numpy for the conversion from OpenCV to DPG textures. There should be a built in conversion in OpenCV, but I don't know exactly how. Someone suggested using convertTo() and cvtColor(). Or maybe something like image = cv2.cvtColor(view , cv2.COLOR_BGR2RGB).

Hope this helps.

[keck-in-space]

This is very helpful, thank you so much!

[keck-in-space]

OK whoa. This basically solved my issues to use OpenCV with numpy to load the image and bypass dpg.load_image. I'm getting 60 FPS and the images are loading in super quickly. This is great! Thank you, @bandit-masked for the links and help!

[bandit-masked]

So that's interesting. OpenCV loads images much faster than DPG. There should be something to learn from OpenCV in that respect then...

[keck-in-space]

It seems to be at least twice as fast.

[bandit-masked]

We can't be having that, obviously ;-) DPG should at least match OpenCV's performance!

[keck-in-space]

I'm kind of surprised how much of a performance boost I got out of this... I did a little more profiling and found that the total time for both the cvload and the conversion is sometimes about the same as DPG's load call.

dpg.load_image time:          0.11977076530456543
cv2.imread w/conversion time: 0.10743284225463867

While other times it differs significantly

dpg.load_image time:           0.12674665451049805
cv2.imread w/conversions time: 0.06961202621459961

Is there some rendering penalty done by dpg.load_image? I went from about 15 FPS to 60 FPS, which seems like too big a leap for what amounts to a maximum performance improvement of 2x.

[mkotlarz]

Remember OpenCV's base image structure is a numpy array so there is great synergy here., you just need to remember the h,w and bgr vs rgb issues of opencv.

I am looking for a way to put the videocapture method outside of the rendering loop so it doesnt slow it down to 30fps.