Frames dropped using VideoWriter function with USB camera

[nschapp]

Hi there, I'm a researcher using Bonsai to synchronize video of animal behavior during recording of neural activity, a task where a tight temporal match between this data and video feed of their behavior is crucial. I'm having an issue with the VideoWriter function though, where the FPS it is set at (30) is not at all what the FPS in the resulting avi appears to be, despite the paired CSV writer appearing to record all 30 frames per second and the camera capturing at a normal 30 FPS when used outside of Bonsai. Additionally, switching the camera to an integrated webcam in the laptop for testing restoring normal FPS on the recorded avi, meaning this problem is somewhat specific to this usb camera.

There were a couple posts online about similar issues, but I'm not sure they were entirely worked out when I read into them. The usb camera is not especially high resolution to cause a problem there, but I'm not sure why it's dropping so many frames per second. In case it should matter for this, this particular instance of Bonsai I'm running on this setup is being done offline and hasn't been updated in over a year, though I should think that's unrelated to the original source of the problem.

Any clue how to fix the VideoWriter losing frames when it writes to the avi file?

[nschapp]

Follow-up from looking through the Github answers, because there were a lot here that didn't seem to be on previous searches for Bonsai problems I did before.

This thread (https://github.com/orgs/bonsai-rx/discussions/1737) seems to indicate it's an exposure problem, though the issue if that is true then becomes one of how to adjust the exposure settings for a USB camera in Bonsai. For something like a Basler camera with its own programming apparatus this is simple, but the camera I am using is an ELP usb 2.0 camera. Still trying to figure out if adjusting exposure to manual is something that can be easily configured either in Bonsai or in some device settings on the computer.

[nschapp]

Found an older thread that discusses how to modify exposure in Bonsai (https://groups.google.com/g/bonsai-users/c/y0kmWNRvS84/m/wqn7ozmbBgAJ?utm_medium=email&utm_source=footer), but this hasn't quite fixed it yet. The properties button indicated on this page doesn't seem to open anything on my end when clicked. There's another button on the CaptureProperties (Collection) in the Misc list below it, and while it allows setting exposure to manual or auto, setting to manual does NOT fix the problem, and in fact exacerbates it significantly. It seems as though a specific manual exposure value will be needed for a particular FPS, or just another solution entirely.

[bruno-f-cruz]

Your exposure should be smaller than 1/FPS. What is the value you are currently setting?

[nschapp]

Hmm, I'm not sure where to set that. I can open the cameraproperties window through the videocapture device node, but the "Value" set by the exposure setting I can change doesn't seem to make sense. Do you know what values it's supposed to take?

Here's a picture I took because this specific computer I'm using to run this is offline-only.

[bruno-f-cruz]

VideoCaptureDevice uses a directshow backend so not sure if this still applies, but with the OpenCV interface (which i suspect should follow the same standard), exposure is expressed in seconds = pow(2, X), a value of 0 would give you an exposure of 1second, which would cap your frame rate to above 1hertz.

See doc here

[nschapp]

I don't think that's right. I entered "5" for the value to see if I could get close to the value of 30 fps for this function since 2^4.907 ~= 30, and I couldn't put in a decimal value, but the output didn't change at all. I'm not sure if changing the value even did anything.