Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding FPS metadata to Video formats #105

Closed
wants to merge 14 commits into from

Conversation

keyaloding
Copy link

@keyaloding keyaloding commented Jul 12, 2024

Description

Right now, our video classes such as Video and MediaVideo can't store information about the fps of the video. Adding attributes to these classes that store this data will allow the user to set the frame rate, rather than setting it to a default value. I will write new tests and load videos with different frame rates in the GUI. I will also update the documentation, as necessary.

Types of changes

  • Bugfix
  • New feature
  • Refactor / Code style update (no logical changes)
  • Build / CI changes
  • Documentation Update
  • Other (explain)

Does this address any currently open issues?

#103

Outside contributors checklist

  • Review the guidelines for contributing to this repository
  • Read and sign the CLA and add yourself to the authors list
  • Make sure you are making a pull request against the develop branch (not main). Also you should start your branch off develop
  • Add tests that prove your fix is effective or that your feature works
  • Add necessary documentation (if appropriate)

Thank you for contributing to SLEAP!

❤️

Summary by CodeRabbit

  • New Features
    • Introduced frame rate calculation and retrieval for video files.
    • Enhanced video playback experience with accurate frame rate information.
  • Bug Fixes
    • Resolved issues related to frame rate handling in video imports.

Copy link
Contributor

coderabbitai bot commented Jul 12, 2024

Walkthrough

The recent updates enhance the VideoBackend class and its subclasses by introducing a _frame_rate attribute, computed using OpenCV for specific video formats. This allows users to easily access video frame rates via the new frame_rate property, improving functionality and metadata accessibility within the codebase.

Changes

File(s) Change Summary
sleap_io/io/video.py, sleap_io/model/video.py Added _frame_rate attribute to VideoBackend and subclasses. Updated from_filename method to calculate frame rate using OpenCV. Introduced frame_rate property for retrieving the video's frame rate.
tests/io/test_video_backends.py Added a new assertion in test_video_backend_from_filename to verify the frame rate of the video backend being tested.

Poem

In the realm of video streams so bright,
A frame rate feature now takes flight.
With OpenCV's insightful gaze,
It calculates in wondrous ways.
Code refined, efficiency in sight,
Our videos now dance in perfect light.

🐰📹✨

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Copy link

codecov bot commented Jul 12, 2024

Codecov Report

Attention: Patch coverage is 88.88889% with 1 line in your changes missing coverage. Please review.

Project coverage is 96.03%. Comparing base (5a30110) to head (c1c925c).
Report is 1 commits behind head on main.

Files Patch % Lines
sleap_io/model/video.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #105      +/-   ##
==========================================
- Coverage   96.04%   96.03%   -0.01%     
==========================================
  Files          17       17              
  Lines        2022     2044      +22     
==========================================
+ Hits         1942     1963      +21     
- Misses         80       81       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@keyaloding keyaloding marked this pull request as ready for review July 15, 2024 17:27
@keyaloding keyaloding self-assigned this Jul 15, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 95b9c81 and 5620a2b.

Files selected for processing (3)
  • sleap_io/io/video.py (9 hunks)
  • sleap_io/model/video.py (4 hunks)
  • tests/io/test_video_backends.py (1 hunks)
Additional comments not posted (8)
tests/io/test_video_backends.py (1)

17-17: LGTM! Assertion for frame_rate is correctly added.

The assertion verifies that the frame_rate attribute is correctly set to 15.0.

sleap_io/model/video.py (2)

65-65: LGTM! _frame_rate parameter is correctly added.

The _frame_rate parameter is correctly passed to the VideoBackend.from_filename method.

Also applies to: 83-83, 98-98


113-121: LGTM! frame_rate property is correctly added.

The frame_rate property correctly retrieves the frame rate from the backend.

sleap_io/io/video.py (5)

52-52: LGTM! _frame_rate attribute and frame_rate property are correctly added.

The _frame_rate attribute and frame_rate property are correctly implemented.

Also applies to: 58-58, 197-201


69-69: LGTM! _frame_rate parameter and frame rate calculation are correctly added.

The _frame_rate parameter is correctly passed, and the frame rate is correctly calculated using OpenCV.

Also applies to: 83-83, 88-92, 113-113, 122-122


327-327: LGTM! _frame_rate attribute is correctly added.

The _frame_rate attribute is correctly implemented.


483-483: LGTM! _frame_rate attribute is correctly added.

The _frame_rate attribute is correctly implemented.


725-725: LGTM! _frame_rate attribute is correctly added.

The _frame_rate attribute is correctly implemented.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 5620a2b and c40c814.

Files selected for processing (1)
  • sleap_io/io/video.py (8 hunks)
Additional context used
Ruff
sleap_io/io/video.py

89-90: Use a single if statement instead of nested if statements

(SIM102)

Additional comments not posted (5)
sleap_io/io/video.py (5)

52-52: Addition of _frame_rate attribute in class docstring

The addition of the _frame_rate attribute in the class docstring of VideoBackend is well-documented, providing clear information about its purpose.


58-58: Initialization of _frame_rate attribute

The _frame_rate attribute is properly initialized to None, which is appropriate for optional values. This allows for flexibility in cases where the frame rate might not be applicable or available.


83-83: Documentation for _frame_rate parameter

The documentation for the _frame_rate parameter in the from_filename method is clear and concise, explaining its purpose effectively.


122-122: Passing _frame_rate to HDF5Video constructor

The _frame_rate is correctly passed to the HDF5Video constructor, ensuring that the frame rate information is preserved across different video backends.


197-201: Property method for frame_rate

The frame_rate property method is correctly implemented to return the _frame_rate attribute. This provides a clean API for accessing the frame rate of a video.

sleap_io/io/video.py Outdated Show resolved Hide resolved
sleap_io/io/video.py Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between c40c814 and c1c925c.

Files selected for processing (1)
  • sleap_io/io/video.py (7 hunks)
Additional context used
Ruff
sleap_io/io/video.py

194-195: Use a single if statement instead of nested if statements

(SIM102)

Additional comments not posted (2)
sleap_io/io/video.py (2)

324-324: Addition of _frame_rate attribute to MediaVideo.

The addition of the _frame_rate attribute to the MediaVideo class is consistent with the PR objectives and aligns with the changes made to the VideoBackend class.


480-480: Addition of _frame_rate attribute to HDF5Video.

The addition of the _frame_rate attribute to the HDF5Video class is consistent with the PR objectives and aligns with the changes made to the VideoBackend class.

sleap_io/io/video.py Show resolved Hide resolved
sleap_io/io/video.py Outdated Show resolved Hide resolved
@keyaloding keyaloding requested a review from talmo July 15, 2024 21:34
@keyaloding
Copy link
Author

keyaloding commented Jul 15, 2024

I am getting the error ValueError: The number of original videos should equal the number of camera devices. in main.py and nwb.py and I'm not sure how to address it. Neither of these files uses the VideoBackend class, which is the only class that I modified, so I'm having trouble tracing this issue back to the changes that I made.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between c1c925c and c8e8ab0.

Files selected for processing (1)
  • sleap_io/io/video.py (6 hunks)
Files skipped from review as they are similar to previous changes (1)
  • sleap_io/io/video.py

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range, codebase verification and nitpick comments (1)
sleap_io/io/video.py (1)

58-58: Ensure _frame_rate is correctly handled in VideoBackend class.

The _frame_rate attribute is initialized to None and is returned by a method, but there are no other assignments or method calls involving _frame_rate found. This suggests that _frame_rate might not be properly set to a meaningful value before being used.

  • File: sleap_io/io/video.py
  • Lines:
    • _frame_rate initialization: sleap_io/io/video.py: _frame_rate = None
    • Method returning _frame_rate: sleap_io/io/video.py: return self._frame_rate

Please ensure _frame_rate is correctly set to a valid float value before being used in any operations.

Analysis chain

Ensure _frame_rate is correctly handled in VideoBackend class.

The _frame_rate attribute has been initialized to None. Ensure that it is correctly set and used in the class methods.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that `_frame_rate` is correctly set and used in the `VideoBackend` class.

# Test: Search for the `_frame_rate` usage. Expect: Correct handling in the class methods.
rg --type python '_frame_rate' -A 5

Length of output: 66


Script:

#!/bin/bash
# Description: Verify that `_frame_rate` is correctly set and used in the `VideoBackend` class.

# Test: Search for the `_frame_rate` usage. Expect: Correct handling in the class methods.
rg --type py '_frame_rate' -A 5

Length of output: 3490


Script:

#!/bin/bash
# Description: Verify that `_frame_rate` is correctly set and used in the `VideoBackend` class.

# Test: Search for assignments to `_frame_rate` and method calls involving `_frame_rate`.
rg --type py '_frame_rate\s*=' -A 5
rg --type py 'self\._frame_rate' -A 5

Length of output: 942

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between c8e8ab0 and 8a1ef98.

Files selected for processing (2)
  • sleap_io/io/video.py (6 hunks)
  • sleap_io/model/video.py (4 hunks)
Files skipped from review as they are similar to previous changes (1)
  • sleap_io/model/video.py
Additional comments not posted (5)
sleap_io/io/video.py (5)

52-52: Addition of _frame_rate attribute to VideoBackend class.

The addition of the _frame_rate attribute aligns with the PR objectives and enhances the functionality of the VideoBackend class by storing the frame rate of the video.


330-330: Addition of _frame_rate attribute to MediaVideo class.

The addition of the _frame_rate attribute in the docstring aligns with the PR objectives and enhances the functionality of the MediaVideo class by storing the frame rate of the video.


486-486: Addition of _frame_rate attribute to HDF5Video class.

The addition of the _frame_rate attribute in the docstring aligns with the PR objectives and enhances the functionality of the HDF5Video class by storing the frame rate of the video.


728-728: Addition of _frame_rate attribute to ImageVideo class.

The addition of the _frame_rate attribute aligns with the PR objectives and enhances the functionality of the ImageVideo class by storing the frame rate of the video.


195-203: Optimize conditional checks for file extension in frame_rate property.

The nested if statements checking for file extensions can be optimized into a single conditional expression. This simplifies the code and improves readability.

- if isinstance(self.filename, str) and any(
-     self.filename.endswith(ext) for ext in video_extensions
- ):
+ if isinstance(self.filename, str) and any(self.filename.endswith(ext) for ext in video_extensions):

Likely invalid or redundant comment.

@@ -77,6 +79,7 @@ def from_filename(
frames. If False, will close the reader after each call. If True (the
default), it will keep the reader open and cache it for subsequent calls
which may enhance the performance of reading multiple frames.
_frame_rate: Frame rate of the video.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update type hint for _frame_rate parameter in from_filename method.

The type hint for _frame_rate should be Optional[float] instead of Optional[bool].

- _frame_rate: Optional[bool] = None,
+ _frame_rate: Optional[float] = None,

Committable suggestion was skipped due to low confidence.

Comment on lines 198 to 202
video_extensions = ["mp4", "avi", "mov", "mj2", "mkv", "slp"]
if isinstance(self.filename, str) and any(
self.filename.endswith(ext) for ext in video_extensions
):
return cv2.VideoCapture(self.filename).get(cv2.CAP_PROP_FPS)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUBGROUP REVIEW: We don't need to check the video extension or filename here since this property of frame_rate would only ever be accessed if filename pointed to a VIdeo object anyways

def frame_rate(self) -> Optional[float]:
"""Frames per second of the video."""
video_extensions = ["mp4", "avi", "mov", "mj2", "mkv"]
if not any(self.filename.endswith(ext) for ext in video_extensions):
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This conditional is necessary because ImageVideo inherits from VideoBackend. If we try to check the frame rate of ImageVideo objects, which are stored as image files, we will get an error.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 8a1ef98 and 47fb60e.

Files selected for processing (1)
  • sleap_io/io/video.py (6 hunks)
Files skipped from review as they are similar to previous changes (1)
  • sleap_io/io/video.py

@keyaloding keyaloding closed this Aug 27, 2024
@keyaloding keyaloding deleted the kloding/add_video_fps_data branch August 27, 2024 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants